Databricks Platform Engineer

Platform Architecture & Engineering responsibilities

• Design and implement scalable Databricks Lakehouse platforms on AWS and/or Azure aligned to client requirements
• Architect end-to-end data platforms including ingestion, storage (Delta Lake), processing, and consumption layers
• Build and configure cloud infrastructure using infrastructure-as-code (e.g. Terraform & Declarative Automation Bundles (DAB's))
• Establish secure, compliant environments including networking (VNet/VPC, Private Link), identity (IAM/Entra ID), data governance (Unity Catalog), and access controls
• Define environment strategies (dev/test/prod), CI/CD pipelines, and release processes for Databricks deployments
• Implement monitoring, logging, cost optimisation, and performance tuning across the platform
• Design and implement data pipelines using Delta Live Tables, Auto Loader, and Databricks Workflows for both batch and streaming workloads <...