Databricks Spark Data Engineer

Databricks and Apache Spark specialists can build scalable data solutions for a six-month contract supporting a top financial client. This hands-on role focuses on enterprise-grade pipelines and large-scale datasets.The Databricks Developer designs, develops, and optimises data pipelines across distributed computing environments. You will use Auto Loader, Spark Structured Streaming, Unity Catalog, and Lakehouse Federation. The work includes solution design, stakeholder collaboration, and practical technical recommendations. Banking or financial services experience is highly preferred.Key Responsibilities:
Design and optimise Databricks and Apache Spark pipelinesBuild batch, Auto Loader, and streaming ETL pipelinesMaintain catalogs, schemas, tables, views, functions, and volumesImplement SCD 1, SCD 2, and change data captureImprove performance through partitioning and liquid clusteringRequirements:
Strong hands-on Databricks and Apache Spark experienceExperience with Azure Databricks or Delta LakeStrong Unity Catalog and Lakehouse Federation experienceEnterprise experience;
banking experience is preferredStrong communication and stakeholder management skillsApply Databricks, Spark, streaming ETL, and governance expertise to enterprise data engineering for a financial client.
#J-18808-Ljbffr