Databricks Spark Data Engineer
BeachHead
Build scalable data solutions as a Databricks Developer using Apache Spark and advanced data engineering practices with a top financial client. This six-month contract focuses on enterprise-grade pipelines.Design, develop, and optimise Databricks solutions across large-scale datasets and distributed computing environments. Build batch and streaming ETL with Auto Loader and Spark Structured Streaming. Apply Unity Catalog, Lakehouse Federation, SCD methodologies, and change data capture while collaborating with stakeholders on practical solution design.Key Responsibilities:
Design and optimise Databricks and Apache Spark data pipelinesBuild batch and streaming ETL using Auto LoaderMaintain catalogs, schemas, tables, views, functions, and volumesImplement SCD 1, SCD 2, and change data captureApply partitioning and liquid clustering for performanceRequirements:
Hands-on Databricks and Apache Spark development experienceExperience with Azure Databricks or Delta LakeStrong Unity Catalog and Lakehouse Federation experienceEnterprise experience;
banking experience preferredStrong communication and stakeholder management skillsApply Databricks, Spark, ETL, and governance expertise to deliver reliable enterprise data solutions for financial services.
#J-18808-Ljbffr
Design and optimise Databricks and Apache Spark data pipelinesBuild batch and streaming ETL using Auto LoaderMaintain catalogs, schemas, tables, views, functions, and volumesImplement SCD 1, SCD 2, and change data captureApply partitioning and liquid clustering for performanceRequirements:
Hands-on Databricks and Apache Spark development experienceExperience with Azure Databricks or Delta LakeStrong Unity Catalog and Lakehouse Federation experienceEnterprise experience;
banking experience preferredStrong communication and stakeholder management skillsApply Databricks, Spark, ETL, and governance expertise to deliver reliable enterprise data solutions for financial services.
#J-18808-Ljbffr
Vacancy posted more than 2 months ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Databricks Spark Data Engineer. Be the first to apply!
Related searches
- data integration developer Winnipeg, MB
- data cabling engineer Winnipeg, MB
- python data engineer Winnipeg, MB
- remote data engineer Winnipeg, MB
- junior data engineer Winnipeg, MB
- data centre engineer Winnipeg, MB
- gcp data engineer Winnipeg, MB
- data engineer Winnipeg, MB
- big data engineer Winnipeg, MB
- data engineering intern summer Winnipeg, MB
