Lead Data Engineer, Databricks

TekRek has partners with a global financial services business is modernizing its enterprise data platform to support complex reporting, analytics, governance, and operational workflows across the organization.

The technical environment is centered on Databricks and AWS, with a strong focus on scalable data processing, secure access controls, real-time pipelines, and modern Lakehouse architecture.

This is a hands‑on technical leadership role for a Senior Data Engineer or Lead Data Engineer who can lead a team while staying close to architecture, delivery, and platform performance. You will guide engineering standards, build Databricks‑based solutions, and help mature a centralized enterprise data platform with a strong focus on scalability, security, and reliability.

What You Will Do

• Lead and mentor data engineers including code reviews, design reviews, and technical knowledge sharing
• Design and build Databricks Lakehouse solutions for enterprise‑scale data processing and analytics
• Develop, optimize, and monitor ETL and ELT pipelines for performance, reliability, and data quality
• Build structured streaming pipelines and use Delta Live Tables for ingestion and transformation workflows
• Apply Unity Catalog, IAM practices, Terraform, and Spark optimization techniques to improve governance, access control, and platform efficiency

What You Bring

• 10+ years of data engineering experience, including several years working hands‑on with Databricks
• Proven experience leading technical teams while remaining active in engineering delivery
• Strong Python, Spark, SQL, data modeling, and pipeline optimization skills
• Experience with AWS or similar cloud platforms, with exposure to services such as S3, Glue, or Lambda
• Working knowledge of Delta Lake, Lakehouse architecture, Git, ETL/ELT patterns, and AI‑assisted development practices

Why This Role

This is a strong fit for a Databricks‑focused data engineering lead who wants meaningful platform ownership inside a complex enterprise environment. The role offers a mix of team leadership, hands‑on architecture, cloud data engineering, governance, streaming, and performance work without being removed from the technical details.