Senior Databricks Engineer
KData AI
We are seeking an experienced
Senior Databricks Engineer
with a deep background in distributed data systems to architect, scale, and optimize enterprise-grade lakehouse solutions. In this role, you will take technical ownership of end-to-end batch and streaming pipelines, implement robust governance and security controls via Unity Catalog, drive platform-wide performance tuning and FinOps initiatives, and mentor engineering talent across the data organization.
Key Responsibilities
We are seeking an experienced
Senior Databricks Engineer
with a deep background in distributed data systems to architect, scale, and optimize enterprise-grade lakehouse solutions. In this role, you will take technical ownership of end-to-end batch and streaming pipelines, implement robust governance and security controls via Unity Catalog, drive platform-wide performance tuning and FinOps initiatives, and mentor engineering talent across the data organization.
Key Responsibilities
Lakehouse Architecture & Modeling:
Spearhead the end-to-end technical design and implementation of highly scalable lakehouse pipelines adhering to the
Medallion Architecture (Bronze, Silver, Gold), dimensional modeling principles, and robust Change Data Capture (CDC) patterns.
Pipeline Development & Orchestration:
Build high-throughput, fault-tolerant batch and real-time streaming ingestion pipelines using
PySpark, Spark SQL, Delta Lake, Auto Loader, and Databricks Workflows / Lakeflow Jobs .
Performance Tuning & FinOps Leadership:
Profile workloads, analyze execution plans, and optimize compute configurations to drive down runtimes and minimize Databricks Unit (DBU) consumption using liquid clustering, Photon, serverless compute, and automated table maintenance (OPTIMIZE, VACUUM).
Enterprise Governance & Security:
Architect and enforce fine-grained access control policies (row/column-level security, data masking), data lineage tracking, and auditing using
Databricks Unity Catalog
across multi-workspace and multi-region environments.
DevOps, IaC & CI/CD:
Establish automated deployment pipelines, testing frameworks, and code promotion standards using
Databricks Asset Bundles (DABs), Terraform, and CI/CD tools (GitHub Actions, Azure DevOps, or GitLab CI).
Technical Mentorship & Cross-Functional Alignment:
Serve as the subject matter expert on the Databricks ecosystem, establishing code quality benchmarks, leading architecture reviews, and mentoring mid-to-senior engineers while partnering closely with Analytics and Machine Learning teams.
Requirements
Required Qualifications
Experience:
8+ years
of hands-on software and data engineering experience, with at least
5+ years
of dedicated, deep technical experience architecting and building production-grade solutions on the
Databricks platform .
Core Languages:
Advanced, expert-level proficiency in
Python
(PySpark, Spark Core) and
Advanced SQL
(complex analytical queries, window functions, and physical query execution plan tuning).
Lakehouse Internals:
Deep operational mastery of
Delta Lake internals
(ACID guarantees, transaction logs, data skipping, Z-ordering/liquid clustering, compaction, and schema evolution) along with Structured Streaming.
Cloud Architecture & Networking:
Substantial production experience across cloud storage and networking architectures ( AWS, Azure, or GCP ), including private link, VPC peering/vNet injection, IAM roles, and secure storage access patterns (e.G., ADLS Gen2, AWS S3).
Governance & Access Management:
Proven track record designing and managing
Databricks Unity Catalog
at scale (catalogs, schemas, volumes, external locations, privilege inheritance, and metastore integrations).
Software Engineering Rigor:
Strong background in enterprise-level version control workflows (Git), automated unit/integration testing (TDD), modular software design, and automated CI/CD practices.
Preferred Qualifications
Certifications:
Databricks Certified Data Engineer Professional or Databricks Certified Machine Learning Professional.
Infrastructure as Code (IaC):
Production experience provisioning workspaces, clusters, and access permissions using
Terraform .
Modern Data Stack:
Practical experience with dbt (dbt-databricks), Apache Kafka, Azure Event Hubs, Delta Live Tables (DLT), or MLflow integration.
BI & Consumption Layer:
Hands-on experience optimizing Databricks Serverless SQL Warehouses for downstream consumption tools such as Power BI, Tableau, or Looker.
#J-18808-Ljbffr
Senior Databricks Engineer
with a deep background in distributed data systems to architect, scale, and optimize enterprise-grade lakehouse solutions. In this role, you will take technical ownership of end-to-end batch and streaming pipelines, implement robust governance and security controls via Unity Catalog, drive platform-wide performance tuning and FinOps initiatives, and mentor engineering talent across the data organization.
Key Responsibilities
We are seeking an experienced
Senior Databricks Engineer
with a deep background in distributed data systems to architect, scale, and optimize enterprise-grade lakehouse solutions. In this role, you will take technical ownership of end-to-end batch and streaming pipelines, implement robust governance and security controls via Unity Catalog, drive platform-wide performance tuning and FinOps initiatives, and mentor engineering talent across the data organization.
Key Responsibilities
Lakehouse Architecture & Modeling:
Spearhead the end-to-end technical design and implementation of highly scalable lakehouse pipelines adhering to the
Medallion Architecture (Bronze, Silver, Gold), dimensional modeling principles, and robust Change Data Capture (CDC) patterns.
Pipeline Development & Orchestration:
Build high-throughput, fault-tolerant batch and real-time streaming ingestion pipelines using
PySpark, Spark SQL, Delta Lake, Auto Loader, and Databricks Workflows / Lakeflow Jobs .
Performance Tuning & FinOps Leadership:
Profile workloads, analyze execution plans, and optimize compute configurations to drive down runtimes and minimize Databricks Unit (DBU) consumption using liquid clustering, Photon, serverless compute, and automated table maintenance (OPTIMIZE, VACUUM).
Enterprise Governance & Security:
Architect and enforce fine-grained access control policies (row/column-level security, data masking), data lineage tracking, and auditing using
Databricks Unity Catalog
across multi-workspace and multi-region environments.
DevOps, IaC & CI/CD:
Establish automated deployment pipelines, testing frameworks, and code promotion standards using
Databricks Asset Bundles (DABs), Terraform, and CI/CD tools (GitHub Actions, Azure DevOps, or GitLab CI).
Technical Mentorship & Cross-Functional Alignment:
Serve as the subject matter expert on the Databricks ecosystem, establishing code quality benchmarks, leading architecture reviews, and mentoring mid-to-senior engineers while partnering closely with Analytics and Machine Learning teams.
Requirements
Required Qualifications
Experience:
8+ years
of hands-on software and data engineering experience, with at least
5+ years
of dedicated, deep technical experience architecting and building production-grade solutions on the
Databricks platform .
Core Languages:
Advanced, expert-level proficiency in
Python
(PySpark, Spark Core) and
Advanced SQL
(complex analytical queries, window functions, and physical query execution plan tuning).
Lakehouse Internals:
Deep operational mastery of
Delta Lake internals
(ACID guarantees, transaction logs, data skipping, Z-ordering/liquid clustering, compaction, and schema evolution) along with Structured Streaming.
Cloud Architecture & Networking:
Substantial production experience across cloud storage and networking architectures ( AWS, Azure, or GCP ), including private link, VPC peering/vNet injection, IAM roles, and secure storage access patterns (e.G., ADLS Gen2, AWS S3).
Governance & Access Management:
Proven track record designing and managing
Databricks Unity Catalog
at scale (catalogs, schemas, volumes, external locations, privilege inheritance, and metastore integrations).
Software Engineering Rigor:
Strong background in enterprise-level version control workflows (Git), automated unit/integration testing (TDD), modular software design, and automated CI/CD practices.
Preferred Qualifications
Certifications:
Databricks Certified Data Engineer Professional or Databricks Certified Machine Learning Professional.
Infrastructure as Code (IaC):
Production experience provisioning workspaces, clusters, and access permissions using
Terraform .
Modern Data Stack:
Practical experience with dbt (dbt-databricks), Apache Kafka, Azure Event Hubs, Delta Live Tables (DLT), or MLflow integration.
BI & Consumption Layer:
Hands-on experience optimizing Databricks Serverless SQL Warehouses for downstream consumption tools such as Power BI, Tableau, or Looker.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Senior Databricks Engineer in Winnipeg, MB vacancy
- Senior Databricks Data Engineer Location: Toronto (Hybrid - minimum 2 days per week onsite at the client's downtown Toronto office) Contract Length: 2 Months Key ResponsibilitiesDatabricks Engineering and Migration Migrate and refactor existing ETL/ELT pipelines,...DatabricksSeniorContract workWork at office2 days per week
- Montreal-based Senior Databricks Platform Engineer specializing in Lakehouse automation, Unity Catalog governance, and reliable remote platform operations.This contract role owns enterprise Databricks environments across development, test, and production. You will build reusable...DatabricksSeniorContract workRemote work
- Lead Databricks migrations with Python, PySpark, and SQL from a hybrid Toronto engagement. Modernize production pipelines for two-week Agile delivery.This senior data engineering contract spans two months and focuses on migrating existing ETL/ELT workloads into Databricks....DatabricksSeniorContract work
- Databricks expertise meets customer intelligence at Xplore Inc. in this hands-on technical lead role, focused on gold-layer data and AI-ready analytics foundations.As Senior Intelligence Analytics Engineer, you will design curated datasets, validate pipelines, and build production...DatabricksSenior
- ...Fluent, LLC in Canada is seeking a Senior MLOps Engineer to co-own MLOps practices, establishing standards for deployment, serving, scaling and cost-efficient infrastructure. You will work with ML scientists, Data Engineering and Product to ensure reliable production systems...DatabricksSenior
- Lead enterprise data platforms with Databricks and Apache Spark at ShyftLabs, working three days weekly from its downtown Toronto... ...office. Shape cloud solutions for Fortune 500 clients.As a Senior / Lead Data Engineer, you will own delivery from discovery through production deployment...DatabricksSeniorWork at office
- Lead Databricks data platform architecture at ShyftLabs from our downtown Toronto office three days weekly, using Spark and Python.As a Senior / Lead Data Engineer, you will guide enterprise-scale platforms for Fortune 500 clients. Apply Databricks, Apache Spark, Python, SQL...DatabricksSeniorWork at office
$84k - $128k per year
Senior Data Engineer (Databricks Focused)This site uses cookies to provide you with a more responsive and personalised service. By using this site you agree to our use of cookies. Please read our privacy statement for more information on the cookies we use and how to delete...DatabricksSeniorFull timeManual laborWork at officeLocal areaFlexible hours- Lead enterprise Data and AI engineering with Databricks and AWS in Vanguard’s hybrid working model, building scalable platforms for analytics.Vanguard seeks an experienced Senior Lead Data and AI Engineer with 9–12+ years in Data Engineering and modern cloud data platforms....DatabricksSenior
- Build advertising data infrastructure remotely across Canada with Databricks, Apache Spark, and AI-native engineering practices. Own pipelines powering campaign analytics.This Senior Data Engineer role focuses on advertising analytics, campaign performance, and business intelligence...DatabricksSeniorRemote work
- Lead Azure Databricks platform engineering at Canada Life in a hybrid Toronto or London role, using cloud administration and operational support expertise.As a Senior Data Platform Engineer, you will design, adapt, and maintain cloud solutions supporting Canada Life business...DatabricksSenior
$130k - $135k per year
...Cloud & DevOps, Data, and Software Engineering, servicing an array of noteworthy... ...Engineer with strong expertise in Databricks to join our data team. The ideal candidate... ...characteristic protected by law.Seniority level Seniority level Mid-Senior levelEmployment type Employment...DatabricksSeniorLong term contractFull timeContract workFlexible hours- We are seeking a highly skilled Senior DevOps Engineer with extensive experience in Azure, Terraform, Databricks, and DevOps practices.This role involves designing, implementing, and managing scalable cloud infrastructure, automating deployments, and ensuring compliance with...DatabricksSenior
- The Databricks Engineer will design, develop, and optimize large-scale data pipelines using Databricks and Azure cloud services. This role requires advanced expertise in Spark/PySpark, Delta Lake technologies, data governance, and cloud-based data engineering. The engineer...Databricks
- A progressive financial services company in Canada is seeking a Senior Data Engineer. You will set standards for data platforms and act as a technical anchor for scalable systems. Ideal candidates have 5+ years of experience along with advanced skills in SQL and Python. You...DatabricksSenior
- NTT DATA seeks a Senior Data Engineer to join our Toronto, Ontario team. The role focuses on designing, building, and optimizing scalable data... ...workflows. Ideal candidates will have 8+ years with Databricks, PySpark, Python and SQL, plus strong Azure data services experience...DatabricksSenior
- ...1984, we started out as a team of three engineers. Today, we have grown to become a global... ...-tenant, multi-cloud data platform on Databricks (GCP & Azure) to connect, govern, secure... ...about and act on enterprise data. As a Senior AI Software Engineer on the Data Fabric...DatabricksSeniorLong term contractWork at officeRemote workFlexible hours
- Ledcor Group is hiring a Senior Data Engineer to join the Data & Analytics Platform team. You will design, build, and maintain enterprise-scale... ...strong Python, PySpark, and SQL skills, experience with Databricks and cloud platforms, and a solid background in data #J-18808...DatabricksSenior
- ...Ledcor is seeking a Senior Data Engineer to design, build, and maintain enterprise-scale data solutions across construction and infrastructure businesses. You’ll work on a Databricks-based platform with medallion architecture, partnering with analysts, developers, and stakeholders...DatabricksSenior
- Location: Montreal, QC (REMOTE) Type: Contract Role Overview As a Senior Databricks Data Platform Engineer you will own the engineering automation and day to day health of our enterprise Databricks Lakehouse platform This is a platform-first role rather than building...DatabricksSeniorContract workRemote work
- A leading data solutions company is seeking a Senior Data Engineer to join their fully remote team. In this role, you will design and build scalable ETL/ELT pipelines using Databricks on AWS, ensuring data quality and collaborating closely with cross-functional teams. The ideal...DatabricksSeniorRemote workFlexible hours
$75 - $95 per hour
Databricks DeveloperLocation: Toronto, ON Onsite Flexibility: Hybrid Contract Details Position Type: Contract... ...-- Job SummaryOur client is seeking an experienced Senior Azure Databricks Data Engineer to design, build and support scalable enterprise data...DatabricksSeniorContract workFor contractors- ## Job Description# Databricks Data EngineerApply**Job#: 3057559****Job Description:****Databricks Data Engineer****Overview:**Apex Systems is seeking a Databricks Data Engineer in Toronto, ON to support a large-scale data migration and centralization initiative into an enterprise...DatabricksHourly payContract workManual labor
- Lead a talented engineering team at Databricks, focused on building innovative software solutions in a SaaS environment. This role emphasizes recruitment, talent development, and process efficiency. As an Engineering Manager, you will oversee a full stack engineering team...DatabricksLong term contract
- Job Title: Senior Databricks Data Platform Engineer Location: Canada, Remote Seniority on the skill/s required on this requirement: Senior Level Estimated Duration: 1 year contract with possible extension Work authorization: Candidate must be authorized to work in...DatabricksSeniorContract workRemote work
- Optimize Databricks costs and automate delivery remotely in Canada using FinOps, Terraform, CI/CD, and platform administration skills.Mindtris is seeking a Senior Databricks Data Platform Engineer for a one-year contract with possible extension. The role emphasizes platform...DatabricksSeniorContract workRemote work
- ...cloud platforms, including AWS, Azure and Databricks, we deliver strategic solutions that... ...emerging technologies. We are seeking a Senior Databricks Architect to join our growing... ...for solution delivery, including guiding engineers and ensuring best practices. Define and implement...DatabricksSeniorRemote workFlexible hours
- TekStaff's Client has a current vacancy for a Site Reliability Engineer ( Databricks) This is a 12 month contract located in Toronto, ON/ Hybrid... ...etc) and Agile principles Readiness and motivation (as senior or lead developer and valued subject matter expert) to...DatabricksSeniorContract work
- Skan AI in Canada seeks a Senior Database Administrator to own the health, performance, and reliability of... ...environments across PostgreSQL, MongoDB, and Databricks. This highly technical role focuses on platform engineering, monitoring, and cost-efficient operation. The ideal...DatabricksSenior
- ...Coconut Software is seeking a Senior Data Engineer for a 12-month contract in Canada. You will own data engineering on the Data Enablement team, building production pipelines in Databricks and migrating analytics to a lakehouse architecture. You will collaborate with analytics...DatabricksSeniorContract workRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Databricks Engineer. Be the first to apply!
Related searches
- ingénieur d'usine Winnipeg, MB
- vibration engineer Winnipeg, MB
- evaluation engineer Winnipeg, MB
- architectural engineer Winnipeg, MB
- ingénieur génie rural Winnipeg, MB
- contact center engineer Winnipeg, MB
- stage ingenieur Winnipeg, MB
- antenna engineer Winnipeg, MB
- completions engineer Winnipeg, MB
- junior aws engineer Winnipeg, MB

