Principal Data Engineer - Databricks
Astra-North Infoteck Inc.
Principal Data Engineer – Databricks | Spark | Delta Lake | PySpark | Data Lakehouse | AWS/AzureJob DescriptionLocation: Toronto
Work Model: Onsite (4 days/week)
Key Requirements
12–18 years of overall Data Engineering experience.
8+ years of experience with Enterprise Data Warehouse and Data Lake platforms.
5+ years of hands‑on experience with Databricks and Apache Spark at scale.
Strong experience modernizing legacy Cloudera platforms (CDH/CDP, Hive, HBase, Impala, Spark) to Databricks Lakehouse.
Experience redesigning ingestion, transformation, and consumption patterns from HDFS-based architecture to cloud object storage and Delta Lake.
Experience refactoring legacy Hive/Impala logic into PySpark and Spark SQL ELT pipelines.
Experience ensuring data reconciliation, audit integrity, and consistency during migration.
Experience designing and governing Enterprise Data Warehouse and Data Lake/Lakehouse architectures.
Experience implementing layered architectures including:
Curated/Conformed Layer
Experience modernizing traditional Enterprise Data Warehouse platforms into scalable Lakehouse architectures.
Strong experience with finance and risk data models, including:
Financial Hierarchies
Credit Risk Models
Liquidity Risk Models
Experience enabling reporting use cases including aggregation, drill-down, and drill-back capabilities.
Experience building and managing semantic/consumption layers for BI, reporting, and analytics.
Ability to define business metrics, dimensions, hierarchies, and KPIs.
Experience with Databricks SQL, Delta Tables, and dbt or similar frameworks.
Strong experience developing and optimizing large-scale data pipelines using:
Experience optimizing workloads using Z-ORDER, OPTIMIZE, caching, and cluster configurations.
Experience implementing data governance, data quality frameworks, reconciliation controls, and exception handling.
Experience establishing data lineage and metadata management.
Knowledge of data security, access control, and compliance standards.
Experience with cloud platforms such as AWS or Azure.
Experience with CI/CD pipelines using:
Familiarity with orchestration tools such as:
Experience with dbt is a plus.
Ability to act as a technical authority and lead architecture decisions.
Experience mentoring senior engineers and establishing engineering standards.
Strong stakeholder management skills with finance, risk, analytics, and governance teams.
Ability to translate complex data structures into business-ready insights.
Nice to Have
Experience in Banking, Financial Services, Insurance (BFSI), Capital Markets, or regulatory reporting.
Oracle Financials
Experience supporting AI/ML workloads.
Databricks or cloud certifications.
Key Responsibilities
Lead Cloudera to Databricks transformation initiatives.
Design and implement enterprise Data Lakehouse and Data Warehouse solutions.
Build scalable, high-performance data pipelines and modern data architectures.
Drive data modernization, governance, quality, and security initiatives.
Support regulatory, management, and analytical reporting platforms.
Provide technical leadership, mentor engineering teams, and establish best practices.
#J-18808-Ljbffr
Work Model: Onsite (4 days/week)
Key Requirements
12–18 years of overall Data Engineering experience.
8+ years of experience with Enterprise Data Warehouse and Data Lake platforms.
5+ years of hands‑on experience with Databricks and Apache Spark at scale.
Strong experience modernizing legacy Cloudera platforms (CDH/CDP, Hive, HBase, Impala, Spark) to Databricks Lakehouse.
Experience redesigning ingestion, transformation, and consumption patterns from HDFS-based architecture to cloud object storage and Delta Lake.
Experience refactoring legacy Hive/Impala logic into PySpark and Spark SQL ELT pipelines.
Experience ensuring data reconciliation, audit integrity, and consistency during migration.
Experience designing and governing Enterprise Data Warehouse and Data Lake/Lakehouse architectures.
Experience implementing layered architectures including:
Curated/Conformed Layer
Experience modernizing traditional Enterprise Data Warehouse platforms into scalable Lakehouse architectures.
Strong experience with finance and risk data models, including:
Financial Hierarchies
Credit Risk Models
Liquidity Risk Models
Experience enabling reporting use cases including aggregation, drill-down, and drill-back capabilities.
Experience building and managing semantic/consumption layers for BI, reporting, and analytics.
Ability to define business metrics, dimensions, hierarchies, and KPIs.
Experience with Databricks SQL, Delta Tables, and dbt or similar frameworks.
Strong experience developing and optimizing large-scale data pipelines using:
Experience optimizing workloads using Z-ORDER, OPTIMIZE, caching, and cluster configurations.
Experience implementing data governance, data quality frameworks, reconciliation controls, and exception handling.
Experience establishing data lineage and metadata management.
Knowledge of data security, access control, and compliance standards.
Experience with cloud platforms such as AWS or Azure.
Experience with CI/CD pipelines using:
Familiarity with orchestration tools such as:
Experience with dbt is a plus.
Ability to act as a technical authority and lead architecture decisions.
Experience mentoring senior engineers and establishing engineering standards.
Strong stakeholder management skills with finance, risk, analytics, and governance teams.
Ability to translate complex data structures into business-ready insights.
Nice to Have
Experience in Banking, Financial Services, Insurance (BFSI), Capital Markets, or regulatory reporting.
Oracle Financials
Experience supporting AI/ML workloads.
Databricks or cloud certifications.
Key Responsibilities
Lead Cloudera to Databricks transformation initiatives.
Design and implement enterprise Data Lakehouse and Data Warehouse solutions.
Build scalable, high-performance data pipelines and modern data architectures.
Drive data modernization, governance, quality, and security initiatives.
Support regulatory, management, and analytical reporting platforms.
Provide technical leadership, mentor engineering teams, and establish best practices.
#J-18808-Ljbffr
Vacancy posted 57 minutes ago
Similar jobs that could be interesting for youBased on the Principal Data Engineer - Databricks in Winnipeg, MB vacancy
- ~12–18 years of overall Data Engineering experience ~8+ years of experience with enterprise Data Warehouse and Data Lake platforms ~5+ years of hands‑on experience with Databricks and Apache Spark at scale ~ Strong experience modernizing legacy Cloudera platforms including...Databricks
- Principal Data Engineer work in Toronto centers on Databricks, Apache Spark, and Lakehouse modernization. This onsite role requires four days weekly.This principal-level position leads enterprise data engineering initiatives across Databricks, Spark, Delta Lake, PySpark, and...DatabricksWork at office
- ...Hybrid – 3 days remoteHourly Range : $90 – $95Job Overview As a Data Engineer, you will be responsible for designing, building and... ...Azure Data Warehouse, Azure Synapse Analytics Services, Azure Databricks, Microsoft Purview, and Power BIWork within the agile SCRUM work...DatabricksLong term contract
- ## Job Description# Databricks Data EngineerApply**Job#: 3057559****Job Description:****Databricks Data Engineer****Overview:**Apex Systems is seeking a Databricks Data Engineer in Toronto, ON to support a large-scale data migration and centralization initiative into an enterprise...DatabricksHourly payContract workManual labor
- Enhance your data engineering skills as a Databricks Engineer focused on Azure cloud solutions. Utilize your extensive Spark experience to optimize and deliver large-scale data pipelines. This role demands over 10 years of expertise in data engineering, particularly within...Databricks
- Senior Databricks Data Engineer Location: Toronto (Hybrid - minimum 2 days per week onsite at the client's downtown Toronto office) Contract Length: 2 Months Key ResponsibilitiesDatabricks Engineering and Migration Migrate and refactor existing ETL/ELT pipelines,...DatabricksContract workWork at office2 days per week
- Data Engineer - Databricks (Remote)We work with global enterprises to design and build data platforms that power digital products used by millions of users. Our projects involve large-scale datasets, complex distributed pipelines, and real business impact. We’re looking for...DatabricksFull timeRemote workMonday to fridayFlexible hours
- Own Databricks infrastructure, Terraform deployments, and streaming reliability at Homebase, with required Toronto hub attendance... .... Build the platform beneath AI-powered products.As a Data and ML Platform Engineer, you will control how data enters, runs through, and returns...DatabricksWork from home
- Build scalable data integration solutions with Databricks and PySpark at Teqfocus, using SQL and Python to process enterprise data. Work hands-on across... ...master data ingestion workflows.Teqfocus seeks a Data Engineer with five or more years of professional experience in data...Databricks
- Parkland Corporation is expanding its Enterprise BI team with a BI Developer focused on data engineering in an Azure Databricks environment. You will design, build, test, and support trusted data products and pipelines, leveraging Delta Lake, Unity Catalog, and related Databricks...Databricks
- Build scalable data solutions as a Databricks Developer using Apache Spark and advanced data engineering practices with a top financial client. This six-month contract focuses on enterprise-grade pipelines.Design, develop, and optimise Databricks solutions across large-scale...DatabricksContract work
- Python, PySpark, SQL, Scala et Databricks structurent ce poste de Lead Technique Data chez Quality Foundations, Ubisoft.Vous contribuerez aux plateformes de données modernes, aux pipelines et aux architectures analytiques. La mission comprend l’adoption de solutions configurables...Databricks
- Databricks and Apache Spark specialists can build scalable data solutions for a six-month contract supporting a top financial client. This hands-on role focuses on... ...ETL, and governance expertise to enterprise data engineering for a financial client. #J-18808-LjbffrDatabricksContract work
- Job Title: Senior Databricks Data Platform Engineer Location: Canada, Remote Seniority on the skill/s required on this requirement: Senior Level Estimated Duration: 1 year contract with possible extension Work authorization: Candidate must be authorized to work...DatabricksContract workRemote work
- Elevate your data engineering career at CPUS Engineering as a Data Engineer focused on Databricks. Enjoy a hybrid work model while designing and implementing data-driven applications. This role is essential for creating, supporting, and optimizing data lake and warehouse...Databricks
- We are seeking a hands-on Data Engineer with 5+ years of experience to design and deliver scalable data integration solutions. The ideal candidate has strong expertise in Databricks, PySpark, SQL and Python, with proven experience transforming data into complex JSON structures...DatabricksFor subcontractor
- Build Python and PySpark data pipelines at Electric Mind, supporting a financial services... ...or by contract. You will maintain Databricks pipelines using SQL, medallion architecture... ...data models. Working alongside a senior engineer, you will write clean transform code, resolve...DatabricksContract work
$105k - $120k per year
...yr - CA$120,000.00/yrThe Opportunity We are looking for a Principal Data Engineer to join our Data and Analytics Group within the Technology... ...CloudImplement modern data and analytics practices using tools such as Databricks, Snowflake, Salesforce, Python, and SQLDesign, develop,...DatabricksLong term contractInternship$84k - $128k per year
...personal and professional growth. Your OpportunityBDO Digital is seeking an experienced and technically proficient Senior Data Engineer (Databricks Focused) to join our Technology Advisory Services practice. This role blends technical leadership with hands‑on data...DatabricksManual laborWork at officeLocal areaFlexible hours- ...OpportunityJoin a leading institutional investment organization as a Senior Data Engineer supporting a major Eagle platform upgrade. You’ll play a key... ...reconciliation and regression-testing solutions using Azure Databricks, Python/PySpark, Spark SQL, and Oracle SQL/PLSQL. You’ll...DatabricksContract work2 days per week3 days per week
- Lead enterprise Data and AI engineering with Databricks and AWS in Vanguard’s hybrid working model, building scalable platforms for analytics.Vanguard seeks an experienced Senior Lead Data and AI Engineer with 9–12+ years in Data Engineering and modern cloud data platforms....Databricks
- Build finance data platforms with Databricks, PySpark, and advanced SQL in a hybrid role requiring four days weekly onsite in downtown Toronto. Transform... ...reporting foundations.Jarvis is hiring a Senior Data Engineer for a financial services client. You will own finance data...Databricks
- Lead enterprise data platforms with Databricks and Apache Spark at ShyftLabs, working three days weekly from its downtown Toronto office. Shape cloud... ...solutions for Fortune 500 clients.As a Senior / Lead Data Engineer, you will own delivery from discovery through production...DatabricksWork at office
- Databricks and Azure leadership define this hybrid Director, Data Engineering opportunity at Gore Mutual. Build trusted data products for enterprise analytics and AI.As Director, Data Engineering, you will lead teams across Data Products, Data Platforms, Data Engineering, and...Databricks
$135k - $175k per year
...investors, multinationals, and private clients worldwide. Our data engineering initiatives represent one of our most critical technical... ...Citco's enterprise‑wide Centralized Data Platform (CDP) built on Databricks. This role requires a deep, hands‑on understanding of Databricks...DatabricksWorldwideFlexible hours$120k - $150k per year
...days onsite. On behalf of our client, Affinity is seeking a Principal Engineer, Data Solutions who will hold a senior technical leadership... ...transformation frameworks Modern cloud data warehouses (Fabric, Databricks) Power BI or Tableau (DAX, calculated fields, performance...DatabricksLong term contractManual labor- Shape client data initiatives at BDO through Databricks architecture, stakeholder communication, and hands-on cloud data engineering.BDO Digital seeks a Senior Data Engineer who can lead client engagements while delivering robust analytics and machine learning data solutions...Databricks
- Lead Databricks data platform architecture at ShyftLabs from our downtown Toronto office three days weekly, using Spark and Python.As a Senior / Lead Data Engineer, you will guide enterprise-scale platforms for Fortune 500 clients. Apply Databricks, Apache Spark, Python, SQL...DatabricksWork at office
$80 - $96 per hour
...Hybrid (Downtown, Toronto)Duration: 12 monthsOur client, a leading financial institution in Downtown Toronto, is looking for a Principal Data Engineer to set up, maintain, and scale data infrastructure and platforms. The successful candidate will have the opportunity to work...Hourly pay- ...Ledcor is seeking a Senior Data Engineer to design, build, and maintain enterprise-scale data solutions across construction and infrastructure businesses. You’ll work on a Databricks-based platform with medallion architecture, partnering with analysts, developers, and stakeholders...Databricks
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Principal Data Engineer - Databricks. Be the first to apply!
Related searches
- mechanical engineering director Winnipeg, MB
- mechanical engineering project manager Winnipeg, MB
- directeur ingénierie Winnipeg, MB
- chief power engineer Winnipeg, MB
- director software engineering Winnipeg, MB
- project engineer assistant project manager Winnipeg, MB
- engineering project manager Winnipeg, MB
- chief engineer Winnipeg, MB
- general engineer Winnipeg, MB
- data integration developer Winnipeg, MB

