Databricks Engineer - Azure Data Pipelines & Lakehouse
Compunnel Inc.
The Databricks Engineer will design, develop, and optimize large-scale data pipelines using Databricks and Azure cloud services.This role requires advanced expertise in Spark/PySpark, Delta Lake technologies, data governance, and cloud-based data engineering.The engineer will collaborate with cross-functional teams to deliver scalable, reliable, and high-performance data solutions that support analytics, BI, and business initiatives.Key ResponsibilitiesBuild and maintain scalable ETL/ELT data pipelines using Databricks.Use PySpark/Spark and SQL to process, cleanse, and transform large datasets.Integrate data from diverse sources such as Azure Blob Storage, ADLS, and relational/non-relational databases.Work with teams across the organization to prepare data for BI dashboards and reporting tools.Partner with business stakeholders to understand requirements and deliver tailored data solutions.Performance & Optimization
Optimize Databricks workloads for performance, scalability, and cost efficiency.Monitor, troubleshoot, and resolve issues in data pipelines to ensure accuracy and reliability.Governance & Security
Implement data governance, access controls, and security frameworks using Unity Catalog.Ensure compliance with enterprise data policies and regulatory standards.Deployment
Use Databricks Asset Bundles for deploying jobs, notebooks, and configurations across environments.Maintain version control for Databricks artifacts and collaborate with the team on development best practices.Required Qualifications
10+ years of overall experience in data engineering.Recent experience in financial services, banking, or capital markets.Strong expertise in Databricks, including:Unity CatalogLakehouse ArchitectureTable TriggersDatabricks RuntimeProficiency with Azure cloud services.Deep understanding of Spark and PySpark for big data processing.Strong programming skills in Python.Experience with relational databases.Knowledge of Databricks Asset Bundles and GitLab for version control.Preferred Qualifications
Familiarity with advanced Databricks Runtime configurations.Experience with streaming frameworks such as Spark Streaming.
#J-18808-Ljbffr
Optimize Databricks workloads for performance, scalability, and cost efficiency.Monitor, troubleshoot, and resolve issues in data pipelines to ensure accuracy and reliability.Governance & Security
Implement data governance, access controls, and security frameworks using Unity Catalog.Ensure compliance with enterprise data policies and regulatory standards.Deployment
Use Databricks Asset Bundles for deploying jobs, notebooks, and configurations across environments.Maintain version control for Databricks artifacts and collaborate with the team on development best practices.Required Qualifications
10+ years of overall experience in data engineering.Recent experience in financial services, banking, or capital markets.Strong expertise in Databricks, including:Unity CatalogLakehouse ArchitectureTable TriggersDatabricks RuntimeProficiency with Azure cloud services.Deep understanding of Spark and PySpark for big data processing.Strong programming skills in Python.Experience with relational databases.Knowledge of Databricks Asset Bundles and GitLab for version control.Preferred Qualifications
Familiarity with advanced Databricks Runtime configurations.Experience with streaming frameworks such as Spark Streaming.
#J-18808-Ljbffr
Vacancy posted 54 minutes ago
Similar jobs that could be interesting for youBased on the Databricks Engineer - Azure Data Pipelines & Lakehouse in Winnipeg, MB vacancy
- ...de données senior, niveau 2, pour concevoir et construire des pipelines sur Databricks et orchestrer des flux de données variés. Vous travaillerez... ...continue des données, en collaboration avec les équipes produit et data. Rejoignez une équipe dynamique orientée résultats et impact....PipelineDatabricks
- Azure Data Engineers with Databricks expertise will build scalable pipelines using SQL, Python, and cloud ETL tools across Azure environments.This position focuses on designing, implementing, and optimizing large-scale data pipelines for scalability, reliability, and performance...PipelineDatabricks
- ...HRbrain Inc. is seeking a Data Science Developer to deliver and sustain analytics pipelines, models, and data products. The role focuses on building scalable storage, automating pipelines, and crafting dashboards for insights. Responsibilities include knowledge transfer...PipelineDatabricksWork at office
- Parkland Corporation is expanding its Enterprise BI team with a BI Developer focused on data engineering in an Azure Databricks environment. You will design, build, test, and support trusted data products and pipelines, leveraging Delta Lake, Unity Catalog, and related Databricks...PipelineDatabricks
- Lead Azure Databricks engineering at Seaspan in a hands-on team leadership role focused on scalable data solutions and reliable delivery. Direct Data Engineers while shaping Lakehouse architecture, pipelines, and enterprise data products.As Team Lead, Data Engineering, you...PipelineDatabricks
- Astra-North Infoteck Inc. in Toronto, Canada, seeks an experienced L4 Engineer with hands-on data engineering, cloud platforms, and application development. You will work across Azure Databricks, Databricks ML, and Java/Python backend tasks to support complex data solutions....PipelineDatabricks
- A progressive financial services company in Canada is seeking a Senior Data Engineer. You will set standards for data platforms and act as a technical anchor for scalable systems. Ideal candidates have 5+ years of experience along with advanced skills in SQL and Python. You...Databricks
- ...Coconut Software is seeking a Senior Data Engineer for a 12-month contract in Canada. You will own data engineering on the Data Enablement team, building production pipelines in Databricks and migrating analytics to a lakehouse architecture. You will collaborate with analytics...PipelineDatabricksContract workRemote work
- Build Azure data pipelines with Tata Consultancy Services using SQL, Python, and cloud data platforms. Apply big data expertise to structured and semi-structured datasets.As an Azure Data Engineer, you will design and develop cloud-based pipelines from business requirements...PipelineDatabricks
- NTT DATA seeks a Senior Data Engineer to join our Toronto, Ontario team. The role focuses on designing, building, and optimizing scalable data solutions, including lakehouse pipelines and data ingestion workflows. Ideal candidates will have 8+ years with Databricks, PySpark...PipelineDatabricks
- ...Pathlion is seeking a skilled Data Engineer to join our cloud data platform team in Fredericton, NB. You will analyze requirements, design and implement lakehouse solutions and automated pipelines using Azure and Databricks. You will collaborate with IT and business partners...PipelineDatabricks
- ...North Infoteck Inc. is seeking an accomplished Data Engineering/Architecture lead in Toronto to drive the Cloudera to Databricks transformation. You will shape finance and... ...requires deep expertise in Databricks, Spark, data lakehouse architectures, and cloud platforms (AWS/Azure...Databricks
- We are seeking an experienced Databricks Engineer to join our delivery team on a contract basis in Calgary to design, build, and optimize modern data platforms on Azure Databricks. In this strategic role, you will work directly with both Data Elephant and client data and...PipelineDatabricksContract work
- Shape Azure data architecture through advanced modeling, governance, and lakehouse design for complex public-sector healthcare environments. Work spans enterprise-scale... ...on Azure Data Factory, Data Lake Storage, and Databricks experienceFive-plus years supporting public-sector...Databricks
- ...and operate classic, generative and agentic AI solutions on Azure and Databricks for a major public energy player in Quebec. Montreal-based with... ...; deliverables in French. You will implement MLflow pipelines, model registries and deployment, and lead CI/CD, IaC, and security...PipelineDatabricks
$91.33k - $114k per year
...KF Aerospace in Kelowna, BC, is seeking a Data Engineer to design, develop, and modernize our enterprise data platform using Microsoft Fabric, Azure, SQL Server, and Power BI. You will build reliable data pipelines, Lakehouse architectures, and enterprise reporting to enable...PipelineFull time- Ledcor Group is hiring a Senior Data Engineer to join the Data & Analytics Platform team. You will design, build, and maintain enterprise-... ...requires strong Python, PySpark, and SQL skills, experience with Databricks and cloud platforms, and a solid background in data #J-18808...PipelineDatabricks
$92.58 per hour
...Hourly Rate: $92.58 Job OverviewAs a Staff Data Engineer/Data Architect, you will be responsible... ..., design and delivery of scalable data pipelines and data products which enable... ...leader and subject matter expert on the data lakehouse, data warehousing and modeling activities...PipelineDatabricksHourly payRemote work- ...Ledcor is seeking a Senior Data Engineer to design, build, and maintain enterprise-scale data solutions across construction and infrastructure businesses. You’ll work on a Databricks-based platform with medallion architecture, partnering with analysts, developers, and stakeholders...Databricks
$115k - $160k per year
...customer experience with agile engineering and problem-solving creativity... ...experience across technology, data sciences, consulting, and... ...optimize batch and streaming data pipelines across modern cloud data platforms... ...Web Services, Microsoft Azure, or Google Cloud Platform;...PipelineDatabricksContract work- Develop enterprise ETL pipelines with Databricks, Azure Data Factory, and PySpark in Toronto. Transform data from diverse systems into curated downstream feeds.Apex Systems seeks a Databricks Data Engineer for a large-scale data migration and centralization initiative. The...PipelineDatabricks
- ...CIBC is seeking a Senior Data Engineer to lead the design and delivery of data products on modern cloud platforms. You will collaborate with... ...best practices, and contribute hands-on with SQL, Python, and Azure-based technologies to build scalable, reliable data solutions supporting...Pipeline
- ...services.Behind those APIs sits data our teams must be able to... ...onward, then build and operate the pipelines that run on it: ingestion,... ...BonusLakehouse/warehouse platforms (Databricks, Snowflake, or equivalent) and... ...object storage (AWS/Azure), lakehouse/warehouse platform...PipelineDatabricksContract workWork at office
- Engineer secure, scalable Azure data platforms with Databricks, Spark, and Delta Lake for BDO Digital’s Technology Advisory Services practice.This senior engineering role combines platform development with technical leadership across client engagements. Build reliable pipelines...PipelineDatabricks
- Capgemini is seeking a data engineering lead to design, build, and scale data pipelines using Azure Data Factory and Azure Databricks in Brampton, Canada. You will guide a 10–15 person team and collaborate with executives, product, and analytics stakeholders to deliver robust...PipelineDatabricksContract work
- ...technology company in Ontario is seeking an experienced Data Architect to design and architect enterprise-scale data... ...candidate will have over 8 years of experience in data engineering, strong proficiency with Databricks and Apache Spark, and a solid understanding of cloud...Databricks
- A leading IT consulting firm is seeking a Senior Azure Data Engineer in Toronto, Ontario. The candidate will design and optimize scalable data pipelines within the Microsoft Fabric and SAP BW ecosystem. Qualifications include over 7 years of experience in data engineering,...PipelineFull time
- ~12–18 years of overall Data Engineering experience ~8+ years of experience with enterprise Data... ...~5+ years of hands‑on experience with Databricks and Apache Spark at scale ~ Strong... ...Modernize Cloudera platforms to Databricks Lakehouse architecture ~ Redesign ingestion,...PipelineDatabricks
- Enhance your data engineering skills as a Databricks Engineer focused on Azure cloud solutions. Utilize your extensive Spark experience to optimize and deliver large-scale data pipelines. This role demands over 10 years of expertise in data engineering, particularly within...PipelineDatabricks
- ...Compunnel, Inc. is seeking an Azure Cloud Data Engineer responsible for designing and maintaining scalable cloud-based data platforms. Your role includes developing high-performance ETL pipelines and optimizing large-scale data processing solutions within Azure environments...PipelineDatabricks
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Databricks Engineer - Azure Data Pipelines & Lakehouse. Be the first to apply!
Related searches
- data integration developer Winnipeg, MB
- data centre engineer Winnipeg, MB
- data engineer Winnipeg, MB
- python data engineer Winnipeg, MB
- big data engineer Winnipeg, MB
- gcp data engineer Winnipeg, MB
- remote data engineer Winnipeg, MB
- data cabling engineer Winnipeg, MB
- data engineering intern summer Winnipeg, MB
- junior data engineer Winnipeg, MB

