RQ11486 - Data Ops/Cloud Data Engineer - Senior

Client: Ministry of Public and Business Service Delivery and Procurement – RQ11486

We are seeking a Senior Data Ops/Cloud Data Engineer with extensive experience in designing, developing, automating, and optimizing complex data pipelines and cloud-based data platforms. The ideal candidate will have strong hands‐on expertise in Databricks, Delta Live Tables (DLT), Unity Catalog or equivalent technologies, Azure Data Factory, advanced SQL, PySpark, data modelling, and enterprise‐scale data migration and integration. The successful candidate will be responsible for building and optimizing ETL/ELT processes, implementing CI/CD practices, managing cloud data services, and supporting data engineering initiatives across multiple platforms, including Oracle, Azure Data Lake, Databricks, and Microsoft Fabric.

Role Snapshot

• Job Title: RQ11486 – Data Ops/Cloud Data Engineer – Senior
• Client: Ministry of Public and Business Service Delivery and Procurement
• Work Location: 40 St. Clair Avenue West, Toronto, Ontario, Canada
• Work Arrangement: Onsite – 5 Days per Week
• Estimated Start Date: October 5, 2026
• Estimated End Date: March 31, 2027
• Business Days: 123.00
• Extension: 60.00
• Hours: 7.25 hours per day
• Security Level: No Clearance Required

Must Have

• Experience designing, building, automating, and optimizing complex data ETL/ELT processes using Databricks, Delta Live Tables (DLT), or equivalent technologies.
• Experience developing unified data platforms using Databricks and Unity Catalog or equivalent technologies.
• Experience with relational databases, including Oracle, MySQL, and SQL Server; relational and dimensional data modelling; advanced SQL; query optimization; data replication; and database administration.
• Advanced SQL skills, including PL/SQL and T‑SQL.
• Experience with big data processing frameworks, including PySpark.
• Proficiency in creating Azure Data Factory pipelines for copy activities and developing custom data pipelines in Databricks with CI/CD using Azure DevOps Git integration.
• Experience with data migration and integration across different platforms, including:
  • Oracle to Azure Data Lake
  • Oracle to Databricks
  • Oracle to Microsoft Fabric

Description

To respond to the priorities of the Government of Ontario through the passage of Bill 5, Protect Ontario by Unleashing Our Economy Act, 2025 , the resource may be partially assigned to Bill 5‑related projects or initiatives on an as‑needed basis.

Responsibilities

• Design and develop data pipelines from source systems to end users.
• Optimize data pipelines.

General Skills

• Experience with cloud data platforms, data management, and data exchange tools and technologies.
• Experience with commercial and open‑source data and database development and management, specializing in data storage settings and managing Cloud Data as a Service (DaaS), application Database as a Service (DBaaS), Data Warehouses as a Service (DaaS), and other storage platforms in both cloud and on‑premises environments.
• Experience with data pipeline and workflow development, orchestration, deployment, and automation, specializing in programming and pipelines to create and manage data flows and data movement.
• Experience as a Cloud Data Engineer, including familiarity with and experience in different programming languages and the ability to integrate multiple platforms to create data pipelines, automate tasks, and write scripts.
• Experience with DataOps principles, best practices, implementation, and Agile project development and deployment.
• Experience with Continuous Integration/Continuous Development/Deployment (CI/CD) and data provisioning automation.
• Experience with digital products, data analysis, data exchange, data provisioning, and data security.
• Extensive expert experience in designing, developing, and implementing data conversion and migration of Very Large Data (VLD) across Online Analytical Processing (OLAP) and Online Transaction Processing (OLTP) environments to Cloud Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS) environments.
• Experience in the design, development, and implementation of fact/dimension models, data mapping, data warehouses, data lakes, and data lakehouses for enterprise environments.
• Experience managing cloud data services for project delivery, including storage, repositories, data lakes, data lakehouses, key vaults, virtual machines, disks, and related services.
• Experience with structured, semi‑structured, and unstructured data collection, ingestion, provisioning, and exchange; technological development of enterprise data warehouse, data lake, and data lakehouse solutions; and operational support.
• Experience with DataOps performance monitoring and tuning.
• Excellent analytical, problem‑solving, and decision‑making skills; verbal and written communication skills; presentation skills; interpersonal skills; and negotiation skills.
• A team player with a track record of meeting deadlines.

Desirable Skills And Experience

• Data lake design and development, with a focus on extracting and transforming data from various sources and loading it into a lakehouse using medallion architecture.
• Advanced skills and hands‑on experience with Oracle databases, Azure SQL Server, and Azure Data Factory.
• Experience with cloud technologies, including Azure, Google Cloud, and AWS.
• Experience with PL/SQL for data extraction, transformation, and loading; initial setup; and other ETL experience, including management and support activities such as troubleshooting, performance tuning, failover, and recovery.
• Experience with scripting languages, including Python, Unix shell, and Scala.

Experience And Skill Set Requirements

• Mandatory Skills and Experience — 70%

Candidates Must Demonstrate Strong Hands‑on Experience In The Mandatory Technical Skills And Responsibilities Outlined In The Must Have And General Skills Sections, Including:

• Designing, building, automating, and optimizing ETL/ELT data pipelines using Databricks, Delta Live Tables (DLT), or equivalent technologies.
• Developing unified data platforms using Databricks and Unity Catalog or equivalent technologies.
• Working with relational databases, data modelling, advanced SQL, PL/SQL, T‑SQL, query optimization, data replication, and database administration.
• Applying big data processing frameworks, including PySpark.
• Developing Azure Data Factory pipelines and custom Databricks data pipelines with CI/CD and Azure DevOps Git integration.
• Performing data migration and integration across Oracle, Azure Data Lake, Databricks, and Microsoft Fabric.
• Applying DataOps principles, CI/CD, data provisioning automation, cloud data management, pipeline orchestration, performance monitoring, and tuning.
• Supporting enterprise‑scale data conversion, migration, warehousing, lake, and lakehouse solutions.
• Desirable Skills and Experience — 25%
• Data lake and lakehouse design and development using medallion architecture.
• Advanced hands‑on experience with Oracle databases, Azure SQL Server, and Azure Data Factory.
• Knowledge of Azure, Google Cloud, and AWS.
• PL/SQL‑based data extraction, transformation, loading, ETL management, troubleshooting, performance tuning, failover, and recovery.
• Scripting experience with Python, Unix shell, and Scala.
• Education — 3%
• Education‑related qualifications and experience as applicable to the assignment requirements.
• Public Sector Experience — 2%
• Public sector experience relevant to the assignment and the priorities of the Government of Ontario.

The candidate is required to work from the office five days per week.