senior data engineer azure databricks / data engineering- tdjp00058306.

Our client, is seeking a deeply experienced, senior-level Senior Data Engineer to join their Enterprise Cloud Data Platform division.

In this advanced technical role, you will serve as a subject matter expert and key contributor responsible for developing, maintaining, and scaling enterprise-grade data engineering solutions. You will focus heavily on designing and operationalizing high-performing, reliable, and secure data pipelines within a complex cloud ecosystem. This position involves architecting data ingestion, transformation, and distribution frameworks capable of seamlessly processing multi-millions of records daily while enforcing rigid governance and performance standards. You will act as an innovator and agent of change, driving modern data engineering practices and creating reusable frameworks that bridge technical delivery with core business priorities.

Duration: 12-Month Contract

Schedule: Monday to Friday – Core Business Hours

Work Arrangement: Hybrid (4 days per week on-site at the corporate office, 1 day work from home)

Advantages

Enterprise Financial Scale Sandbox: Own and optimize high-throughput data processing platforms that ingest and manipulate multi-millions of transactional and operational records daily.

Cutting-Edge Data Tech Stack: Work extensively with advanced cloud data features, including Azure Databricks Unity Catalog, Lakeflow data management, and automated Airflow job orchestration.

Strategic Technical Leadership: Drive engineering strategy, establish reusable component frameworks, and champion the integration of AI utilities into the software development lifecycle (SDLC).

Long-Term Contract Visibility: Secure a stable 12-month foundational contract within a mature, highly collaborative enterprise technology landscape.

Responsibilities
Pipeline Engineering & Platform Orchestration
Scalable Data Solutions: Design, build, and deploy production-grade data pipelines and transformation layers using Azure Databricks and Python notebooks.

Automated Job Orchestration: Configure, manage, and optimize automated data ingestion workflows and scheduling pipelines using Azure Data Factory (ADF) and Airflow.

Data Lake & Database Administration: Develop complex database schemas, manage storage pools within Azure Data Lake Storage (ADLS), and execute high-performance SQL development across Azure SQL environments.

API Service Enablement: Design and implement robust API service layers to facilitate seamless, secure data consumption for downstream analytics applications and business consumers.

Data Governance Enforcements: Ensure complete end-to-end data transparency, auditing, and organizational control by leveraging Databricks Unity Catalog and Lakeflow features.

Strategic Advisory, Governance & Compliance
Collaboration & Requirements Synthesis: Partner across Technology and Operations groups to understand business context, interpret complex data needs, and translate them into automated, self-serve data pipelines.

Risk & Control Consultation: Consult with enterprise governance groups (operational risk, compliance, and audit teams) to ensure data solutions strictly align with internal data security principles and financial industry regulations.

Continuous Improvement & Innovation: Identify opportunities to maximize operational efficiency through reusable code structures, workflow automation patterns, and AI-assisted engineering deliverables (prompts, instructions, and intelligent agents).

Knowledge Transfer: Mentor team members, share cloud data architecture best practices, and contribute to technical alignment sessions with engineering peers and platform vendors.

Qualifications
Experience: Minimum 7+ years of progressive professional experience operating as a Data Engineer, Big Data Specialist, or Cloud Data Architect within a large-scale corporate environment.

Core Cloud Platform Depth: Extensive hands-on development expertise inside the Azure cloud data suite, with specialized focus on Azure Databricks, Python notebooks, and SQL development.

Ecosystem Component Fluency: Proven, referenceable experience deploying software solutions utilizing ADLS, ADF, Azure SQL, and Airflow for large-scale data ingestion and data transformation tasks.

Governance Tool Mastery: Direct, practical exposure working with Unity Catalog and Lakeflow capabilities within the Databricks ecosystem.

API Connectivity Depth: Demonstrated skill set establishing API service layers for real-time or batch-oriented application integrations.

Education: University or Post-Graduate degree in Computer Science, Software Engineering, Mathematics, or a related quantitative field.

Soft Skills: Outstanding communication, facilitation, and stakeholder management skills. A proactive professional showing deep technical ownership, capable of untangling complex, non-standard processing situations and collaborating smoothly across multidisciplinary business lines.

Nice-to-Have Skills & Assets
Prior professional experience designing and shipping scalable solutions that process multi-millions of records daily under tight operational SLAs.

Hands-on experience integrating AI into the SDLC using reusable prompts, custom skills, tailored instructions, or autonomous engineering agents.

Background in building enterprise-grade software products and shared-service microservices frameworks within the financial services or banking sectors.

Summary
If you are a battle-tested Senior Data Engineer who pairs an absolute command of Azure Databricks, Python, and Airflow with the architectural depth needed to process multi-millions of records daily, this 12-month hybrid contract offers a premier technology canvas. Bring your pipeline optimization skills, data governance discipline, and passion for automation to our industry-defining cloud engineering squad today!

Randstad Canada is committed to fostering a workforce reflective of all peoples of Canada. As a result, we are committed to developing and implementing strategies to increase the equity, diversity and inclusion within the workplace by examining our internal policies, practices, and systems throughout the entire lifecycle of our workforce, including its recruitment, retention and advancement for all employees. In addition to our deep commitment to respecting human rights, we are dedicated to positive actions to affect change to ensure everyone has full participation in the workforce free from any barriers, systemic or otherwise, especially equity-seeking groups who are usually underrepresented in Canada's workforce, including those who identify as women or non-binary/gender non-conforming; Indigenous or Aboriginal Peoples; persons with disabilities (visible or invisible) and; members of visible minorities, racialized groups and the LGBTQ2+ community.

Randstad Canada is committed to creating and maintaining an inclusive and accessible workplace for all its candidates and employees by supporting their accessibility and accommodation needs throughout the employment lifecycle. We ask that all job applications please identify any accommodation requirements by sending an email to View email address on randstad.com to ensure their ability to fully participate in the interview process.

This posting is for existing and upcoming vacancies. show more