Sr. Data Engineer

Job Title: Senior Data Engineer Location: Greater Toronto Area, ON (Hybrid: 4 days onsite / 1 day remote) Salary Range: $120,000 – $140,000 base salary + quarterly discretionary bonusPosition Overview
We are seeking a Senior Data Engineer to design, build, and maintain the production data and AI foundation on the Azure Databricks Lakehouse platform. Reporting to IT leadership, you will play a hands‑on role in scaling our enterprise data architecture (Medallion architecture across Bronze, Silver, and Gold layers), optimizing Databricks clusters hosted in Azure, and enabling next‑generation AI pipelines (RAG, vector search, LLM integrations, and custom MCP servers). This position requires a senior mindset—someone who brings deep hands‑on expertise with Azure Databricks and SQL Server, takes ownership of end‑to‑end data lifecycle design, and can communicate effectively with technical leads, business stakeholders, and executive leadership.Key Responsibilities
Lakehouse Engineering & Databricks Operations
Data Pipeline Development: Design, build, and optimize scalable ETL/ELT pipelines using Python, PySpark, Spark SQL, and Delta Lake.Medallion Architecture: Implement and maintain robust Bronze, Silver, and Gold data layers to ensure clean, structured, and audit‑ready data.Azure Databricks Infrastructure: Configure and manage Databricks clusters, notebook workflows, and Unity Catalog governance for access control and data lineage.Azure SQL Server Integration: Build, tune, and optimize database schemas, queries, data normalization models, and integration pathways with Azure SQL Server.AI Enablement & Platform Architecture
RAG & Vector Search: Build and support Retrieval‑Augmented Generation (RAG) pipelines over enterprise data, including document ingestion, embeddings generation, and vector database indexing.MCP & LLM Workflows: Build and maintain Model Context Protocol (MCP) servers and LLM API integrations to securely expose Lakehouse data to AI agents and business applications.Data Quality & Governance: Establish automated data quality checks, schema evolution standards, and performance optimization across all pipelines.Stakeholder Engagement & Team Collaboration
Act as a technical subject matter expert, bridging data engineering with business analytics and AI use cases.Translate complex data architectures into business context when interfacing with cross‑functional business units and stakeholders.Qualifications & Competencies Mandatory Requirements
Proven Hands‑On Experience: Direct, verifiable production experience building, deploying, and maintaining data solutions on Azure Databricks Lakehouse (theoretical knowledge or high‑level exposure alone will not suffice).Core Tech Stack: Advanced proficiency in Python, PySpark, SQL (Spark SQL), and Delta Lake.Governance: Hands‑on experience implementing data access controls, lineage, and governance using Unity Catalog.Database & Architecture: In‑depth understanding of data normalization, relational database modeling, query tuning, and integration patterns with Azure SQL Server.Communication: Exceptional verbal and written communication skills with the confidence and maturity to collaborate directly with business partners and executive stakeholders.Work Requirement
Ability to work 4 days per week onsite at our office.Strongly Preferred / AI Qualifications
GenAI / RAG Infrastructure: Experience developing production RAG workflows (embeddings, vector search, retrieval evaluation).Model Context Protocol (MCP): Experience building or integrating custom MCP servers or connecting AI agents to enterprise data systems.LLM Integration: Hands‑on integration of large language model APIs (e.G., Azure OpenAI) into backend data pipelines.Microsoft Ecosystem: Familiarity with broader Azure data tools;
Microsoft Fabric exposure is an asset.AI may be used to support certain administrative aspects of our recruitment process. However, all applications and resumes are reviewed by qualified human reviewers, and all screening and hiring decisions are made solely by the hiring team. The posted salary range reflects expected compensation for this role in accordance with Ontario pay transparency requirements. Actual compensation will be determined based on the successful candidate’s experience, skills, and qualifications and may fall anywhere within the range.
#J-18808-Ljbffr