Sr Data Engineer

Who we areML arteka is one of the fastest-growing IT Services and Consulting companies in Canada and serves some of the most iconic brands, including Google, Loblaws, Staples, Telus, Bell, Rogers, CIBC, BMO, CMLS, Fossil, Fitbit and more. Over the last decade, we have achieved 100% client retention and earned more than 20 notable industry recognition. This reflects our reliability in delivering what we promise and the quality we bring to every engagement. Join our growth-driven culture and help us lead, innovate, and simplify. We are looking for a team members who embody our values:
Empathetic - we listen, we understand, and are considerate of everyone
Persevering - we dont give up on challenges or people and go all the way
Inclusive - we value everyone and their perspectives
Reliable - we deliver what we promise, and more
Innovative - we make things better than they were yesterday
The Role :Build the source-to-Silver ingestion layer for a complex data and AI platform transformation at a regulated financial institution. Own the design and delivery of 30+ source connectors across low, medium, and high complexity tiers — file-based, SQL-based, and API-based — on a medallion architecture running on AWS. You are not a pipeline configuration resource. You are an engineer who understands what the data means, where it breaks, and why, and you build connectors that hold up in production under schema drift, late arrivals, and source quality gaps.
You Will :
Design, build, test, and own source connectors from S3 landing to validated Silver records. Each connector covers the full four-component stack: S3 trigger configuration, Bronze layer write (immutable raw copy with provenance metadata), Silver layer transformation (schema enforcement, field mapping, type casting, null and range checks, deduplication, bad record quarantine), and source arrival monitoring feeding the pipeline audit table.
Translate source-to-target mapping specifications — provided by the client data analyst team — into connector logic that handles both single-field mappings and multi-field derivations: conditional expressions, arithmetic combinations across multiple source fields, and lookup-based enrichment.
Investigate and document source data behavior during the discovery phase: field semantics, undocumented transformations currently embedded in legacy code, frequency and format variability, and data quality gaps that must either be replicated or explicitly flagged in the new platform.
Implement bad record handling that quarantines rather than drops — structured failure reasons written to a quarantine prefix with enough context for a business analyst to diagnose the issue without engineering involvement.
Build and maintain the source arrival monitoring layer: expected arrival windows per source, SLA breach alerting, missed file notification, and source health dashboard feeding CloudWatch.
Work directly with the client's embedded data engineers and data analyst in a co-development model — peer-reviewing their connector code against platform standards, contributing to theirs, and maintaining shared code quality across a jointly owned codebase.
Participate in the parallel run: investigate reconciliation breaks traced to the Bronze or Silver layer, distinguish source defects from connector defects, and produce a root cause statement the QA lead can include in the Phase Gate Report.
Write and maintain connector unit tests and integration tests as first-class deliverables, not afterthoughts — your connectors are tested before they reach QA, not handed over for testing.
Who You Are :
Strong Python engineering skills — you write production-quality code, not scripts. You know the difference between code that works once and code that holds up under bad data, late files, and schema changes three months into a project.
Experienced building data ingestion pipelines on AWS: S3-triggered processing, Glue ETL jobs or equivalent, schema enforcement, and pipeline observability. You have dealt with sources that do not behave the way their documentation says.
Familiar with Apache Iceberg table format for Bronze and Silver layers: partition design, schema evolution, and how Iceberg handles late-arriving and out-of-order records.
You read a source data dictionary and immediately know which fields will cause problems, which data types will drift, and which business rules embedded in the legacy system are about to become your transformation problem.
Comfortable working in co-development settings where the client's engineers own connectors alongside yours — you review their code honestly, accept feedback on yours, and treat the shared codebase as a shared responsibility.
Experience with orchestration frameworks (MWAA, Step Functions, Airflow, or Control-M): you understand how your connector fits into the dependency graph and what happens downstream when it fails.
Detail-oriented on data provenance — you build Bronze layers that make forensic investigation possible, not just fast processing.
Operates effectively in regulated environments where every data movement needs a traceable audit record and every schema change needs a documented rationale.
What to expectBesides Competitive Salary and working for one of Canadas Best Managed and Top 50 Technology Companies, you should expect:
Purposeful Work: autonomy, resources, and time you need to do what matters for our clients and most importantly - you
Positive Environment: recognition coupled with a flexible environment that celebrates diversity, inclusivity and encourages everyone to achieve their best
Growth & Development: opportunities to grow professionally and personally through mentorship, training, and development programs
Transparency & Trust: direction, purpose, mission, and expectations are never a secret;
you will always know where you stand as well as where you are going
Exclusive Perks: discounts on shopping, the gym, insurance, mobile phone plans, not to mention an office smoothie bar and enough coffee to make you vibrate
Meaningful Benefits: life happens. You will get health, dental, vision, disability, travel insurance, and much more all covered by us
Unforgettable Events: team bonding, BBQs, office lunches (pizza, always pizza), gala dinners, and any other excuse to have fun
Inclusion and equal opportunity employmentML arteka is an equal opportunity employer committed to diversity and inclusion. We are pleased to consider all qualified applicants for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, protected veterans status, Aboriginal/Native American status, or any other legally-protected factors. Disability-related accommodation during the application process is available upon request.
Are you ready to join the award-winning team and help us accelerate digital transformation?#J-18808-Ljbffr