Senior Data Engineer - Ml Platform

Senior Data Engineer, ML Platform
The roleYou sit on the ML team beside Machine Learning Engineers and own the data infrastructure for model training, inference and monitoring. Today strong software engineers do the data work without lineage, pipeline observability or quality guarantees.
What you'll own
Data lake architecture: lay out a partly built bronze/silver/gold structure properly for long-term ML use.
ML training pipelines: extract from siloed stores, transform, and feed structured and unstructured data into training.
Feature stores: build and own both offline and online stores, and know the trade-offs between them.
Observability and data quality: monitoring, alerting, validation and lineage from day one.
Kafka integration: work with Java teams to consume application events into the pipeline reliably.
Quality frameworks: automated checks and schema validation the ML team can trust.
Engineering design: lead architecture decisions and document trade-offs. As much a software role as a data role.
Requirements
5+ years of software development, 3+ focused on data engineering for data science or ML teams.
Built ML training pipelines and lake or lakehouse architecture, not just analytics pipelines.
Production-grade Python and SQL that other engineers can maintain.
Strong Spark, including optimization at terabyte scale (partitioning, broadcast joins, predicate pushdown).
Snowflake, Databricks, Delta Lake or equivalent.
Feature store experience (offline vs online, freshness, training-serving skew);
ideally Feast or Tecton.
MLOps basics: lineage, versioning, quality monitoring, leakage prevention.
Kafka or another event-driven source in an ingestion pipeline.
Airflow or equivalent set up from scratch at production scale.
CI/CD, monitoring and alerting as standard practice.
Nice to haveunstructured text pipelines (clinical notes, documents)
healthcare data such as HL7, FHIR or claims
RAG or vector search in production
DVC or LakeFS
GCP (Vertex AI, BigQuery, Cloud Composer, Pub/Sub)
open-source or published work.
#J-18808-Ljbffr