Machine Learning Engineer senior (H/F)

GitLab is an open core software company that develops the most comprehensive AI-powered DevSecOps Platform, used by more than 100,000 organizations. Our mission is to enable everyone to contribute to and co-create the software that powers our world. Thanks to products like Duo Enterprise, and Duo Workflow, customers get the benefit of AI at every stage of the SDLC. The same principles built into our products are reflected in how our team works: we embrace AI as a core productivity multiplier. All team members are encouraged and expected to incorporate AI into their daily workflows to drive efficiency, innovation, and impact across our global organisation.

Are you passionate about building robust frameworks to evaluate and ensure the reliability of AI models? As a Machine Learning Engineer on GitLab's AI framework team, you'll play a critical role in shaping the future of AI-powered features at GitLab. This is an exciting opportunity to work on impactful projects that directly influence the quality of GitLab's AI capabilities.
You'll help merge cutting-edge evaluation tools, optimize dataset management, and scale our validation infrastructure. Working closely with other AI feature teams, you'll ensure that every AI feature we deliver is robust, reliable, and meets the highest quality standards.
Some challenges in this role include designing scalable solutions for LLM evaluation, consolidating disparate validation tools, and contributing to GitLab's innovative AI roadmap.
Some examples of our projects:
org / AI Powered / ELI5
~ org / ModelOps / AI Model Validation and Research / AI Evaluation / Prompt Library

Design and implement technical evaluators for LLM assessment.
Build scalable evaluation pipelines and frameworks.
Optimize performance across ML evaluation systems.
Support improvements to GitLab's AI-powered tools through validation.
Strong understanding of ML model architectures, including public vs. private implementations.
Expertise in ML evaluation metrics and dataset management.
Demonstrated ability to build production-grade ML infrastructure.
Practical experience with Python-based ML frameworks and evaluation tools (e.g., Ability to collaborate in an asynchronous, remote-first environment.
The AIF team ensures that AI models across GitLab are reliable and well-validated. We focus on building robust evaluation frameworks, consolidating tools, and streamlining processes to scale validation efforts across GitLab's AI infrastructure. Working on high-impact projects, the team partners with AI feature teams to deliver quality-focused solutions that enhance user trust and product performance.
All remote, asynchronous work environment
Flexible Paid Time Off
Equity Compensation & Employee Stock Purchase Plan
Parental leave
Home office support
All of our roles are remote, however some roles may carry specific location-based eligibility requirements. Our Talent Acquisition team can help answer any questions about location after starting the recruiting process.
Privacy Policy: Please review our Recruitment Privacy Policy. GitLab's policies and practices relating to recruitment, employment, career development and advancement, promotion, and retirement are based solely on merit, regardless of race, color, religion, ancestry, sex (including pregnancy, lactation, sexual orientation, gender identity, or gender expression), national origin, age, citizenship, marital status, mental or physical disability, genetic information (including family medical history), discharge status from the military, protected veteran status (which includes disabled veterans, recently separated veterans, active duty wartime or campaign badge veterans, and Armed Forces service medal veterans), or any other basis protected by law. GitLab will not tolerate discrimination or harassment based on any of these characteristics. See also GitLab's EEO Policy and EEO is the Law. If you have a disability or special need that requires accommodation, please let us know during the recruiting process.