Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Machine Learning Engineer, AI Framework New

Full-time

GitLab

Job Responsibility:

GitLab is an open core software company that develops the most comprehensive AI-powered DevSecOps Platform, used by more than 100,000 organizations. Our mission is to enable everyone to contribute to and co-create the software that powers our world. When everyone can contribute, consumers become contributors, significantly accelerating the rate of human progress. This mission is integral to our culture, influencing how we hire, build products, and lead our industry. We make this possible at GitLab by running our operations on our product and staying aligned with our values. Learn more about Life at GitLab.

Thanks to products like Duo Enterprise, and Duo Workflow, customers get the benefit of AI at every stage of the SDLC. The same principles built into our products are reflected in how our team works: we embrace AI as a core productivity multiplier. All team members are encouraged and expected to incorporate AI into their daily workflows to drive efficiency, innovation, and impact across our global organisation.

An overview of this role

Are you passionate about building robust frameworks to evaluate and ensure the reliability of AI models? As a Machine Learning Engineer on GitLab's AI framework team, you'll play a critical role in shaping the future of AI-powered features at GitLab. This is an exciting opportunity to work on impactful projects that directly influence the quality of GitLab's AI capabilities.

You'll help merge cutting-edge evaluation tools, optimize dataset management, and scale our validation infrastructure. Working closely with other AI feature teams, you'll ensure that every AI feature we deliver is robust, reliable, and meets the highest quality standards.

Some challenges in this role include designing scalable solutions for LLM evaluation, consolidating disparate validation tools, and contributing to GitLab's innovative AI roadmap.

Some examples of our projects:

Consolidating Evaluation Tooling | The GitLab HandbookGitLab.org / AI Powered / ELI5

  • GitLab.org / ModelOps / AI Model Validation and Research / AI Evaluation / Prompt Library

What You'll Do

  • Design and implement technical evaluators for LLM assessment.
  • Contribute to evaluation infrastructure consolidation efforts.
  • Build scalable evaluation pipelines and frameworks.
  • Develop and manage datasets and evaluation metrics.
  • Collaborate with feature teams to integrate validation solutions.
  • Optimize performance across ML evaluation systems.
  • Support improvements to GitLab's AI-powered tools through validation.
  • Ensure all solutions align with GitLab's infrastructure and security protocols.

What You'll Bring

  • Proven experience designing and implementing LLM evaluation systems.
  • Strong understanding of ML model architectures, including public vs. private implementations.
  • Expertise in ML evaluation metrics and dataset management.
  • Demonstrated ability to build production-grade ML infrastructure.
  • Practical experience with Python-based ML frameworks and evaluation tools (e.g., Langsmith, ELI5).
  • Excellent problem-solving skills with an engineering mindset.
  • Ability to collaborate in an asynchronous, remote-first environment.
  • Familiarity with open-source development and contribution is a plus.

About the team

The AIF team ensures that AI models across GitLab are reliable and well-validated. We focus on building robust evaluation frameworks, consolidating tools, and streamlining processes to scale validation efforts across GitLab's AI infrastructure. Working on high-impact projects, the team partners with AI feature teams to deliver quality-focused solutions that enhance user trust and product performance.

How GitLab will support you
  • Benefits to support your health, finances, and well-being
  • All remote, asynchronous work environment
  • Flexible Paid Time Off
  • Team Member Resource Groups
  • Equity Compensation & Employee Stock Purchase Plan
  • Growth and development budget
  • Parental leave
  • Home office support

Please note that we welcome interest from candidates with varying levels of experience; many successful candidates do not meet every single requirement. Additionally, studies have shown that people from underrepresented groups are less likely to apply to a job unless they meet every single qualification. If you're excited about this role, please apply and allow our recruiters to assess your application.

Country Hiring Guidelines: GitLab hires new team members in countries around the world. All of our roles are remote, however some roles may carry specific location-based eligibility requirements. Our Talent Acquisition team can help answer any questions about location after starting the recruiting process.

Privacy Policy: Please review our Recruitment Privacy Policy. Your privacy is important to us.

GitLab is proud to be an equal opportunity workplace and is an affirmative action employer. GitLab's policies and practices relating to recruitment, employment, career development and advancement, promotion, and retirement are based solely on merit, regardless of race, color, religion, ancestry, sex (including pregnancy, lactation, sexual orientation, gender identity, or gender expression), national origin, age, citizenship, marital status, mental or physical disability, genetic information (including family medical history), discharge status from the military, protected veteran status (which includes disabled veterans, recently separated veterans, active duty wartime or campaign badge veterans, and Armed Forces service medal veterans), or any other basis protected by law. GitLab will not tolerate discrimination or harassment based on any of these characteristics. See also GitLab's EEO Policy and EEO is the Law. If you have a disability or special need that requires accommodation, please let us know during the recruiting process.

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Machine Learning Engineer, AI Framework New in Remote vacancy
  •  ...more information, visit . At Reddit, machine learning sits at the heart of how millions of...  ...systems and marketplace dynamics, our ML engineers tackle some of the most interesting and...  ...and real-time decision systems Applied AI and LLM-driven experiences that improve... 
    Senior
    Remplacement
    Long term contract
    For contractors
    Flexible hours

    Reddit

    Remote
    10 days ago
  •  ...conversation into a competitive advantage. Cresta’s unified AI platform combines conversational AI agents, real-time human agent...  ...day. Come build that future at Cresta. About the role: Machine Learning Engineers at Cresta work across several high-impact AI initiatives.... 
    Senior
    Work at office
    Local area
    Remote work
    Work from home
    Home office

    Cresta

    Remote
    10 days ago
  • $160k - $200k per year

     ...Wiser.  Job Description Wiser Solutions is hiring a Senior ML Engineer to help build our customer-facing AI platform and get the first features into customers'...  ...and code review  ~ A habit of building quickly to learn: you'd rather test an idea in two days than debate it... 
    Senior
    Full time
    Work at office
    Local area
    Worldwide

    Wiser Solutions

    Remote
    14 days ago
  • usd153k - usd213k per year

     ...ML Fraud team, you’ll build and improve machine learning systems that make real-time transaction...  ...You’ll work closely with experienced ML engineers, platform partners, and cross-functional...  ...’ll do - You will lead development of new fraud prediction models using a mix of approaches... 
    Senior
    Work at office
    Remote work
    Flexible hours

    Affirm

    Remote
    10 days ago
  • $180k - $190k per year

     ...through regular in-person events. Learn more about our flexible approach...  ...Overview Instacart’s Enterprise AI organization builds intelligent products...  ...grocery industry. Our teams apply machine learning, optimization, and software engineering to solve complex problems for... 
    Senior
    Permanent employment
    Work at office
    Remote work
    Work from home
    Flexible hours

    Instacart

    Remote
    10 days ago
  • $214.51k - $252.36k per year

     ...adaptability, and the confidence to explore new tools, especially AI, to drive smarter, faster work. You don...  ...an expert, but you should be ready to learn quickly and think critically. We...  ...consumer experiences. We’re looking for a Senior Machine Learning Engineer to serve as the... 
    Senior
    Full time
    Work at office
    Remote work
    Flexible hours

    Fetch

    Remote
    10 days ago
  • $161.5k - $232.5k per year

     ...back their lives. Join us on our mission to reverse diabetes and obesity in one billion people. About the Role As a Senior Engineer, Machine Learning on Virta’s Applied AI team, you build the models that shape how Virta shows up for each member: who needs a coach’s attention... 
    Senior
    Full time
    Work at office
    Remote work

    Virta Health

    Remote
    14 days ago
  • $243.33k - $282.08k per year

     ...providing a unique platform to learn and connect with others. Poe...  ...millions of users run multiple AI agents on shared context and tools...  ...Team and Role: Our small engineering team works on challenging problems...  ...think big and experiment with new ideas. As a remote-first... 
    Senior
    Full time
    Remote work
    Monday to friday
    Flexible hours

    Quora

    Remote
    6 days ago
  •  ...intelligent, modern applications that combine cutting-edge engineering with practical machine learning. We’re looking for a Machine Learning Engineer who is...  ...theory, hands-on with LLMs, and know how to ship high-impact AI features, this role is for you. What You’ll Do... 
    Full time
    Internship

    Numrah

    Remote
    2 days ago
  •  ...all applications and next steps. Our partner is looking for a Machine Learning Engineer based in Canada. As a Machine Learning Engineer, you will...  ...particular focus on Large Language Models (LLMs) and intelligent AI-powered applications. You will take ownership of the ML... 
    Internship

    Jobgether

    Remote
    3 days ago
  •  ...impact as we build for the long term. About the role: The Samsara AI team builds end-to-end AI solutions for our customers as well as core ML infrastructure for Samsara. As a Staff Machine Learning Engineer, you will be working with petabyte-scale sensor, diagnostic, video,... 
    Long term contract
    Full time
    Remote work
    Relocation package

    Samsara

    Remote
    14 days ago
  • $154k - $216k per year

     ...the security, stability, and resources of our parent company, Hearst, with over 100 years of experience. The Staff Machine Learning Engineer is a senior individual contributor responsible for building and evolving the machine learning engineering capabilities that enable... 
    Senior
    Full time
    Remote work
    Flexible hours

    ODG by MCG

    Remote
    11 days ago
  •  ...users worldwide with real-time AI-powered insights that drive meaningful...  ...and organizations can attract new audiences, delight customers,...  ...amplifying individual voices. Learn more at surveymonkey.com. What...  ..., we harness AI and Machine Learning to delight customers at... 
    Full time
    Internship
    Remote work
    Worldwide

    SurveyMonkey

    Remote
    3 days ago
  • $26 - $99 per hour

     ...Teach live online classes on training and evaluating machine learning models with AI help: data splits, baselines, leakage and the right metric. You...  ...for each class you teach. What the work is for YPAI Learn teaches people to work with AI tools in live classes. Your... 
    Contract work
    Remote work

    YPAI

    Remote
    9 days ago
  • usd198.56k - usd248.2k per year

     ...how the world interacts, we’re acquiring new skills and experiences that make work feel...  ...work We use Artificial Intelligence (AI) to help make our hiring process efficient...  ...Join the team as Twilio’s next Staff Machine Learning Engineer on Twilio’s AI/ML Platform team.... 
    Long term contract
    Local area
    Remote work
    Worldwide

    Twilio

    Remote
    10 days ago
  • $181k - $241k per year

     ...tradeoffs on what matters most. About the role Join Affirm as a Machine Learning Engineering Manager and lead our Commerce Optimization Machine Learning...  ...how they work Pay Grade - P Equity Grade - 7 Employees new to Affirm typically come in at the start of the pay range. Affirm... 
    Long term contract
    Full time
    Work at office
    Remote work

    Affirm

    Remote
    2 days ago
  •  ...Required Qualifications A BS in Computer Science, Machine Learning, Artificial Intelligence, Applied Mathematics, Statistics, Engineering, or a closely related quantitative discipline....  ...at a well-established technology company, AI laboratory, research organization, or other... 
    Part time
    Immediate start

    micro1

    Remote
    6 days ago
  •  ...applications and next steps. Our partner is looking for a Staff Machine Learning Engineer – Medical Devices based in Canada. As a Staff Machine...  ...Engineer, you will play a pivotal role in developing innovative AI-powered medical technologies designed to improve cardiac diagnostics... 

    Jobgether

    Remote
    3 days ago
  •  ...Growing! We're looking for a leader to own the machine learning systems behind post-purchase commerce —...  ...and the intelligence layer that agentic AI runs on. Hundreds of millions of...  ...systems: Graphite , our identity resolution engine that ties fragmented consumer data into a... 
    Senior
    Full time
    Remote work
    Shift work

    Narvar

    Remote
    10 days ago
  •  ...applications and next steps. Our partner is looking for a Staff Machine Learning Engineer, Sensors & Intelligence based in Canada. This is an...  ...impactful work, and collaborate with engineering teams to bring new capabilities to users. You will also mentor colleagues and help... 
    Full time

    Jobgether

    Remote
    15 days ago
  • $40 - $150 per hour

     ...Plausible-sounding ML advice is often subtly wrong. As a Machine Learning Engineer you'll stress-test how models reason about ML systems and write...  ..., debugging, and productionizing ML systems. Review AI output for subtle errors: leaky evaluations, wrong loss formulations... 
    Weekly pay
    Full time
    Contract work
    For contractors
    Remote work
    Flexible hours

    DataAnnotation

    Remote
    a month ago
  •  ...About Fleet AI Fleet AI builds training environments where AI agents practice real work and get graded on it. As a Fleet Fellow, Domain Expert (Machine Learning Engineering), you'll contribute specialized expertise to help shape research benchmarks across active Fleet AI... 
    Hourly pay
    Full time
    Contract work
    For contractors
    Remote work
    Flexible hours

    Fleet AI

    Remote
    9 days ago
  •  ...Required Qualifications A BS in Computer Science, Machine Learning, Artificial Intelligence, Applied Mathematics, Statistics, Engineering, or a closely related quantitative discipline....  ...at a well-established technology company, AI laboratory, research organization, or other... 
    Part time
    Immediate start

    micro1

    Remote
    4 days ago
  • $80 per hour

     ...Nov 19 → How Ethos hiring works Your application is a 10-minute call. Ethos's AI interviewer asks about your career, then matches you to projects like Machine Learning Engineer. Keep your CV open and cover these: 4+ years as a software engineer, data engineer... 
    Hourly pay
    Weekly pay
    Contract work
    Immediate start
    Remote work
    Flexible hours

    Ethos

    Remote
    a month ago
  •  ...applications and next steps. Our partner is looking for an AI Research Engineer (Agentic Reinforcement Learning) based in Canada. As an AI Research Engineer...  ...autoregressive architectures, and unified multimodal frameworks. Through rigorous experimentation and benchmarking,... 
    Full time
    Remote work

    Jobgether

    Remote
    16 days ago
  • usd133k - usd183k per year

     ...no surprises, and no tradeoffs on what matters most. As a Machine Learning Engineer on our team, you will be at the forefront of developing high-...  ...decisioning at Affirm Implement and scale data pipelines, new features, and algorithms that are essential to our production... 
    Work at office
    Remote work
    Flexible hours

    Affirm

    Remote
    10 days ago
  • $125 per hour

     ...professional experience Demonstrated expertise in Software Engineering How long hiring takes Across the platforms we work...  ...60% Within 3 months ~80% Why this role $125/hr for Senior Machine Learning Engineer / Model Evaluations Expert work puts your Software... 
    Senior
    Hourly pay
    Contract work
    Remote work

    Ethos

    Remote
    a month ago
  •  ...that generate high-quality visual output at scale.   As a Senior AI Engineer, you build the intelligence behind that shift: LLM agents that...  ...automated pipelines that turn messy client material into structure machines can work with.   The hard problem in generative AI is no... 
    Senior
    Permanent employment
    Full time
    Manual labor
    Shift work

    INDG

    Remote
    3 days ago
  •  ...Fincons Group built its reputation on foreseeing and interpreting new business models and the rapid evolution of IT systems by...  ...people. To support our customer, we are looking for a: AI/ML ENGINEER - Senior TASKS: Develop and deploy AI/ML solutions, including machine... 
    Senior
    Full time
    Work at office
    Worldwide

    Fincons Group

    Remote
    11 days ago
  •  ...all applications and next steps. Our partner is looking for a Senior AI & MLOps Engineer based in Canada. As a Senior AI & MLOps Engineer, you will...  ...intersection of Agentic AI, Generative AI, cloud engineering, and machine learning operations to bring innovative AI products from... 
    Senior
    Remote work

    Jobgether

    Remote
    7 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Machine Learning Engineer, AI Framework New. Be the first to apply!