Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Research Scientist, Model Evaluation

Cohere

Who are we?Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what’s best for our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products. Join us on our mission and shape the future!
Why this role?Evaluation is critical to making progress in scaling intelligence. As models continue to become superhuman in many real-world use cases, we must continue to develop new evaluation techniques that accurately reflect what models are already capable of, as well as set the agenda for what future models should be capable of. In this role, you are responsible for creating these next-generation evaluation methods and infrastructure to measure LLM progress.
As a Senior Research Scientist, Model Evaluation, you will:
Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish.
Work on highly cross-functional teams to translate model feedback into trustworthy, repeatable evaluations.
Conduct research to advance the state-of-the-art in LLM evaluation methods, including training LLM judges;
refining LLM-based datasynthesis pipelines;
and improving evaluation efficiency.
Build scalable and reusable tools for digging into model performance.
You may be a good fit if:
You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities.
You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality.
You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about.
You have strong software engineering skills.
We value and celebrate diversity and strive to create an inclusive work environment for all. We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs.
Full-Time Employees at Cohere enjoy these Perks:
An open and inclusive culture and work environment
Work closely with a team on the cutting edge of AI research
Weekly lunch stipend, in-office lunches & snacks
Full health and dental benefits, including a separate budget to take care of your mental health
100% Parental Leave top-up for up to 6 months
Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
6 weeks of vacation (30 working days!)
#J-18808-Ljbffr
Vacancy posted 59 minutes ago
Similar jobs that could be interesting for youBased on the Senior Research Scientist, Model Evaluation in Winnipeg, MB vacancy
  • Cohere is seeking a Senior Research Scientist focused on AI evaluation methods to support our innovative foundation models. This remote or on-site role emphasizes creating evaluation benchmarks and tools that enhance LLM capabilities. As a Senior Research Scientist at Cohere... 
    Senior
    Remote work

    Cohere Inc.

    Winnipeg, MB
    1 hour ago
  •  ...stack is powered by Waabi World, which aims to deliver realistic, scalable, controllable, and efficient simulation. As a Research Scientist in World Models, you will develop algorithms and productionize the next generation of World Models that can reason about complex,... 
    Suggested

    Waabi

    Winnipeg, MB
    1 hour ago
  •  ...governance at BMO Financial Group through model validation, quantitative testing, and...  .... Work at 100 King Street West.This Senior Applied AI Scientist role sits in the Model Risk Management...  ...solutions.Key Responsibilities: Evaluate model soundness, quality, robustness,... 
    Senior

    BMO

    Winnipeg, MB
    1 hour ago
  • Develop GEN AI and AI Agent solutions as a Senior Data Scientist with BDO Canada, using deep learning and large language models.BDO Canada seeks a senior professional to create...  ...language models for business needsTrain and evaluate models for accuracy and explainabilityCommunicate... 
    Senior

    BDO Canada

    Winnipeg, MB
    1 hour ago
  • Advance GenAI evaluation and testing at TD in Toronto or Montréal, working with LLMs, NLP, and deep learning models. The role is office focused.This senior scientist position leads research and a team responsible for validating Generative AI, Agentic AI, LLM, NLP, and other... 
    Senior
    Work at office

    TD Bank Group

    Winnipeg, MB
    1 hour ago
  •  ...Role Our client, a world-renowned research institution and technology leader,...  ...Remote Principal AI Research Scientist with deep expertise in Generative Models . This fully remote role is for a...  ...forefront of AI research by continuously evaluating new methodologies and technologies... 
    Long term contract
    Remote work

    Placements24

    Winnipeg, MB
    1 hour ago
  • Research Associate (Senior Research Scientist)Below you will find the details for the position including any supplementary documentation and questions you...  ...laboratory validation, field deployment, performance evaluation, data analysis, documentation, troubleshooting, and iterative... 
    Senior
    Permanent employment
    Full time
    Temporary work
    Part time
    For subcontractor
    Summer work
    Work at office
    Local area
    Remote work
    Afternoon shift

    University of Regina Careers

    Winnipeg, MB
    1 hour ago
  •  ...Senior Evaluation and Performance Analyst Department: Executive and Indigenous Affairs This Job Opening is limited to employees with...  ...administration, or a relevant field of study with a focus on research methods and/or program evaluation, demonstrating training and/... 
    Senior
    Long term contract

    gov

    Winnipeg, MB
    7 days ago
  • Lead advanced statistical modeling at DAC as a Senior Data Scientist, developing rigorous methods with Python or R in a three-day in-office schedule.DAC seeks...  ...analytical challenges. The work includes building models from scratch, applying hierarchical Bayesian methods and... 
    Senior
    Work at office

    DAC

    Winnipeg, MB
    1 hour ago
  • Lead predictive modeling with Databricks and MLflow as a Senior Data Scientist based in Canada. Work remotely on scalable, production-ready machine learning solutions...  ...Senior Data Scientist, you will develop models for customer lifetime value, conversion propensity,... 
    Senior
    Remote work

    Jobgether

    Winnipeg, MB
    1 hour ago
  • Apply data science and Python expertise to evaluate coding agents through reproducible...  ...solution verification.This Competitive Data Scientist role focuses on building reliable tasks...  ..., machine learning, quantitative modeling, or research competition experience.Key Responsibilities... 

    Great Value Hiring

    Winnipeg, MB
    1 hour ago
  •  ...Aviva Canada in Toronto, ON is seeking a Data Scientist to join the Commercial P&C Pricing Modelling & Sophistication team, building models and data-driven solutions that improve customer experience and inform business decisions. You’ll manage the full model lifecycle from... 
    Senior

    Aviva Canada

    Winnipeg, MB
    59 minutes ago
  •  ...Department develops, monitors and maintains our credit risk models so that we can offer innovative financing products to...  ...while ensuring Desjardins’s long-term viability. As a Senior Credit Risk Modelling Data Scientist, you help meet the organization’s needs for risk quantification... 
    Senior
    Long term contract
    Internship
    Remote work
    Flexible hours

    Federation Caisses Desjardins

    Winnipeg, MB
    59 minutes ago
  •  ...Pathlion in Toronto, Ontario, is seeking a Senior Data Analytical Specialist/Scientist to develop advanced analytics, predictive models, and AI-driven insights addressing priority business opportunities. You will apply statistical and machine learning techniques across enterprise... 
    Senior

    Pathlion

    Winnipeg, MB
    1 hour ago
  • $150k - $200k per year

    Senior Data Scientist - Operations ResearchAbout Xanadu: Xanadu’s mission is to build quantum computers...  ...Xanadu, we are learners, innovators, researchers, collaborators and problem solvers. We...  ...to develop optimisation and forecasting models that improve the movement of wafer lots... 
    Senior
    Immediate start

    Xanadu

    Winnipeg, MB
    59 minutes ago
  • As a Senior Data Scientist in the Credit Risk Analytics team at National Bank, you’ll be responsible for the IFRS 9 and stress testing model development and performance monitoring, as well as IFRS 9 reporting and analytics. This role will also need to support other closely... 
    Senior

    National Bank of Canada

    Winnipeg, MB
    1 hour ago
  • Évaluez des modèles IA/ML chez Power Factors, en concevant des métriques, des références et des protocoles reproductibles pour l’énergie renouvelable. Travaillez sur des données d’actifs renouvelables.Ce poste de scientifique IA/ML se concentre sur l’évaluation du modèle de... 

    Power Factors

    Winnipeg, MB
    1 hour ago
  • $180k - $200k per year

     ...Job Title: Staff Data Scientist (AI & Machine Learning) Location: Vancouver...  ...our AI organization. While a Senior Data Scientist excels at building and deploying specific models, you are responsible for the...  ...the gap between cutting-edge research and enterprise-grade... 
    Senior
    3 days per week

    TEEMA Group

    Winnipeg, MB
    7 days ago
  •  ...capabilities for the enterprise, and evaluation is one of the core controls that makes that possible. As Staff Data Scientist, AI Evaluations, you will be a senior technical leader in the data...  ...function responsible for how RBC measures model and agent quality, safety, risk,... 
    Senior
    Full time
    Worldwide
    Flexible hours

    ODAIA

    Winnipeg, MB
    1 hour ago
  • Lead innovative environmental sensing research at the University of Regina as a Senior Research Scientist. This role focuses on AI-enabled systems and interdisciplinary collaboration to tackle real-world challenges. As a Senior Research Scientist, you will support the Canada... 
    Senior

    University of Regina Careers

    Winnipeg, MB
    1 hour ago
  • $80.15k - $125.95k per year

     ...Learn how you can start or jumpstart your career with us. Research Scientist – Mechanical Engineering (Thermal Materials for Electronics &...  ...Heat transfer (conduction, convection, phase change) Thermal modeling and simulation (e.g., FEM/CFD tools such as ANSYS, COMSOL, or... 

    Qnity Electronics, Inc.

    Winnipeg, MB
    1 hour ago
  • Evaluate frontier AI models with electrical engineering judgment, circuit analysis expertise, and flexible remote work from your own computer.DataAnnotation seeks an experienced electrical engineer for independent contract work evaluating real engineering tasks. Apply circuit... 
    Contract work
    For contractors
    Remote work
    Flexible hours

    DataAnnotation

    Winnipeg, MB
    1 hour ago
  •  ...have yet: an Enterprise World Model. While the rest of the...  ...where it became Microsoft’s AI research center in Canada. We've built...  ...and grow a team of Research scientists and Research Engineers, helping...  ...post-training alignment and evaluation Ability to move fluidly between... 
    Internship
    Work at office

    Skyfall.ai

    Winnipeg, MB
    1 hour ago
  •  ...contribute to cutting-edge AI research in Trust and Safety, with a...  ...particular focus on understanding and evaluating the risks of autonomous AI...  ...Role The Applied Research Scientist will advance the...  ...implementing, and evaluating ML models or AI systems, particularly in... 
    Casual work
    Work at office
    Flexible hours
    Shift work

    Alberta Machine Intelligence Institute (Amii)

    Winnipeg, MB
    1 hour ago
  •  ...WhatJobs Direct is actively seeking a highly accomplished Senior AI Research Scientist with a specialization in Reinforcement Learning (RL) to join...  ...developing new algorithms and methodologies. Design, implement, and evaluate RL agents for complex decision-making tasks. Collaborate... 
    Senior
    Remote work
    Work from home
    Home office
    Flexible hours

    Placements24

    Winnipeg, MB
    1 hour ago
  •  ...talent in the space.We are looking for a Senior Data Scientist with a good blend of data analytics...  ...background, practical experience in Operation research strategies and Pricing Analytics within...  ....Develop and implement predictive models and optimization algorithms to improve... 
    Senior
    Local area
    Remote work

    Tiger Analytics

    Winnipeg, MB
    1 hour ago
  • Apply LLM and GenAI expertise as an Applied Scientist developing fair, accurate hiring models for Amazon’s Intelligent Talent Acquisition organization.Global...  ...design models, develop NLP solutions, and collaborate with Research Scientists and Software Engineers. Your work supports... 

    Ml Jobs

    Winnipeg, MB
    1 hour ago
  •  ...leading lithium-ion battery research and development company located...  ...by a team of passionate scientists, engineers, and technical professionals...  ...stakeholders.Support the evaluation, acquisition, installation,...  ...at technical meetings and to senior leadership.Contribute new ideas... 
    Senior
    Casual work
    Work from home

    E-One Moli Energy

    Winnipeg, MB
    1 hour ago
  • $32.97 - $48.6 per hour

    Working Title: Evaluation and Insights Analyst Job Class: Research Analysis Specialist Senior Agency: Department of Employment and Economic Development Job ID: 96431Location...  ...analyses, developing evaluation methods or models, and turning data findings into... 
    Senior
    Hourly pay
    Remplacement
    Long term contract
    Temporary work
    Part time
    Work at office
    Local area
    Remote work
    Shift work
    Day shift

    Minnesota Direct Care and Treatment

    Winnipeg, MB
    1 hour ago
  • Analyze fraud traffic and build detection models as a Senior II Data Scientist at Akamai. Work across teams while improving security product accuracy in a flexible FlexBase environment.You will address customer challenges through large-scale data science and production-ready... 
    Senior
    Flexible hours

    Akamai Career Site

    Winnipeg, MB
    1 hour ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Research Scientist, Model Evaluation. Be the first to apply!