Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Research Scientist, Model Evaluation

Full-time

cohere

Who are we?Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what’s best for our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products. Join us on our mission and shape the future! Why this role?Evaluation is critical to making progress in scaling intelligence. As models continue to become superhuman in many real-world use cases, we must continue to develop new evaluation techniques that accurately reflect what models are already capable of, as well as set the agenda for what future models should be capable of. In this role, you are responsible for creating these next-generation evaluation methods and infrastructure to measure LLM progress. As a Senior Research Scientist, Model Evaluation, you will:

  • Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish.
  • Work on highly cross-functional teams to translate model feedback into trustworthy, repeatable evaluations.
  • Conduct research to advance the state-of-the-art in LLM evaluation methods, including training LLM judges; refining LLM-based data synthesis pipelines; and improving evaluation efficiency.
  • Build scalable and reusable tools for digging into model performance.
You may be a good fit if:
  • You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities.
  • You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality.
  • You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about.
  • You have strong software engineering skills.
If some of the above doesn’t line up perfectly with your experience, we still encourage you to apply! We value and celebrate diversity and strive to create an inclusive work environment for all. We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an [Accommodations Request Form]( and we will work together to meet your needs. Full-Time Employees at Cohere enjoy these Perks:
  • ? An open and inclusive culture and work environment
  • ?‍? Work closely with a team on the cutting edge of AI research
  • ? Weekly lunch stipend, in-office lunches & snacks
  • ? Full health and dental benefits, including a separate budget to take care of your mental health
  • ? 100% Parental Leave top-up for up to 6 months
  • ? Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
  • ? Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
  • ✈️ 6 weeks of vacation (30 working days!)
Vacancy posted 8 days ago
Similar jobs that could be interesting for youBased on the Senior Research Scientist, Model Evaluation in Toronto, ON vacancy
  • $250k per year

     ...build cutting-edge foundation AI models and end-to-end products that...  .... Cohere is a team of researchers, engineers, designers, and more...  ...Join us! Role Overview: Evaluation is critical to making progress...  ...evaluations. Conduct research to advance the state-of-the-art... 
    Senior
    Full time
    Work at office
    Local area
    Remote work
    Home office

    Cohere

    Toronto, ON
    6 days ago
  • $100 - $150 per hour

     ...technical talent with leading AI research labs. Headquartered in San...  .... Position: Data Scientist Talent Network Type:...  ...data analyses , statistical modeling work , machine learning pipelines...  ...and technical reports. Evaluate AI-generated or human-created... 
    Senior
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    24 days ago
  •  ...Job Description: Research Scientist Intern Are you a curious and innovative thinker with experience...  ...retrieval, and AI agent design and evaluation. What does Thomson Reuters Labs do? We...  ...including training and evaluating language models and building AI Agents. Are versed... 
    Suggested
    Long term contract
    Full time
    Internship
    Local area
    Flexible hours

    Thomson Reuters

    Toronto, ON
    8 days ago
  •  ...in the space. We are looking for a Senior Data Scientist with a good blend of data analytics background...  ..., practical experience in Operation research strategies and Pricing Analytics within...  .... Develop and implement predictive models and optimization algorithms to improve... 
    Senior
    Full time
    Local area
    Remote work

    Tiger Analytics Inc.

    Toronto, ON
    more than 2 months ago
  • $140k - $250k per year

     ...variety of positions, ranging from junior to senior levels, and will evaluate your application in its entirety. Layer 6 is the AI research centre of excellence for TD Bank Group. We...  .... As a Research Machine Learning Scientist, you will Join a world-class team of machine... 
    Senior
    Temporary work
    Work from home

    TD Bank

    Toronto, ON
    5 days ago
  • $140k - $175k per year

     ...Thomson Reuters Labs is seeking a Lead Research Engineer to lead the design and delivery...  ...for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable...  ...tools, collaborating closely with research scientists, engineering teams, and product... 
    Full time
    Internship
    Work at office
    Local area
    Flexible hours

    Thomson Reuters

    Toronto, ON
    8 days ago
  • $81.6k - $115.2k per year

     ...provide you more specific details for this role. Job Description: The Model Validation (MV) group in Enterprise Risk, Risk Management is...  ...of all analytical models used for TD. The position reports to Senior Manager, Insurance Model Validation. Detailed accountabilities include... 
    Senior
    Full time
    Work from home

    TD

    Toronto, ON
    3 days ago
  •  ...value.  Reporting to the Director, Data Science, as a Senior Data Scientist you will focus on supporting Canadian business units in...  ...to implementation  Focus on feature engineering, model training, model evaluation, and prompt engineering for LLMs  GenAI & Agentic AI... 
    Senior
    Contract work
    Work at office
    Work from home

    Sun Life Assurance Company of Canada

    Toronto, ON
    5 days ago
  • $100k - $145k per year

     ...at the forefront of applied research in an industry setting? TR Labs...  ...Reuters. We are seeking scientists with a passion for solving problems...  .... About the Role As a Senior Applied Scientist, you will lead...  ...technical judgment and evaluation expertise ~ Excellent communication... 
    Senior
    Full time
    Work at office
    Local area
    Flexible hours
    2 days per week
    3 days per week

    Thomson Reuters

    Toronto, ON
    8 days ago
  • $129.4k - $179.4k per year

     ...suitable for experienced data scientists who also have strong communications...  .... You will support Global AI Model Validation and Governances...  ...activities. ~ When appropriate evaluate whether AI models are reliable...  ...AI/ML model development AI/ML research model validation model risk... 
    Senior
    Long term contract
    Full time
    Temporary work
    Local area
    Flexible hours

    Manulife

    Toronto, ON
    1 day ago
  • $75k - $100k per year

     ...organization, we want to hear from you! We are seeking a Data Scientist, Risk Analytics & Modeling, to work under the direction of the Director - Risk...  ...and deliver robust risk machine learning and statistical models that support better credit decision-making, portfolio... 
    Full time
    Internship
    Work at office
    Flexible hours

    Financeit

    Toronto, ON
    a month ago
  • $85.5k - $185k per year

     ...s most critical AML and Fraud models. This role is designed for an...  ...of complex models, influence senior stakeholders, and ensure adherence...  ...management, execution, evaluation and sustainment of initiatives...  ...quality of execution. Leads the research and development for validation... 
    Senior
    Contract work
    Part time
    Shift work

    Bank of Montreal

    Toronto, ON
    5 days ago
  •  ...We are seeking an experienced Senior Data Architect/Modeller to lead the review, analysis, and design of...  ...governance policies. Conduct technical research to integrate industry best practices...  ...: Expert analytical skills to evaluate source data structures and develop detailed... 
    Senior
    Contract work
    Remote work
    Flexible hours

    Randstad Canada

    Toronto, ON
    5 days ago
  • $69k - $114k per year

     ...Job Type:  Permanent   Work Model:  Hybrid   Reference code: 135144 Primary Location:  Toronto, ON All Available Locations:  Toronto, ON; Calgary, AB; Halifax, NS; Montreal, QC; Vancouver, BC   Our Purpose   At Deloitte, our Purpose is to make an impact... 
    Senior
    Permanent employment
    Work at office
    Flexible hours
    Shift work
    Early shift

    Deloitte

    Toronto, ON
    6 days ago
  • $55 - $80 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...state-of-the-art large language models . Create high-quality data by developing...  ...problems in your domain. Evaluate and improve AI model performance with a... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    a month ago
  •  ...What We Do We build AI models to enable smaller, faster, and more successful clinical...  ...outcomes. Our multidisciplinary team of AI scientists, clinicians, and business operators is on...  ...the state of the art Contribute to research publications and present findings at... 
    Senior

    Altis Labs, Inc.

    Toronto, ON
    5 days ago
  •  ...anywhere.  Summary If you love...  Predictive Modeling & Machine Learning: Building and deploying sophisticated models to forecast network demand, predict failures,...  ....  Mentor and Lead: Guide junior data scientists, establish best practices for modeling and analysis... 
    Senior
    Full time
    Work at office
    3 days per week

    Bell

    Toronto, ON
    5 days ago
  • $100k - $115k per year

     ...do next. When M1 retention dips or subscriber activations spike, you'll know whether to reach for a cohort breakdown, a segmentation model, or a conversation with the product analyst who owns that surface — and you'll know which one actually answers the question. How will... 
    Senior
    Long term contract
    Full time
    Flexible hours

    Spin Master

    Toronto, ON
    8 days ago
  • $120 - $180 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ..., and Jack Dorsey . Position: Senior Data Scientist and Data Engineering Experts...  ...exploratory analyses , statistical modeling , machine learning pipelines , ETL... 
    Senior
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    8 days ago
  • $75k - $85k per year

     ...marketing solutions through the network’s Creative and CXM specialisms. The Senior Data Scientist, will be part of a cross-functional team, specifically focused on developing mixed marketing models for our clients. The ideal candidate is both well versed in the data modeling... 
    Senior
    Permanent employment
    Full time
    Contract work
    Internship

    dentsu

    Toronto, ON
    8 days ago
  • $104.73k - $150k per year

     ...complex SCM business problems into clear technical requirements for data scientists and engineers. Pipeline & Solution Facilitation: Assist in the design and facilitation of data pipelines and AI models, working hands-on with the team to ensure solutions are tailored to... 
    Senior
    Temporary work
    Local area
    Remote work
    Worldwide
    Shift work

    Celestica International LP

    Toronto, ON
    11 days ago
  • $100k - $145k per year

     ...Thomson Reuters Labs. We are seeking a Senior Research Engineer who will bring expertise in AI...  ...Experience collaborating with applied data scientists to translate research work in production...  ...What’s in it For You? Hybrid Work Model: We’ve adopted a flexible hybrid working... 
    Senior
    Full time
    Work at office
    Local area
    Remote work
    Flexible hours
    2 days per week
    3 days per week

    Thomson Reuters

    Toronto, ON
    8 days ago
  • $136k - $170k per year

     ...domestically and abroad. As the Staff Data Scientist for AV, you will be the science...  ...designing an experiment or building a causal model as you are walking a senior leader through a recommendation. In...  ...in Economics, Statistics, Operations Research, Applied Math, Computer Science or a... 
    Senior
    Hourly pay
    Full time
    Work at office
    Flexible hours
    3 days per week

    Lyft

    Toronto, ON
    1 day ago
  • $120k - $150k per year

     ...Position Overview We are looking for a Senior-Level Data Scientist to join our Fraud Detection team —...  ...equally comfortable building production ML models and getting hands-on with individual...  ..., SaaS, marketplace risk, or security research). ~ Solid understanding of both... 
    Senior
    Full time

    DataVisor

    Toronto, ON
    25 days ago
  • $170k - $200k per year

     ...S'more about the team We are hiring a Senior Data Scientist in our Customer Value Forecasting Squad...  ...to help build scalable machine learning models and predictive frameworks that drive value...  ...machine learning lifecycle, including research, feature formulation, model development... 
    Senior
    Full time
    Work at office
    Remote work
    Work from home
    Flexible hours
    2 days per week
    3 days per week

    HelloFresh

    Toronto, ON
    2 days ago
  • $81.6k - $115.2k per year

     ...specific details for this role. Job Description: Department Overview: Model Risk Management (MRM) group is responsible for establishing and...  ...contribution to TD and your own success. Summary : The Senior Analyst, Model Risk Governance, is responsible for managing various... 
    Senior
    Full time
    Work from home

    TD

    Toronto, ON
    4 days ago
  • $85k - $110k per year

     ...Reach3 Insights – Senior Research Consultant  Rival Group, comprised of Rival Technologies, Reach3...  ...working in a flexible hybrid and remote model. For this role, our priority is hiring...  ...team to ensure a fair and personalized evaluation process. If you don’t see yourself... 
    Senior
    Internship
    Manual labor
    Work at office
    Immediate start
    Remote work
    Home office
    Flexible hours
    Shift work

    Reach3 Insights

    Toronto, ON
    2 days ago
  • $81.6k - $115.2k per year

     ...Description: Department Overview: The Model Validation (MV) group is a centralized...  ...By ensuring an objective and independent evaluation of models, the model validation function...  ...Description: The position reports to Senior Manager, Retail Credit Risk, Model Validation... 
    Senior
    Work from home

    TD Bank

    Toronto, ON
    5 days ago
  • $150 per hour

     ...technical talent with leading AI research labs. Headquartered in San...  ...that challenge frontier models, ensuring they are solved in...  ...attempts. Focus on inference, evaluation, and systems work rather...  ...research engineer , research scientist , or applied scientist .... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    24 days ago
  • $110k - $125k per year

     ...best of both worlds with our hybrid work model, allowing you to balance remote work...  ...celebrate your contributions. The Job: Senior Data Scientist Reporting directly to the Senior...  ..., forecasting, segmentation, and model evaluation techniques.  Experience with feature... 
    Senior
    Temporary work
    Work at office
    Remote work

    Momentum Financial Services Group

    Toronto, ON
    a month ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Research Scientist, Model Evaluation. Be the first to apply!