Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Research Scientist in AI Evaluation

Cohere Inc.

Cohere is seeking a Senior Research Scientist focused on AI evaluation methods to support our innovative foundation models. This remote or on-site role emphasizes creating evaluation benchmarks and tools that enhance LLM capabilities. As a Senior Research Scientist at Cohere, you will lead the development of next-generation evaluation techniques critical for AI model scaling. Collaborate with cross-functional teams to refine model feedback and improve evaluation processes. Your contributions will be pivotal in advancing the standards for assessing AI performance and capability alignment. Key Responsibilities:

  • Create ambitious new evaluation benchmarks for AI models
  • Collaborate cross-functionally for trustworthy evaluations
  • Conduct research on LLM evaluation methods
  • Develop scalable tools for performance analysis
Requirements:
  • Experience in building AI prototypes and evaluations
  • Proficiency in data review for quality assurance
  • Commitment to rigorous measurement practices
  • Strong software engineering skills
Drive innovation in AI evaluation techniques with your expertise at Cohere and significantly impact model performance. #J-18808-Ljbffr

Vacancy posted 16 hours ago
Similar jobs that could be interesting for youBased on the Research Scientist in AI Evaluation in Montréal, QC vacancy
  •  ...numériques et des partenariats clients. En tant qu'ingénieur QA / Évaluation IA, vous rejoindrez une équipe dynamique et expérimentée dans...  ...digital experiences and client partnerships. As a QA / AI Evaluation Engineer, you will join a highly motivated and experienced... 
    Suggested
    Full time
    Apprenticeship
    Immediate start

    Appnovation Technologies

    Montréal, QC
    a month ago
  •  ...work. Today, ServiceNow is the AI control tower for business...  ...essai. ####### As a Staff Researcher, you will work on research questions...  ...menaces, confidentialité, évaluation des agents Expérience de...  ...detection, privacy, agent evaluation  Experience with AI systems... 
    Suggested
    Work at office
    Immediate start
    Remote work
    Flexible hours

    ServiceNow

    Montréal, QC
    5 days ago
  •  ...Our formula for success is to hire exceptional people, encourage their ideas and reward their results. As an  AI/ML Research Intern , you will be an integral member of a team of experienced technologists, quantitative researchers, and traders. You will collaborate closely... 
    Suggested
    Summer work
    Internship
    Work at office
    Immediate start
    Day shift

    Drw Inc

    Montréal, QC
    more than 2 months ago
  •  ...work. Today, ServiceNow is the AI control tower for business...  ...préentraînement, ajustement fin et évaluation. Notre objectif est d’...  ...AI Model Training team leads research and engineering across the full...  ...Collaborate daily with research scientists, engineers, and product managers... 
    Suggested
    Daily paid
    Full time
    Apprenticeship
    Internship
    Work at office
    Immediate start
    Remote work
    Worldwide
    Flexible hours

    ServiceNow

    Montréal, QC
    28 days ago
  • $70 per hour

     ...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.  Participation is...  ...evaluate clinical EHR vignettes for a medical AI evaluation program. General and internal medicine are our... 
    Suggested
    Hourly pay
    Permanent employment
    Part time
    10 hours per week

    Mindrift

    Montréal, QC
    2 days ago
  •  ...AI Evaluation Specialist Role Type: Contractor Location: Remote — US, Canada, UK, Ireland, Australia, New Zealand We are seeking experienced AI Evaluation Specialists to assess and improve the quality of AI assistant outputs for an enterprise AI training project... 
    Remote job
    For contractors

    YO AI Labs

    Montréal, QC
    13 days ago
  •  ...nos clients. Ton quotidien Comprendre le besoin d'un client, évaluer le ROI et définir un scope de POC qui se tient Réaliser des...  ...fonctionner une idée Profil idéal Un mix entre un.e scientifique AI et un.e vendeur.euse : tu vois les possibilités techniques ET tu... 
    Daily paid
    Work at office
    Flexible hours
    Night shift

    Vooban

    Montréal, QC
    a month ago
  •  ...En tant qu'AI Scientist, tu n'as jamais cru que l'IA se résumait à appeler une API. Plutôt que d'assembler des modèles préfabriqués en espérant que ça tienne, tu comprends ce qui se passe sous le capot et tu choisis le bon modèle (ou le bon enchaînement) pour le vrai problème... 
    Work at office
    Flexible hours

    Vooban

    Montréal, QC
    a month ago
  •  ...This further enables us to detect greenhouse gas emitters and to visualize and quantify their emissions. We are looking for a data scientist to join the Science and Systems team. You will focus on applying computer vision and machine learning techniques to automate data processing... 
    Full time
    Work at office
    Local area
    Remote work
    Flexible hours

    Ghgsat

    Montréal, QC
    2 days ago
  •  ...time with the only hardware and software AI-powered platform built for the factory floor...  ...Autonomous Manipulation Capabilities: Research, prototype, and deploy advanced robotics...  ...plans de développement personnalisés, des évaluations biannuelles et un programme de mentorat.... 
    Apprenticeship
    Internship
    Work from home
    Day shift

    Vention

    Montréal, QC
    20 days ago
  •  ...opérationnels Surveiller, évaluer et améliorer en permanence les...  ...Your team’s dynamic: The Data Scientist will leverage advanced...  ...operational systems Monitor, evaluate, and continuously improve the...  ...model governance, and responsible AI Document analytical methodologies... 
    Full time
    Apprenticeship
    Flexible hours

    Genetec

    Montréal, QC
    more than 2 months ago
  •  ...seeking a machine learning (ML) research developer to join our team working on a novel AI safety agenda. In this role, you...  ...will work closely with ML research scientists to solve difficult training and...  ...and processed by LawZero to evaluate your application for employment... 
    Full time
    Flexible hours

    LawZero

    Montréal, QC
    more than 2 months ago
  •  ...Faites votre marque au sein du groupe Fiscalité foncière à titre d'Évaluateur Agrée, Evaluation valeur foncière. Cette équipe diversifiée de professionnels élabore des stratégies fiscales sur mesure dans un contexte réglementaire complexe et dynamique. En tant que conseiller... 
    Full time
    Work at office
    Local area

    Postes Avec Expérience

    Montréal, QC
    more than 2 months ago
  •  ...sûrs dès leur conception ( safe-by-design ). Nous développons « Scientist AI », un système d'IA de pointe conçu dès le départ pour être à la...  ...les modèles de langage de pointe (LLM frontier ), nous devons évaluer de manière rigoureuse et indépendante chaque affirmation... 
    Apprenticeship
    Work at office
    Flexible hours
    Day shift

    LoiZéro

    Montréal, QC
    13 days ago
  •  ...une vocation à votre passion en rejoignant une culture qui s’épanouit face au changement.  Faites votre marque au sein du groupe Évaluation d’entreprises à titre de conseiller(ère). Cette équipe diversifiée de professionnels travaille avec nos clients sur une panoplie de... 
    Full time
    Work at office

    Postes Avec Expérience

    Montréal, QC
    10 days ago
  •  ...une vocation à votre passion en rejoignant une culture qui s’épanouit face au changement.  Faites votre marque au sein du groupe Évaluation d’entreprises à titre de directeur(trice). Cette équipe diversifiée de professionnels travaille avec nos clients sur une panoplie de... 
    Full time
    Work at office
    Local area

    Postes Avec Expérience

    Montréal, QC
    more than 2 months ago
  •  ...open for the 2027 term, with a duration of 6 or 12 months at the applicant's option. Purpose There is a crisis in academic AI/ML research. What is still worth working on? Paper submissions have doubled on the arXiv and at ML conferences, and yet many submitters cannot... 
    Full time
    Remote work

    Intuition Machines, Inc.

    Montréal, QC
    more than 2 months ago
  • $100k - $160k per year

     ...World's Leading Product Review Company as a  AI Developer ! This is your opportunity to...  ...our Rails backend and existing systems Evaluate and select AI tools, models, and services,...  ...debt and over-engineering You love doing research and comparing products. You typically... 
    Full time
    Work at office
    Flexible hours

    Rtings.com

    Montréal, QC
    more than 2 months ago
  • $150k - $160k per year

     ...Requisition Number: 106471 AI Engineer Location You will have the flexibility to work fully remotely. Insight at a Glance...  ...other data sources. Implement prompt strategies, guardrails, and evaluation loops to improve reliability, safety, and quality. Be... 
    Full time
    Local area
    Remote work

    Insight

    Montréal, QC
    12 days ago
  •  ...Analytics, Digital Engineering, Cloud, Data & AI, Experience Design, and Marketing. Our...  .... This is a production-focused role — not research or prototyping. We are looking for senior,...  ..., and who understand failure modes, evaluation practices, and governance for mission-critical... 
    Full time
    Internship
    Local area
    Worldwide
    Work visa

    Bounteous

    Montréal, QC
    more than 2 months ago
  •  ...rejoindre notre équipe Mila AI Safety Studio pour un mandat...  ...détection des préjudices et l'évaluation automatique. Vous travaillerez...  ...Montréal, Mila brings together researchers specializing in artificial...  ...passionate applied AI Safety Researcher to join our Mila AI Safety Studio... 
    Temporary work
    Apprenticeship
    Internship
    Worldwide

    Mila - Institut québécois d'intelligence artificielle

    Montréal, QC
    16 days ago
  • $60.57k - $100.96k per year

     ...do the right thing for our clients. As a Claims Analytics Data Scientist you will contribute on initiatives through clever use of data and...  ...Contributing to various projects related to incorporation of Gen AI such as determining work drivers, developing and testing KPI's,... 
    Full time
    Work at office
    Local area
    Flexible hours

    Co-operators Career

    Montréal, QC
    more than 2 months ago
  •  ...apply as well. About the role Help shape how Intact uses AI and automation across Risk Management, creating more capacity for...  ..., support, and continuous improvement. Experience in evaluating the reliability of AI results and in implementing controls related... 
    Long term contract
    Flexible hours

    Intact Financial

    Montréal, QC
    1 day ago
  •  ...independent quantitative reviews of the AI models used within BNP Paribas...  ...sales team. Identify and evaluate model risks (including...  ...with BNPP Global Market data scientists to help BNPP be compliant with...  ...models related to quantitative research in finance, such as derivative... 
    Long term contract
    Work at office
    Monday to friday
    Flexible hours
    3 days per week
    1 day per week

    BNP Paribas

    Montréal, QC
    3 days ago
  •  ...The senior clinical scientist is responsible for authoring or contributing to development of clinical and regulatory documents (synopsis,...  ...to produce high-quality scientific documents to support clinical research. You enjoy learning continuously and keeping yourself informed... 
    Remote job
    Permanent employment
    Full time
    Internship
    Work at office
    Work from home
    Flexible hours

    Indero

    Montréal, QC
    14 days ago
  •  ...skills. Our team is composed of polymer scientists, mechanical engineers and software engineers...  ...who may be a great fit.   AON3D uses AI supported tools within portions of the hiring...  ...promote fair, consistent, and objective evaluation. Certain initial screening steps may be... 
    Permanent employment
    Full time
    Internship
    Worldwide

    AON3D Careers

    Montréal, QC
    a month ago
  •  ...développement de projets majeurs en énergie, infrastructures et ressources naturelles. Nous recherchons actuellement un(e) leader senior en évaluation environnementale souhaitant jouer un rôle clé dans la réalisation et le positionnement de projets structurants au Québec et à l’... 
    Local area
    Flexible hours
    Day shift

    Hatch

    Montréal, QC
    1 day ago
  •  ...Mistral provides full-stack AI solutions: from frontier models...  ...multiple stakeholders (CEO/CTO, data scientists, and software engineers) and...  ...resource in externalising our research in production settings....  ...providing guidance on prompting, evaluation, and fine-tuning, and ensuring... 
    Full time
    Internship
    Relocation package

    Mistral

    Montréal, QC
    more than 2 months ago
  •  ...We're hiring a Senior Data Scientist to own machine-learning models...  ...if you're looking for a pure research or notebook-only role, this isn...  ...retraining, champion-challenger evaluation and promotion. Partner...  ...and rollout. Move fast with AI-assisted development -we use... 
    Contract work
    Apprenticeship
    Remote work
    Flexible hours

    Flinks

    Montréal, QC
    a month ago
  •  ...About Mistral Mistral provides full-stack AI solutions: from frontier models to developer tools, applications, and compute. We partner...  ...s internal teams, synthesizing feedback to influence product and research roadmaps. • Develop reusable assets, best practices, and... 
    Long term contract
    Full time
    Relocation package

    Mistral

    Montréal, QC
    more than 2 months ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Research Scientist in AI Evaluation. Be the first to apply!