Senior Research Scientist, Model Evaluation
cohere
Who are we?Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what’s best for our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products. Join us on our mission and shape the future! Why this role?Evaluation is critical to making progress in scaling intelligence. As models continue to become superhuman in many real-world use cases, we must continue to develop new evaluation techniques that accurately reflect what models are already capable of, as well as set the agenda for what future models should be capable of. In this role, you are responsible for creating these next-generation evaluation methods and infrastructure to measure LLM progress. As a Senior Research Scientist, Model Evaluation, you will:
- Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish.
- Work on highly cross-functional teams to translate model feedback into trustworthy, repeatable evaluations.
- Conduct research to advance the state-of-the-art in LLM evaluation methods, including training LLM judges; refining LLM-based data synthesis pipelines; and improving evaluation efficiency.
- Build scalable and reusable tools for digging into model performance.
- You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities.
- You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality.
- You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about.
- You have strong software engineering skills.
- ? An open and inclusive culture and work environment
- ?? Work closely with a team on the cutting edge of AI research
- ? Weekly lunch stipend, in-office lunches & snacks
- ? Full health and dental benefits, including a separate budget to take care of your mental health
- ? 100% Parental Leave top-up for up to 6 months
- ? Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
- ? Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
- ✈️ 6 weeks of vacation (30 working days!)
$250k per year
...build cutting-edge foundation AI models and end-to-end products that... .... Cohere is a team of researchers, engineers, designers, and more... ...Join us! Role Overview: Evaluation is critical to making progress... ...evaluations. Conduct research to advance the state-of-the-art...SeniorFull timeWork at officeLocal areaRemote workHome office$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in San... .... Position: Data Scientist Talent Network Type:... ...data analyses , statistical modeling work , machine learning pipelines... ...and technical reports. Evaluate AI-generated or human-created...SeniorRemote jobContract workSummer work- ...Job Description: Research Scientist Intern Are you a curious and innovative thinker with experience... ...retrieval, and AI agent design and evaluation. What does Thomson Reuters Labs do? We... ...including training and evaluating language models and building AI Agents. Are versed...SuggestedLong term contractFull timeInternshipLocal areaFlexible hours
- ...in the space. We are looking for a Senior Data Scientist with a good blend of data analytics background... ..., practical experience in Operation research strategies and Pricing Analytics within... .... Develop and implement predictive models and optimization algorithms to improve...SeniorFull timeLocal areaRemote work
$140k - $250k per year
...variety of positions, ranging from junior to senior levels, and will evaluate your application in its entirety. Layer 6 is the AI research centre of excellence for TD Bank Group. We... .... As a Research Machine Learning Scientist, you will Join a world-class team of machine...SeniorTemporary workWork from home$140k - $175k per year
...Thomson Reuters Labs is seeking a Lead Research Engineer to lead the design and delivery... ...for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable... ...tools, collaborating closely with research scientists, engineering teams, and product...Full timeInternshipWork at officeLocal areaFlexible hours$81.6k - $115.2k per year
...provide you more specific details for this role. Job Description: The Model Validation (MV) group in Enterprise Risk, Risk Management is... ...of all analytical models used for TD. The position reports to Senior Manager, Insurance Model Validation. Detailed accountabilities include...SeniorFull timeWork from home- ...value. Reporting to the Director, Data Science, as a Senior Data Scientist you will focus on supporting Canadian business units in... ...to implementation Focus on feature engineering, model training, model evaluation, and prompt engineering for LLMs GenAI & Agentic AI...SeniorContract workWork at officeWork from home
$100k - $145k per year
...at the forefront of applied research in an industry setting? TR Labs... ...Reuters. We are seeking scientists with a passion for solving problems... .... About the Role As a Senior Applied Scientist, you will lead... ...technical judgment and evaluation expertise ~ Excellent communication...SeniorFull timeWork at officeLocal areaFlexible hours2 days per week3 days per week$129.4k - $179.4k per year
...suitable for experienced data scientists who also have strong communications... .... You will support Global AI Model Validation and Governances... ...activities. ~ When appropriate evaluate whether AI models are reliable... ...AI/ML model development AI/ML research model validation model risk...SeniorLong term contractFull timeTemporary workLocal areaFlexible hours$75k - $100k per year
...organization, we want to hear from you! We are seeking a Data Scientist, Risk Analytics & Modeling, to work under the direction of the Director - Risk... ...and deliver robust risk machine learning and statistical models that support better credit decision-making, portfolio...Full timeInternshipWork at officeFlexible hours$85.5k - $185k per year
...s most critical AML and Fraud models. This role is designed for an... ...of complex models, influence senior stakeholders, and ensure adherence... ...management, execution, evaluation and sustainment of initiatives... ...quality of execution. Leads the research and development for validation...SeniorContract workPart timeShift work- ...We are seeking an experienced Senior Data Architect/Modeller to lead the review, analysis, and design of... ...governance policies. Conduct technical research to integrate industry best practices... ...: Expert analytical skills to evaluate source data structures and develop detailed...SeniorContract workRemote workFlexible hours
$69k - $114k per year
...Job Type: Permanent Work Model: Hybrid Reference code: 135144 Primary Location: Toronto, ON All Available Locations: Toronto, ON; Calgary, AB; Halifax, NS; Montreal, QC; Vancouver, BC Our Purpose At Deloitte, our Purpose is to make an impact...SeniorPermanent employmentWork at officeFlexible hoursShift workEarly shift$55 - $80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ...state-of-the-art large language models . Create high-quality data by developing... ...problems in your domain. Evaluate and improve AI model performance with a...Remote jobContract workSummer work- ...What We Do We build AI models to enable smaller, faster, and more successful clinical... ...outcomes. Our multidisciplinary team of AI scientists, clinicians, and business operators is on... ...the state of the art Contribute to research publications and present findings at...Senior
- ...anywhere. Summary If you love... Predictive Modeling & Machine Learning: Building and deploying sophisticated models to forecast network demand, predict failures,... .... Mentor and Lead: Guide junior data scientists, establish best practices for modeling and analysis...SeniorFull timeWork at office3 days per week
$100k - $115k per year
...do next. When M1 retention dips or subscriber activations spike, you'll know whether to reach for a cohort breakdown, a segmentation model, or a conversation with the product analyst who owns that surface — and you'll know which one actually answers the question. How will...SeniorLong term contractFull timeFlexible hours$120 - $180 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco,... ..., and Jack Dorsey . Position: Senior Data Scientist and Data Engineering Experts... ...exploratory analyses , statistical modeling , machine learning pipelines , ETL...SeniorRemote jobContract workSummer work$75k - $85k per year
...marketing solutions through the network’s Creative and CXM specialisms. The Senior Data Scientist, will be part of a cross-functional team, specifically focused on developing mixed marketing models for our clients. The ideal candidate is both well versed in the data modeling...SeniorPermanent employmentFull timeContract workInternship$104.73k - $150k per year
...complex SCM business problems into clear technical requirements for data scientists and engineers. Pipeline & Solution Facilitation: Assist in the design and facilitation of data pipelines and AI models, working hands-on with the team to ensure solutions are tailored to...SeniorTemporary workLocal areaRemote workWorldwideShift work$100k - $145k per year
...Thomson Reuters Labs. We are seeking a Senior Research Engineer who will bring expertise in AI... ...Experience collaborating with applied data scientists to translate research work in production... ...What’s in it For You? Hybrid Work Model: We’ve adopted a flexible hybrid working...SeniorFull timeWork at officeLocal areaRemote workFlexible hours2 days per week3 days per week$136k - $170k per year
...domestically and abroad. As the Staff Data Scientist for AV, you will be the science... ...designing an experiment or building a causal model as you are walking a senior leader through a recommendation. In... ...in Economics, Statistics, Operations Research, Applied Math, Computer Science or a...SeniorHourly payFull timeWork at officeFlexible hours3 days per week$120k - $150k per year
...Position Overview We are looking for a Senior-Level Data Scientist to join our Fraud Detection team —... ...equally comfortable building production ML models and getting hands-on with individual... ..., SaaS, marketplace risk, or security research). ~ Solid understanding of both...SeniorFull time$170k - $200k per year
...S'more about the team We are hiring a Senior Data Scientist in our Customer Value Forecasting Squad... ...to help build scalable machine learning models and predictive frameworks that drive value... ...machine learning lifecycle, including research, feature formulation, model development...SeniorFull timeWork at officeRemote workWork from homeFlexible hours2 days per week3 days per week$81.6k - $115.2k per year
...specific details for this role. Job Description: Department Overview: Model Risk Management (MRM) group is responsible for establishing and... ...contribution to TD and your own success. Summary : The Senior Analyst, Model Risk Governance, is responsible for managing various...SeniorFull timeWork from home$85k - $110k per year
...Reach3 Insights – Senior Research Consultant Rival Group, comprised of Rival Technologies, Reach3... ...working in a flexible hybrid and remote model. For this role, our priority is hiring... ...team to ensure a fair and personalized evaluation process. If you don’t see yourself...SeniorInternshipManual laborWork at officeImmediate startRemote workHome officeFlexible hoursShift work$81.6k - $115.2k per year
...Description: Department Overview: The Model Validation (MV) group is a centralized... ...By ensuring an objective and independent evaluation of models, the model validation function... ...Description: The position reports to Senior Manager, Retail Credit Risk, Model Validation...SeniorWork from home$150 per hour
...technical talent with leading AI research labs. Headquartered in San... ...that challenge frontier models, ensuring they are solved in... ...attempts. Focus on inference, evaluation, and systems work rather... ...research engineer , research scientist , or applied scientist ....Remote jobContract workSummer work$110k - $125k per year
...best of both worlds with our hybrid work model, allowing you to balance remote work... ...celebrate your contributions. The Job: Senior Data Scientist Reporting directly to the Senior... ..., forecasting, segmentation, and model evaluation techniques. Experience with feature...SeniorTemporary workWork at officeRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Research Scientist, Model Evaluation. Be the first to apply!
- consultant scientifique Toronto, ON
- research scientist - biology Toronto, ON
- research scientist Toronto, ON
- communication scientifique Toronto, ON
- r&d research scientist Toronto, ON
- scientifique biologie Toronto, ON
- hat model Toronto, ON
- cat modeling Toronto, ON
- teen modeling Toronto, ON
- energy modelling Toronto, ON
