Applied Ai Evaluation Researcher
Kids Help Phone
Applied AI evaluation and user-centred research define this full-time contract role with Kids Help Phone in Toronto, using qualitative and quantitative methods.As an Applied AI Evaluation & Product Researcher, you will evaluate a youth-facing generative AI product. You will develop measurement strategies, conduct UX research, assess risks, and translate evidence into recommendations. The hybrid contract requires working one or two days weekly in the workplace.Key Responsibilities:
Develop evaluation strategies from early testing through post-launch monitoringCreate evaluation matrices linking objectives, indicators, data, and decisionsConduct surveys, usability testing, journey mapping, and co-design activitiesAssess unintended outcomes, bias, safety, privacy, and equity risksSynthesize findings into recommendations, reports, briefings, and decision materialsRequirements:
PhD or equivalent Master’s experience in a related disciplineMinimum five years evaluating generative AI or digital servicesKnowledge of responsible AI, safety, risk, or technology governanceStrong qualitative and quantitative research experienceExperience translating findings into product or implementation recommendationsBring evaluation strategy, UX research, and responsible AI expertise to Kids Help Phone’s youth-focused generative AI initiative.
#J-18808-Ljbffr
Develop evaluation strategies from early testing through post-launch monitoringCreate evaluation matrices linking objectives, indicators, data, and decisionsConduct surveys, usability testing, journey mapping, and co-design activitiesAssess unintended outcomes, bias, safety, privacy, and equity risksSynthesize findings into recommendations, reports, briefings, and decision materialsRequirements:
PhD or equivalent Master’s experience in a related disciplineMinimum five years evaluating generative AI or digital servicesKnowledge of responsible AI, safety, risk, or technology governanceStrong qualitative and quantitative research experienceExperience translating findings into product or implementation recommendationsBring evaluation strategy, UX research, and responsible AI expertise to Kids Help Phone’s youth-focused generative AI initiative.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Applied Ai Evaluation Researcher in Winnipeg, MB vacancy
$95k - $110k per year
...Applied AI Evaluation & Product Researcher (Contract) Applied Research FT Contract Toronto, Toronto, ON, CA Salary Range: $95,000.00 To $110,000.00 Annually Job title Applied AI Evaluation & Product Researcher (12 Month Contract) Department Reports to Director...SuggestedFull timeContract workLive In2 days per week1 day per week- Cohere is seeking a Senior Research Scientist focused on AI evaluation methods to support our innovative foundation models. This remote or on-site role emphasizes creating evaluation benchmarks and tools that enhance LLM capabilities. As a Senior Research Scientist at Cohere...SuggestedRemote work
- ...you can be proud to be with.The AI/ML team within eBay’s Selling... ...ML engineering — moving research from theoretical grounding through... ...eBay’s global marketplace.As an Applied Researcher, you will design... ...actionable listing advice.Developing evaluation frameworks for generative AI...SuggestedManual laborImmediate startWorldwide
$140k - $175k per year
...Thomson Reuters Labs is seeking a Lead Research Engineer to lead the design and delivery... ...software for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable... ..., interpreting evaluation results, and applying findings to engineering decisions....SuggestedInternshipWork at officeLocal areaFlexible hours- ...CIBC in Toronto is seeking a Quantum Machine Learning Research Scientist to drive applied research and design quantum algorithms for banking use cases such as fraud and optimization. You will collaborate with AI researchers, engineers and business stakeholders to translate...Suggested
- ...unique opportunity to contribute to cutting-edge AI research in Trust and Safety, with a particular focus on understanding and evaluating the risks of autonomous AI systems. You’ll... ...-renowned researchers drive fundamental and applied research at the University of Alberta (and...Casual workWork at officeFlexible hoursShift work
- Alberta Machine Intelligence Institute (Amii) in Edmonton seeks an Applied Research Scientist to advance the safety of autonomous AI systems. You will investigate agentic AI safety, develop rigorous evaluation methods, and translate research into practical tools supporting...
$173k - $197.4k per year
...machine learning, conversational AI and design thinking into our... ...to building world-class applied science and engineering teams... ...work touches every aspect of the research life cycle, from partnering with... ...from design through training, evaluation, validation, and implementation...Work at officeWork from home- lululemon is seeking a Senior Director, AI/ML Research and Science to advance the company's AI capabilities through research, development, and... ...impact. You will lead a lean team of AI researchers and applied scientists, balance long-range exploration with near-term business...
- eBay’s AI/ML team within the Selling organization is seeking an Applied Researcher to design and advance our research agenda across machine learning and generative AI. You will contribute to price guidance modeling and the expanding selling-agent platform, translating research...
$180k - $230k per year
...complex capital projects. The AI buildout runs on physical... ...the world, including Crusoe, Applied Digital, and NTT. We raised a... ...projects. Continuous improvement: Evaluate models repeatedly and... ...features, inspired by latest research publications. What you'll doOwn...For contractorsInternshipWork at office- Applied AI Engineer### Applied AI EngineerApply: Remote, Canada: Poland, Virtual:... ...AI runtime, retrieval, guardrails, and evaluation that make the platform's AI features work... ...This Role Is Different*** **Applied, not research.** We use proven commercial models and win...Full timeRemote work
- ...Technology team is seeking two highly experienced Applied AI Engineers to design, build, and operate an... ...-on role focused on production systems, not research or demos, requiring deep understanding of GenAI failure modes, evaluation, and governance. The engineers will...InternshipWork at office
- ...the role and the team In the role of Applied Researcher 3 within the Global Focus Categories on... ...develop new computer vision and multi-input AI solutions. These solutions will address... ...will research, build, train, fine‑tune, evaluate, and deploy models and AI systems at...For contractorsWork at officeImmediate start
- eBay Inc. is seeking a Staff Applied Researcher to advance visual and multimodal AI across the Global Focus Categories. You will research, train, fine-tune, evaluate, and deploy foundation-model-based systems at scale, collaborating with product, engineering and analytics teams...
- ...Directly builds AI-enabled security workflows with OpenAI APIs... ...OpenAI is hiring a Cybersecurity Applied AI Architect to partner with... ...requirements, workflow integrations, evaluation methods, and security... ...feedback to Product, Engineering, Research, Security, and GTM teams....
- We’re seeking a Senior Applied AI Engineer to serve as a forward-deployed AI expert across... ...teamsYou’ll turn the latest machine learning research into practical, production-ready... ...end to end, from framing the question to evaluating model performance and deciding what’s worth...InternshipFlexible hours
- ...ROLE Floyd is PSP's in-house AI platform — a custom analyst... ...built workflows that turn complex research questions into structured,... ...role puts you at the frontier of applied AI inside an institutional... ...tuning, and LLM observability and evaluation toolingExpert in Python (...Summer workWork at officeLocal areaRemote work1 day per week
- ...team dedicated to accelerating Generative AI adoption within a major organization. You... ...enterprise connectors Performing testing, evaluation, and performance optimization of AI... ...Your future team:You will work within an Applied AI team partnering closely with business and...Work at officeRemote work3 days per week
- About Mundo AI Mundo AI builds multimodal data infrastructure for frontier research labs and Fortune 100 companies. Our work covers... ...Role We’re looking for an applied researcher to turn open-ended... ...failure modes, develop novel evaluation methodologies, and design datasets...Long term contractTemporary workWork at office
- Software Developer - Applied AIRemote, Ontario Technology Remote... ...J is the leading generative AI solution for tax professionals... ...building industry-leading tax research software that helps accountants... ...hands of tax professionals Evaluation and data-driven decisions:...Long term contractFull timeImmediate startRemote work
$150k - $180k per year
...CivCheck, we created Guided AI Plan Review™ (GPR) - a... ...offering. We’re seeking a Senior Applied AI Engineer to serve as a forward... ...the latest machine learning research into practical, production‑... ...from framing the question to evaluating model performance and deciding...Full timeContract workPart timeInternshipRemote work- ...all applications and next steps. Our partner is looking for an Applied AI Engineer based in Canada. As an Applied AI Engineer, you will... ..., agent orchestration, retrieval-augmented generation (RAG), evaluation frameworks, and safety controls. Working with proven commercial...Long term contract
- ...build a more efficient and enjoyable world. Role overviewAs an Applied AI Scientist, you design, build, and ship AI solutions that solve... ...)Turn experiments into production-ready components (evaluation, monitoring, iteration)Coach colleagues on applied AI best practices...Work at officeFlexible hours
$140k - $160k per year
...worldwide. And we'd love for you to join us! About the job - Applied AI EngineerJoin the Engineering Team, where you'll help build our... ...or model change actually made things better. Raise the bar for evaluation across the team - judge design, offline and online metrics, and...Manual laborWork at officeWorldwide1 day per week$146.83k - $192.72k per year
...team The Enterprise Data & AI team is a strategic and operational... ...from the very beginning. By applying AI to critical business... ..., and translating advanced AI research into scalable, production-ready... ...experimentation frameworks and evaluation approaches to measure business...Permanent employmentFull timePart timeLocal area$149.3k - $249.3k per year
...the computer vision area who is aspired to apply your skills to bring value to millions of... ...Participate in the design, development, evaluation, deployment and updating of data-driven models for computer vision applications.Research and implement the state-of-the-art computer...- ...building a global community of experts to evaluate, challenge and improve advanced AI systems. We are looking for university students, graduates, researchers and experienced professionals with... ...and potential risks – Apply evaluation guidelines consistently...Hourly payFor contractorsRemote workFlexible hours
- ...leading security-first enterprise AI company. We build cutting-edge... .... Cohere is a team of researchers, engineers, designers, and more... ...Seoul. Join us!Role Overview: Evaluation is critical to making progress... ...experience, we still encourage you to apply.We strive to create an...Full timeWork at officeLocal areaRemote workHome office
- Our client is an energy solutions provider. Our AI-enabled platform supports operations, tracking and decision-making to optimize large... ...Python services, PostgreSQL, Snowflake and AWS. We're looking for an applied AI engineer to build AI with real, measurable benefit across our...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Applied Ai Evaluation Researcher. Be the first to apply!

