Applied Ai Evaluation Researcher
Kids Help Phone
Applied AI evaluation and user-centred research define this full-time contract role with Kids Help Phone in Toronto, using qualitative and quantitative methods.As an Applied AI Evaluation & Product Researcher, you will evaluate a youth-facing generative AI product. You will develop measurement strategies, conduct UX research, assess risks, and translate evidence into recommendations. The hybrid contract requires working one or two days weekly in the workplace.Key Responsibilities:
Develop evaluation strategies from early testing through post-launch monitoringCreate evaluation matrices linking objectives, indicators, data, and decisionsConduct surveys, usability testing, journey mapping, and co-design activitiesAssess unintended outcomes, bias, safety, privacy, and equity risksSynthesize findings into recommendations, reports, briefings, and decision materialsRequirements:
PhD or equivalent Master’s experience in a related disciplineMinimum five years evaluating generative AI or digital servicesKnowledge of responsible AI, safety, risk, or technology governanceStrong qualitative and quantitative research experienceExperience translating findings into product or implementation recommendationsBring evaluation strategy, UX research, and responsible AI expertise to Kids Help Phone’s youth-focused generative AI initiative.
#J-18808-Ljbffr
Develop evaluation strategies from early testing through post-launch monitoringCreate evaluation matrices linking objectives, indicators, data, and decisionsConduct surveys, usability testing, journey mapping, and co-design activitiesAssess unintended outcomes, bias, safety, privacy, and equity risksSynthesize findings into recommendations, reports, briefings, and decision materialsRequirements:
PhD or equivalent Master’s experience in a related disciplineMinimum five years evaluating generative AI or digital servicesKnowledge of responsible AI, safety, risk, or technology governanceStrong qualitative and quantitative research experienceExperience translating findings into product or implementation recommendationsBring evaluation strategy, UX research, and responsible AI expertise to Kids Help Phone’s youth-focused generative AI initiative.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Applied Ai Evaluation Researcher in Winnipeg, MB vacancy
$95k - $110k per year
...Applied AI Evaluation & Product Researcher (Contract) Applied Research FT Contract Toronto, Toronto, ON, CA Salary Range: $95,000.00 To $110,000.00 Annually Job title Applied AI Evaluation & Product Researcher (12 Month Contract) Department Reports to Director...SuggestedFull timeContract workLive In2 days per week1 day per week- Cohere is seeking a Senior Research Scientist focused on AI evaluation methods to support our innovative foundation models. This remote or on-site role emphasizes creating evaluation benchmarks and tools that enhance LLM capabilities. As a Senior Research Scientist at Cohere...SuggestedRemote work
- ...you can be proud to be with.The AI/ML team within eBay’s Selling... ...ML engineering — moving research from theoretical grounding through... ...eBay’s global marketplace.As an Applied Researcher, you will design... ...actionable listing advice.Developing evaluation frameworks for generative AI...SuggestedManual laborImmediate startWorldwide
$118k - $157.5k per year
...all. About The Role And The TeamAs an Applied Researcher 1 within the Computer Vision team, you... ...You will work on training, fine‑tuning, evaluation, and deployment of models across problem... ...inference, and evaluation pipelines for modern AI systems, including workflows that...SuggestedInternshipImmediate start$140k - $175k per year
...Thomson Reuters Labs is seeking a Lead Research Engineer to lead the design and delivery... ...software for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable... ..., interpreting evaluation results, and applying findings to engineering decisions....SuggestedInternshipWork at officeLocal areaFlexible hours- ...CIBC in Toronto is seeking a Quantum Machine Learning Research Scientist to drive applied research and design quantum algorithms for banking use cases such as fraud and optimization. You will collaborate with AI researchers, engineers and business stakeholders to translate...
$40 - $80 per hour
Synthires is seeking detail-oriented Generalists to help benchmark and improve advanced AI models. You will create research questions, answers, and evaluation materials using your academic or professional expertise. The opportunity is open to individuals with experience across...Hourly payPart time10 hours per week$173k - $197.4k per year
...machine learning, conversational AI and design thinking into our... ...to building world-class applied science and engineering teams... ...work touches every aspect of the research life cycle, from partnering with... ...from design through training, evaluation, validation, and implementation...Work at officeWork from home- ...the role and the team In the role of Applied Researcher 3 within the Global Focus Categories Applied... ...new computer vision and multimodal AI solutions. These solutions will address... ...will research, build, train, fine-tune, evaluate, and deploy models and AI systems at scale...Work at officeImmediate start
- ...unique opportunity to contribute to cutting-edge AI research in Trust and Safety, with a particular focus on understanding and evaluating the risks of autonomous AI systems. You’ll... ...-renowned researchers drive fundamental and applied research at the University of Alberta (and...Casual workWork at officeFlexible hoursShift work
- Alberta Machine Intelligence Institute (Amii) in Edmonton seeks an Applied Research Scientist to advance the safety of autonomous AI systems. You will investigate agentic AI safety, develop rigorous evaluation methods, and translate research into practical tools supporting...
- lululemon is seeking a Senior Director, AI/ML Research and Science to advance the company's AI capabilities through research, development, and... ...impact. You will lead a lean team of AI researchers and applied scientists, balance long-range exploration with near-term business...
- eBay’s AI/ML team within the Selling organization is seeking an Applied Researcher to design and advance our research agenda across machine learning and generative AI. You will contribute to price guidance modeling and the expanding selling-agent platform, translating research...
- Applied AI Engineer### Applied AI EngineerApply: Remote, Canada: Poland, Virtual:... ...AI runtime, retrieval, guardrails, and evaluation that make the platform's AI features work... ...This Role Is Different*** **Applied, not research.** We use proven commercial models and win...Full timeRemote work
- ...Technology team is seeking two highly experienced Applied AI Engineers to design, build, and operate an... ...-on role focused on production systems, not research or demos, requiring deep understanding of GenAI failure modes, evaluation, and governance. The engineers will...InternshipWork at office
$180k - $230k per year
...complex capital projects. The AI buildout runs on physical... ...the world, including Crusoe, Applied Digital, and NTT. We raised a... ...projects. Continuous improvement: Evaluate models repeatedly and... ...features, inspired by latest research publications. What you'll doOwn...For contractorsInternshipWork at office- eBay Inc. is seeking a Staff Applied Researcher to advance visual and multimodal AI across the Global Focus Categories. You will research, train, fine-tune, evaluate, and deploy foundation-model-based systems at scale, collaborating with product, engineering and analytics teams...
- ...build a more efficient and enjoyable world. Role overviewAs an Applied AI Scientist, you design, build, and ship AI solutions that solve... ...)Turn experiments into production-ready components (evaluation, monitoring, iteration)Coach colleagues on applied AI best practices...Work at officeFlexible hours
- ...team dedicated to accelerating Generative AI adoption within a major organization. You... ...enterprise connectors Performing testing, evaluation, and performance optimization of AI... ...Your future team:You will work within an Applied AI team partnering closely with business and...Work at officeRemote work3 days per week
$149.3k - $249.3k per year
...the computer vision area who is aspired to apply your skills to bring value to millions of... ...Participate in the design, development, evaluation, deployment and updating of data-driven... ...models for computer vision applications. Research and implement the state-of-the-art...$110k - $150k per year
...capital markets with innovative, AI-driven technology and expert... ...We are looking for an Applied AI Engineer to join the AI Lab... .... You will prototype quickly, evaluate honestly, and know when something... ...technical direction, including evaluations, tooling, MLOps practices, and...InternshipLocal areaRemote workFlexible hours$140k - $160k per year
...worldwide. And we'd love for you to join us! About the job - Applied AI EngineerJoin the Engineering Team, where you'll help build our... ...or model change actually made things better. Raise the bar for evaluation across the team - judge design, offline and online metrics, and...Manual laborWork at officeWorldwide1 day per week- ...all applications and next steps. Our partner is looking for an Applied AI Engineer based in Canada. As an Applied AI Engineer, you will... ..., agent orchestration, retrieval-augmented generation (RAG), evaluation frameworks, and safety controls. Working with proven commercial...Long term contract
- We’re seeking a Senior Applied AI Engineer to serve as a forward-deployed AI expert across... ...teamsYou’ll turn the latest machine learning research into practical, production-ready... ...end to end, from framing the question to evaluating model performance and deciding what’s worth...InternshipFlexible hours
$146.83k - $192.72k per year
...team The Enterprise Data & AI team is a strategic and operational... ...from the very beginning. By applying AI to critical business... ..., and translating advanced AI research into scalable, production-ready... ...experimentation frameworks and evaluation approaches to measure business...Permanent employmentFull timePart timeLocal area- Software Developer - Applied AIRemote, Ontario Technology Remote... ...J is the leading generative AI solution for tax professionals... ...building industry-leading tax research software that helps accountants... ...hands of tax professionals Evaluation and data-driven decisions:...Long term contractFull timeImmediate startRemote work
- ...Your RoleFloyd is PSP's in-house AI platform — a custom analyst... ...built workflows that turn complex research questions into structured,... ...role puts you at the frontier of applied AI inside an institutional... ...multi-model orchestration and evaluation Collaborate with our Data Engineers...Summer workWork at officeLocal areaRemote work1 day per week
$150k - $180k per year
...CivCheck, we created Guided AI Plan Review™ (GPR) - a... ...offering. We’re seeking a Senior Applied AI Engineer to serve as a forward... ...the latest machine learning research into practical, production‑... ...from framing the question to evaluating model performance and deciding...Full timeContract workPart timeInternshipRemote work- About Mundo AI Mundo AI builds multimodal data infrastructure for frontier research labs and Fortune 100 companies. Our work covers... ...Role We’re looking for an applied researcher to turn open-ended... ...failure modes, develop novel evaluation methodologies, and design datasets...Long term contractTemporary workWork at office
- ...building a global community of experts to evaluate, challenge and improve advanced AI systems. We are looking for university students, graduates, researchers and experienced professionals with... ...and potential risks – Apply evaluation guidelines consistently...Hourly payFor contractorsRemote workFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Applied Ai Evaluation Researcher. Be the first to apply!

