Applied Ai Evaluation Researcher
Kids Help Phone
Applied AI evaluation and user-centred research define this full-time contract role with Kids Help Phone in Toronto, using qualitative and quantitative methods.As an Applied AI Evaluation & Product Researcher, you will evaluate a youth-facing generative AI product. You will develop measurement strategies, conduct UX research, assess risks, and translate evidence into recommendations. The hybrid contract requires working one or two days weekly in the workplace.Key Responsibilities:
Develop evaluation strategies from early testing through post-launch monitoringCreate evaluation matrices linking objectives, indicators, data, and decisionsConduct surveys, usability testing, journey mapping, and co-design activitiesAssess unintended outcomes, bias, safety, privacy, and equity risksSynthesize findings into recommendations, reports, briefings, and decision materialsRequirements:
PhD or equivalent Master’s experience in a related disciplineMinimum five years evaluating generative AI or digital servicesKnowledge of responsible AI, safety, risk, or technology governanceStrong qualitative and quantitative research experienceExperience translating findings into product or implementation recommendationsBring evaluation strategy, UX research, and responsible AI expertise to Kids Help Phone’s youth-focused generative AI initiative.
#J-18808-Ljbffr
Develop evaluation strategies from early testing through post-launch monitoringCreate evaluation matrices linking objectives, indicators, data, and decisionsConduct surveys, usability testing, journey mapping, and co-design activitiesAssess unintended outcomes, bias, safety, privacy, and equity risksSynthesize findings into recommendations, reports, briefings, and decision materialsRequirements:
PhD or equivalent Master’s experience in a related disciplineMinimum five years evaluating generative AI or digital servicesKnowledge of responsible AI, safety, risk, or technology governanceStrong qualitative and quantitative research experienceExperience translating findings into product or implementation recommendationsBring evaluation strategy, UX research, and responsible AI expertise to Kids Help Phone’s youth-focused generative AI initiative.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Applied Ai Evaluation Researcher in Winnipeg, MB vacancy
$95k - $110k per year
...Applied AI Evaluation & Product Researcher (Contract) Applied Research FT Contract Toronto, Toronto, ON, CA Salary Range: $95,000.00 To $110,000.00 Annually Job title Applied AI Evaluation & Product Researcher (12 Month Contract) Department Reports to Director...SuggestedFull timeContract workLive In2 days per week1 day per week- Cohere is seeking a Senior Research Scientist focused on AI evaluation methods to support our innovative foundation models. This remote or on-site role emphasizes creating evaluation benchmarks and tools that enhance LLM capabilities. As a Senior Research Scientist at Cohere...SuggestedRemote work
$106k - $156k per year
...immediate 12-month contract opening for a Researcher. About the team The Huawei Human-Machine... ..., hands on research. We focus on agentic AI and multimodal interaction (voice, touch,... ...About the job Research, prototype, and evaluate new AI/ML technologies for multimodal...SuggestedContract workInternshipImmediate start- ...you can be proud to be with.The AI/ML team within eBay’s Selling... ...ML engineering — moving research from theoretical grounding through... ...eBay’s global marketplace.As an Applied Researcher, you will design... ...actionable listing advice.Developing evaluation frameworks for generative AI...SuggestedManual laborImmediate startWorldwide
$176k - $195k per year
...percentage of our revenue back into research and development, ensuring our... ...human-firstand accelerated by AI to create meaningful and... ...of the Advanced Technology / Applied Research team atPointClickCare... ...to prototyping, building and evaluating models that will truly transform...SuggestedLong term contractWork at officeRemote workFlexible hours$120k - $170k per year
...Xanadu, we are learners, innovators, researchers, collaborators and problem solvers... ...role and responsibilities: As an AI Specialist at Xanadu, you will drive applied AI initiatives by deeply analyzing... ...will uncover hidden patterns in research across various technical fields. This...$140k - $175k per year
...Thomson Reuters Labs is seeking a Lead Research Engineer to lead the design and delivery... ...software for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable... ..., interpreting evaluation results, and applying findings to engineering decisions....InternshipWork at officeLocal areaFlexible hours- Alberta Machine Intelligence Institute (Amii) in Edmonton seeks an Applied Research Scientist to advance the safety of autonomous AI systems. You will investigate agentic AI safety, develop rigorous evaluation methods, and translate research into practical tools supporting...
$173k - $197.4k per year
...machine learning, conversational AI and design thinking into our... ...to building world-class applied science and engineering teams... ...work touches every aspect of the research life cycle, from partnering with... ...from design through training, evaluation, validation, and implementation...Work at officeWork from home- ...unique opportunity to contribute to cutting-edge AI research in Trust and Safety, with a particular focus on understanding and evaluating the risks of autonomous AI systems. You’ll... ...-renowned researchers drive fundamental and applied research at the University of Alberta (and...Casual workWork at officeFlexible hoursShift work
- lululemon is seeking a Senior Director, AI/ML Research and Science to advance the company's AI capabilities through research, development, and... ...impact. You will lead a lean team of AI researchers and applied scientists, balance long-range exploration with near-term business...
- eBay’s AI/ML team within the Selling organization is seeking an Applied Researcher to design and advance our research agenda across machine learning and generative AI. You will contribute to price guidance modeling and the expanding selling-agent platform, translating research...
$90k - $110k per year
...healthcare, education, and leisure facilities. Job SummaryThe Applied AI Developer is responsible for the design, development,... ...standards and best practices for responsible AI use. Research and Technology Evaluation: Research emerging AI technologies, models, frameworks,...Long term contractPermanent employmentFull timeWork at officeRemote workFlexible hours3 days per week- ...Technology team is seeking two highly experienced Applied AI Engineers to design, build, and operate an... ...-on role focused on production systems, not research or demos, requiring deep understanding of GenAI failure modes, evaluation, and governance. The engineers will...InternshipWork at office
$180k - $230k per year
...complex capital projects. The AI buildout runs on physical... ...the world, including Crusoe, Applied Digital, and NTT. We raised a... ...projects. Continuous improvement: Evaluate models repeatedly and... ...features, inspired by latest research publications. What you'll doOwn...For contractorsInternshipWork at office- ...the role and the team In the role of Applied Researcher 3 within the Global Focus Categories on... ...develop new computer vision and multi-input AI solutions. These solutions will address... ...will research, build, train, fine‑tune, evaluate, and deploy models and AI systems at...For contractorsWork at officeImmediate start
- eBay Inc. is seeking a Staff Applied Researcher to advance visual and multimodal AI across the Global Focus Categories. You will research, train, fine-tune, evaluate, and deploy foundation-model-based systems at scale, collaborating with product, engineering and analytics teams...
- ...team dedicated to accelerating Generative AI adoption within a major organization. You... ...enterprise connectors Performing testing, evaluation, and performance optimization of AI... ...Your future team:You will work within an Applied AI team partnering closely with business and...Work at officeRemote work3 days per week
- ...environment!With rapid growth comes opportunity. We are looking for an Applied AI Engineer (LLM Applications Engineer) to join our team onsite in... ..., environment parity, model/prompt/agent versioning) and build evaluation and observability for GenAI and agentic systems — tracing and...Full timeWork at officeLocal area
$140k - $160k per year
...worldwide. And we'd love for you to join us! About the job - Applied AI EngineerJoin the Engineering Team, where you'll help build our... ...or model change actually made things better. Raise the bar for evaluation across the team - judge design, offline and online metrics, and...Manual laborWork at officeWorldwide1 day per week$110k - $150k per year
...capital markets with innovative, AI-driven technology and expert... ...We are looking for an Applied AI Engineer to join the AI Lab... .... You will prototype quickly, evaluate honestly, and know when something... ...technical direction, including evaluations, tooling, MLOps practices, and...InternshipLocal areaRemote workFlexible hours- Armilla AI is a cutting-edge startup at the intersection of artificial... ...We're seeking an exceptional Applied Scientist who bridges the worlds of deep AI research and practical, production-grade applications... ...into robust risk assessment and evaluation frameworks. This isn't just about...Shift work
$140k - $165k per year
...and world-class practices in AI for Education, Future-Ready Workforce... .... We are looking for an Applied AI Solutions Specialist to... ...nontechnical colleagues, build and evaluate prototypes, and help the best... ...teams, IT, business units, researchers, monitoring and evaluation teams...Long term contract- About Mundo AI Mundo AI builds multimodal data infrastructure for frontier research labs and Fortune 100 companies. Our work covers... ...Role We’re looking for an applied researcher to turn open-ended... ...failure modes, develop novel evaluation methodologies, and design datasets...Long term contractTemporary workWork at office
- ...ROLE Floyd is PSP's in-house AI platform — a custom analyst... ...built workflows that turn complex research questions into structured,... ...role puts you at the frontier of applied AI inside an institutional... ...tuning, and LLM observability and evaluation toolingExpert in Python (...Summer workWork at officeLocal areaRemote work1 day per week
- ...build a more efficient and enjoyable world. Role overviewAs an Applied AI Scientist, you design, build, and ship AI solutions that solve... ...)Turn experiments into production-ready components (evaluation, monitoring, iteration)Coach colleagues on applied AI best practices...Work at officeFlexible hours
- ...all applications and next steps. Our partner is looking for an Applied AI Engineer based in Canada. As an Applied AI Engineer, you will... ..., agent orchestration, retrieval-augmented generation (RAG), evaluation frameworks, and safety controls. Working with proven commercial...Long term contract
$149.3k - $249.3k per year
...the computer vision area who is aspired to apply your skills to bring value to millions of... ...Participate in the design, development, evaluation, deployment and updating of data-driven models for computer vision applications.Research and implement the state-of-the-art computer...$150k - $180k per year
...CivCheck, we created Guided AI Plan Review™ (GPR) - a... ...offering. We’re seeking a Senior Applied AI Engineer to serve as a forward... ...the latest machine learning research into practical, production‑... ...from framing the question to evaluating model performance and deciding...Full timeContract workPart timeInternshipRemote work- ...Directly builds AI-enabled security workflows with OpenAI APIs... ...OpenAI is hiring a Cybersecurity Applied AI Architect to partner with... ...requirements, workflow integrations, evaluation methods, and security... ...feedback to Product, Engineering, Research, Security, and GTM teams....
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Applied Ai Evaluation Researcher. Be the first to apply!

