Clinical Mental Health Expert — AI Conversation Evaluation
Mercor
Mercor is seeking Clinical Mental Health Experts to support an AI research initiative focused on evaluating the realism and quality of simulated clinical conversations. This is a short-term, fully remote evaluation task designed for mental health professionals who can apply their clinical experience and professional judgment to assess realistic conversational scenarios. The task will take up to 30 minutes to complete.
What You’ll Do
Review pairs of simulated conversations involving clinical mental health scenarios.
Evaluate the conversations based on their realism and authenticity.
Determine which conversation more closely reflects a realistic clinical interaction.
Apply your clinical experience and professional judgment when making evaluations.
Complete the evaluation independently through an online interface.
Ideal Background
We welcome professionals with a relevant degree and clinical mental health expertise. Suitable backgrounds may include:
Clinical Psychology
Counselling Psychology
Mental Health Counselling
Clinical Social Work
Psychiatry
Psychotherapy
Behavioural Health
Marriage and Family Therapy
Other closely related clinical mental health disciplines
Candidates should have:
A relevant degree or professional qualification in a mental health-related field.
Practical experience working in a clinical mental health or behavioural health setting.
Familiarity with real-world conversations between mental health professionals and patients or clients.
Strong professional judgment and the ability to identify whether clinical interactions feel realistic and authentic.
Strong attention to detail and the ability to make independent qualitative assessments.
Why Join
Apply your clinical expertise to the evaluation of advanced AI systems.
Contribute to research aimed at improving the realism and quality of AI-generated interactions.
Be considered for future AI evaluation and research opportunities requiring clinical mental health expertise.
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
- ...TELUS Digital is building a global community of experts to evaluate, challenge and improve advanced AI systems . We are looking for university students... ...computational outputs LLM & NLP AI-generated text and conversations Language, meaning, context and reasoning...SuggestedPart time
- ...What You'll Do Apply your subject-matter expertise to advanced AI training and evaluation work Develop or evaluate challenging problems within your area of expertise Assess AI-generated reasoning for technical accuracy and identify subtle errors or failure modes...SuggestedContract workImmediate startRemote workFlexible hours
- © 2026 Smart Ecosystems. All rights reserved.SuggestedContract work
- ...Join a leading AI lab's cutting-edge GenAI team to be at the core... ...a senior finance domain expert to work directly with a leading... ...challenging finance tasks and evaluation sets, and help build finance-specific... ..., childbirth, reproductive health decisions, or related medical...SuggestedFull timeContract workPart timeFreelanceLive InManual laborRelocationRelocation package
$140k - $180k per year
...equivalent experience in Solutions Engineering, Conversational AI, or Technical Product roles. ~... ...failure modes. ~ Experience building evaluations for non-deterministic systems,... ...development opportunities. Paid parental leave. Health and wellness initiatives....SuggestedFull timeInternshipFlexible hours$50 - $150 per hour
...United States | Canada + 6 more We're hiring PhD-level experts to help train and evaluate advanced AI models. This is remote, flexible contract work where... ...) Prior experience with data annotation or AI model evaluation Experience reviewing or publishing research papers...Contract workTemporary workInternshipRemote workFlexible hours- ...About the Role: At TELUS Digital , we believe that when AI enters the healthcare space, accuracy is a matter of life and death. As a Clinical & Medical AI Expert , you are the guardian of medical truth. You will join an elite circle of healthcare professionals within...Part time
- ...ll use your finance expertise to help evaluate and improve advanced AI systems on complex, realistic... ...professional quality. Develop clear evaluation rubrics and scoring criteria, identifying... ...training will be provided to selected experts. Who You Are We're looking for...Contract work
- ...About the Role: An AI Experience Engineer owns the full lifecycle of the journeys our... ...no-code journeys), Sierra simulations and conversation analytics, GraphQL and REST APIs (as... ...where each one fails • Experience building evaluations for non-deterministic systems: test sets,...Internship
- ...Create and evaluate board game tasks to enhance AI reasoning skills. About Turing Turing is one of the world... ...high-quality human feedback, evaluation, and training data. Role Overview... ...We are seeking Board Game Reasoning Experts to help train and evaluate next-generation...Contract workPart timeFor contractorsFreelanceRemote work
$40 - $125 per hour
...Payouts: Weekly Overview Models answer health questions in a confident, authoritative voice and still get the clinical details wrong: a contraindication they skip... ..., and that is you. As a Health/Medical Expert you'll evaluate AI reasoning across clinical and health...Weekly payFull timeContract workFor contractorsRemote workFlexible hours- ...Build real-world tasks that test how well AI agents handle database engineering. Depending... ..., and regressions where relevant. Evaluate performance measures such as latency, throughput... ...Paid task after selection Selected experts will complete one task, with approximately...Contract workImmediate start
- ...About the Role As a Clinical Medicine AI Specialist, you will directly improve the accuracy and... ...diagnostic support, and patient-facing health information. Your professional judgment... ...Reviewing AI-Generated Clinical Summaries: Evaluating AI’s interpretation of patient...Hourly pay
- ...Join a leading AI lab's cutting-edge GenAI team to be at the core of the AI revolution... ...Overview We are hiring a senior clinical medicine domain expert to work directly with a leading AI lab... ...Design challenging clinical tasks and evaluation sets, and help build medicine-specific...Full timeContract workPart timeFreelanceLive InManual laborRelocationRelocation package
- ...Review AI conversations to identify context gaps and improve clarity. About Turing: Turing... ...: We are seeking Subject Matter Experts (SMEs) with expertise in psychology, social... ...fields to support AI conversation evaluation research. SMEs will review conversations...Full timeTemporary workPart time
$70 per hour
...hourly, remote contractor role, you will work as a Public Health Subject Matter Expert (SME) to review AI-generated public health and population-health responses and/or create expert public health content, evaluating reasoning quality and step-by-step analytical problem-solving...Hourly payContract workFor contractorsImmediate startRemote work- ...Evaluate AI model safety by designing adversarial test prompts and classifications. About Turing: Based in San Francisco, California... ...implied. Project Description: This project supports the evaluation and improvement of safety behavior in large language model...Ongoing contractPart timeFor contractors
- ...AfterQuery builds the expert evaluation data that frontier AI labs use to measure and improve their models. We're hiring health economists as contractors to author and grade the economic evaluations those assessments run on. What you'll actually do • Author realistic economic...Contract workFor contractors
- ...Independent contractor Remote · Contract · Radiological Science · Health Physics · AI Safety · CBRN What this pays Rates are set by the... ...we cannot stand behind. About the Role AI Safety Evaluation Project | Remote | Contract Locations: US, Canada, UK, Europe...Hourly payContract workFor contractorsRemote work
$55 - $100 per hour
...Pay: $55–100/hour You'll evaluate AI-generated responses to legal questions — covering case analysis, statutory interpretation, procedural... ...a practitioner, and flagging anything that could mislead a non-expert relying on the output. Day-to-day work involves reading a...Contract workusd16 per hour
...We are looking for AI Linguistic Evaluators to assess how effectively an AI application performs in Marathi for a short collaboration. Job Type... ...naturalness, and cultural appropriateness. - Providing structured evaluations and feedback following the project instructions and quality...Hourly payDaily paidFull timeFreelanceImmediate startRemote workFlexible hours$13 per hour
...Location: Argentina | Belize | Bolivia + 15 more Help evaluate how an AI assistant uses information from connected Google apps to respond... .... • Follow project guidelines for consistent, confidential evaluation work. Important account-access requirement • This project...Hourly payContract workFor contractorsRemote work- ...Key Responsibilities • Design and implement evaluation metrics for AI models • Conduct thorough testing of model performance across different scenarios • Analyze model outputs for bias, fairness, and accuracy • Collaborate with ML engineers to implement improvements...Contract work
$40 per hour
...review your choices. Accept All Customize Settings Reject All Employers Job seekers Career centers Handshake AI Enterprise AI Research Log in Sign up FRENCH AI EVALUATION SPECIALIST (CANADA) Sign up now Up to $40/hr Depending on the project Bachelor's Remote Part-time Flexible...Part timeRemote workFlexible hours- ...on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Senior Data Scientist AI Evaluation based in Canada. We are seeking a senior-level data scientist to establish and advance the evaluation practices that measure the quality...Full time
$130 per hour
...connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.... ...practicing physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program. General and internal medicine...Hourly payPermanent employmentPart time10 hours per week- ...LCSWs to build the clinical social work scenarios frontier AI labs use to test model judgment and safety. AfterQuery builds the expert evaluation data that frontier AI labs use to measure and improve their models. We're hiring licensed clinical social workers as contractors...Weekly payOngoing contractContract workFor contractorsRemote work10 hours per week
- ...As a Thai Linguist in our Global AI Community, you evaluate and validate speech AI systems from a linguistic... ...Conduct linguistic quality evaluation of speech output including assessment... ...voice similarity, voice generation, and conversational AI evaluation projects with linguistic...Part time
- ...RHIA and RHIT credentialed HIM professionals to build the health data governance scenarios frontier AI labs use to test model judgment. AfterQuery builds the expert evaluation data that frontier AI labs use to measure and improve their models. We're hiring RHIA and RHIT credentialed...Weekly payOngoing contractContract workFor contractorsRemote work10 hours per week
$40 per hour
...experience About the Role Pasiflora AI has immediate openings for Chemical AI Safety Annotators & Subject Matter Experts to evaluate AI-generated responses in chemistry and chemical... ...Science Industrial Hygiene Environmental Health / EHS Occupational or Process Safety...Contract workFor contractorsImmediate startRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Clinical Mental Health Expert — AI Conversation Evaluation. Be the first to apply!
