Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Physician (MD/DO) - Medical AI Evaluation

Part-time

Mindrift

About the company

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This is a task-based, independent contractor opportunity, not an employment relationship. Participation is limited to this project.

Project Highlights

  • Earnings per approved task: a fixed rate for each task, set in advance
  • Fully self-scheduled: no minimum hours and no set schedule; you choose which tasks to take and when
  • Remote: open to US-licensed physicians based in eligible US states
  • No patient contact, no malpractice exposure: you review written clinical cases
  • All contributors are license-verified physicians
  • $50 one-time onboarding reward, paid once your first task on the project is approved

What this project involves

We're looking for physicians (MD/DO) with an active, unrestricted US medical license to evaluate clinical EHR vignettes for a medical AI evaluation project. General and internal medicine are the main focus. Each task is a written clinical case, assessed against the project's acceptance criteria. Contributors may:

  • Evaluate clinical EHR vignettes paired with a question, a proposed answer, and a distractor "trap" answer across diagnosis and treatment tasks, spanning cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems;
  • Score the clinical reasoning quality of benchmark items: vignette accuracy and completeness, whether the vignette gives the answer away, answer correctness and gradeability, and trap quality;
  • Check whether the reasoning chain reaches the answer from vignette facts alone, correct reasoning traces, and write short rationales.

Each case receives three independent blind reads, followed by physician adjudication. Real clinical complexity only. You're improving the AI tools you'll eventually use yourself.

What we look for

This project is a good fit for US-licensed physicians looking for a flexible, time-limited project they can fit around their other commitments. Ideally, contributors will have:

  • Medical degree (MD or DO) and an active, unrestricted US medical license (verified at onboarding and re-checked monthly);
  • Board certification or completed residency training, with direct patient care experience (attending-level preferred). Practising or retired physicians welcome;
  • Broad, multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine);
  • Strong clinical reasoning: differential diagnosis, next-step management, and application of evidence-based guidelines;
  • Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA (a plus);
  • Strong written English (C1+).

This project is not suitable for medical students, unlicensed medical graduates, non-physician clinicians (NPs, PAs, nurses, pharmacists), or non-clinical healthcare titles.

How it works

Apply → Pass qualification(s) → Join the project → Complete tasks → Get paid for approved tasks

Flexibility

You decide whether, when, and how much to contribute. There are no minimum hours, set schedules, or required availability. Task availability varies with project phases. Tasks are submitted through the platform and are approved when they meet the project’s acceptance criteria.

Earnings

Earnings are paid only for completed and approved tasks. Each task type has a fixed rate. There is no time tracking and no hour logging. There are no tiers or speed adjustments to the rate. QA and adjudication tasks have a higher per-task rate.

  • $50 one-time onboarding reward, paid once your first task on the project is approved.
  • Quality incentives: additional earnings for high-quality approved tasks, on terms set out in the project guidelines.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Physician (MD/DO) - Medical AI Evaluation in Toronto, ON vacancy
  • $100 per hour

     ...reviewers are license-verified physicians ~$50 one-time onboarding...  ...specialists with project-based AI opportunities for leading tech...  ...companies, focused on testing, evaluating, and improving AI systems.  Participation...  ...practicing physicians (MD/DO) with an active,... 
    Medical
    Price work
    Remote work
    10 hours per week
    Flexible hours

    Mindrift

    Toronto, ON
    12 days ago
  • $150 per hour

     ...and technical talent with leading AI research labs. Headquartered in...  ...Role Responsibilities Evaluate and compare AI-generated responses on challenging medical problems to improve model reasoning...  ...Qualifications Must-Have ~ MD , DO , PharmD , PhD , or... 
    Medical
    Remote job
    Contract work
    Summer work
    Immediate start

    Mercor

    Toronto, ON
    20 days ago
  • $30 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors...  ...tasks guiding it answering questions and correcting errors. Evaluate the experience and deliverables providing concise feedback on performance... 
    Suggested
    Hourly pay
    Weekly pay
    Full time
    Contract work
    For contractors
    Summer work
    Remote work

    Mercor

    Toronto, ON
    5 days ago
  • $126k - $176k per year

     ...Sr. Legal AI Prompter / Evaluator Job Description Sr. Legal AI Prompters / Evaluators are responsible for improving CoCounsel Legal by writing...  ...case Build gold-standard datasets with SMEs, coordinate evaluations, and design evaluation schemes, including rubrics, metrics,... 
    Suggested
    Remplacement
    Full time
    Work at office
    Local area
    Flexible hours

    Thomson Reuters

    Toronto, ON
    5 days ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...pipelines , and technical reports. Evaluate AI-generated or human-created work against...  ...Provide detailed written justifications for evaluations and scores to maintain transparency.... 
    Suggested
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    25 days ago
  • $90k - $110k per year

     ...marketing and advertising, media, medical affairs and communications,...  ...-winning data science and AI capabilities. Founded in 1997,...  ...literature searches and critically evaluate clinical data using relevant...  ...healthcare audiences, including physicians, specialists, nurses, patients... 
    Medical
    Full time

    Klick Health

    Toronto, ON
    3 hours ago
  • $140k - $175k per year

     ...is seeking a Lead Research Engineer to lead the design and delivery of production-ready software for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable evaluation infrastructure that enables research and product teams to compare large language... 
    Full time
    Internship
    Work at office
    Local area
    Flexible hours

    Thomson Reuters

    Toronto, ON
    9 days ago
  • $150 per hour

     ...and technical talent with leading AI research labs. Headquartered in...  ...Role Responsibilities Evaluate and compare AI-generated responses to challenging medical problems. Analyze model reasoning...  ...Must-Have ~ MD , DO , PharmD , PhD , or advanced... 
    Medical
    Remote job
    Contract work
    Summer work
    Immediate start
    Flexible hours

    Mercor

    Toronto, ON
    a month ago
  • $94 - $119 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Qualifications Must-Have MD , DO , PhD , or doctoral candidate in Medicine...  ...subdomain. Strong command of graduate-level medical knowledge, clinical reasoning, and biomedical... 
    Medical
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    4 days ago
  • $39 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Commitment: 10+ hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation, and other standards... 
    Remote job
    Contract work
    Summer work
    Immediate start

    Mercor

    Toronto, ON
    more than 2 months ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    more than 2 months ago
  • $70 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...hour Location: Remote Role Responsibilities Evaluate AI-generated responses to strengthen reasoning and rigor in model... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ..., and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    more than 2 months ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    more than 2 months ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...pipelines , and A/B test write-ups . Evaluate AI-generated or human-created work against...  ...detailed written justifications for evaluations and scores to maintain transparency and accountability... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $20 - $160 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Position: Generalist Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $20–$160/hour... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    a month ago
  •  ...R2873019 Position Title: Medical Science Liaison Oncology Department...  ...We’re an R&D-driven, AI-powered biopharma company committed...  ...on medical treatment to Physicians, Nurses, and Patient Support Groups...  ...Sanofi. Contributes to the evaluation of grants for educational and... 
    Medical
    Local area
    Remote work
    Work from home
    Toronto, ON
    12 days ago
  • $70 per hour

     ...About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position... 
    Remote job
    Contract work
    Summer work
    Immediate start

    Mercor

    Toronto, ON
    a month ago
  • $85 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $80 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Responsibilities Use frontier AI coding agents to complete and evaluate complex data engineering tasks. Review model-generated... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  •  ...: R2873008 Position Title: Medical Director Oncology Department...  ...Sanofi We’re an R&D-driven, AI-powered biopharma company committed...  ...Required qualifications : ~ MD, PhD, PharmD, or equivalent...  ...activities related to performance evaluation. Information collected and... 
    Medical
    Long term contract
    Local area
    Work from home
    Toronto, ON
    7 days ago
  • $120 - $170 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20–30... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $51.06 - $63.82 per hour

     ...Toronto Western Hospital Emergency Departments evaluate more than 120,000 patients annually and...  ...provides patient care, including medical/nursing students, residents, nursing staff...  ...practitioners, allied health, and staff physicians. Support to the emergency department is provided... 
    Medical
    Hourly pay
    Remplacement
    Permanent employment
    Full time
    Internship
    Flexible hours
    Shift work
    Night shift
    Afternoon shift

    University Health Network

    Toronto, ON
    6 days ago
  • $39 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...providing concise notes on performance. Evaluate AI model interactions to ensure alignment...  ...or healthcare professional (e.g., nurse, physician, pharmacist, therapist). ~3+ years of... 
    Remote job

    Mercor

    Toronto, ON
    10 days ago
  • $39 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors...  ...Licensed or practicing clinician or healthcare professional (e.g. nurse physician pharmacist therapist). ~3 years of professional experience... 
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    Toronto, ON
    5 days ago
  •  ...employee retention. We support the pharmaceutical, biotech, and medical device industries with our proven scientific approach to complex...  ...clinical and safety teams. Job Requirements ~ Medical degree (MD, DO, or international equivalent), ~ Experience with protocol amendments... 
    Medical
    Remote job
    Full time
    Local area

    MMS

    Toronto, ON
    4 days ago
  •  ...Arabic Speaking Addiction Physician Jobs – Relocate to Dubai Location: Toronto, ON...  ...role carries responsibility for the medical management of patients through detox, withdrawal...  ...Fluent Arabic speaker (required) MD, DO, or equivalent medical degree Clinical... 
    Medical
    Contract work
    Internship
    Immediate start
    Relocation
    Relocation package

    HealthOPM

    Toronto, ON
    3 days ago
  •  ...PHYSICIAN Type: Option 1: Full Time, Permanent (35 hours per week) Option 2...  ...services for marginalized, vulnerable, and medically complex clients. The Physician will...  ...Engage in quality improvement, research, and evaluation to strengthen client outcomes and reduce... 
    Medical
    Permanent employment
    Full time
    Part time

    Prosperity Workforce Solutions

    Toronto, ON
    a month ago
  • $48 per hour

     ...Job Title: Medical Radiation Technologist (X-Ray/ Nuclear Medicine Technologist) Reporting...  ...acquire data for use by nuclear medicine physicians in the diagnosis of disease. (Only...  ...and in a fast paced environment. Self-evaluate performance, establish personal objectives... 
    Medical
    Hourly pay
    Permanent employment
    Full time
    Internship
    Flexible hours
    Shift work
    Weekend work
    Afternoon shift

    WELL Health Diagnostic Centres

    Toronto, ON
    6 days ago
  • $116.68k - $166.68k per year

     ...Medical Science Liaison (Cardiometabolic) Medical & Regulatory | Novo Canada Location: Toronto, Ontario     Your impact begins...  ...will bring: ~ An advanced scientific or healthcare degree (PhD, MD or PharmD), or a Master’s degree in a science-related field combined... 
    Medical
    Long term contract
    Temporary work
    Night shift

    Novo Nordisk

    Toronto, ON
    7 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Physician (MD/DO) - Medical AI Evaluation. Be the first to apply!