Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Physician (MD/DO) - Put Medical AI to the Test, Remote

Part-time

Mindrift (Toloka)

About the company

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This is a task-based, independent contractor opportunity, not an employment relationship. Participation is limited to this project.

Project Highlights

  • Earnings per approved task: a fixed rate for each task, set in advance
  • Fully self-scheduled: no minimum hours and no set schedule; you choose which tasks to take and when
  • Remote: open to US-licensed physicians based in eligible US states
  • No patient contact, no malpractice exposure: you review written clinical cases
  • All contributors are license-verified physicians
  • $50 one-time onboarding reward, paid once your first task on the project is approved

What this project involves

We're looking for physicians (MD/DO) with an active, unrestricted US medical license to evaluate clinical EHR vignettes for a medical AI evaluation project. General and internal medicine are the main focus. Each task is a written clinical case, assessed against the project's acceptance criteria. Contributors may:

  • Evaluate clinical EHR vignettes paired with a question, a proposed answer, and a distractor "trap" answer across diagnosis and treatment tasks, spanning cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems;
  • Score the clinical reasoning quality of benchmark items: vignette accuracy and completeness, whether the vignette gives the answer away, answer correctness and gradeability, and trap quality;
  • Check whether the reasoning chain reaches the answer from vignette facts alone, correct reasoning traces, and write short rationales.

Each case receives three independent blind reads, followed by physician adjudication. Real clinical complexity only. You're improving the AI tools you'll eventually use yourself.

What we look for

This project is a good fit for US-licensed physicians looking for a flexible, time-limited project they can fit around their other commitments. Ideally, contributors will have:

  • Medical degree (MD or DO) and an active, unrestricted US medical license (verified at onboarding and re-checked monthly);
  • Board certification or completed residency training, with direct patient care experience (attending-level preferred). Practising or retired physicians welcome;
  • Broad, multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine);
  • Strong clinical reasoning: differential diagnosis, next-step management, and application of evidence-based guidelines;
  • Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA (a plus);
  • Strong written English (C1+).

This project is not suitable for medical students, unlicensed medical graduates, non-physician clinicians (NPs, PAs, nurses, pharmacists), or non-clinical healthcare titles.

How it works

Apply → Pass qualification(s) → Join the project → Complete tasks → Get paid for approved tasks

Flexibility

You decide whether, when, and how much to contribute. There are no minimum hours, set schedules, or required availability. Task availability varies with project phases. Tasks are submitted through the platform and are approved when they meet the project’s acceptance criteria.

Earnings

Earnings are paid only for completed and approved tasks. Each task type has a fixed rate. There is no time tracking and no hour logging. There are no tiers or speed adjustments to the rate. QA and adjudication tasks have a higher per-task rate.

  • $50 one-time onboarding reward, paid once your first task on the project is approved.
  • Quality incentives: additional earnings for high-quality approved tasks, on terms set out in the project guidelines.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Physician (MD/DO) - Put Medical AI to the Test, Remote in Remote vacancy
  •  ...Evaluate AI responses for accuracy in emergency medicine scenarios...  ...accurate, empathetic, and safe medical reasoning. What You’ll Do...  ...Candidate Requirements MD or equivalent; active practice...  ..., with possible extension Remote: Fully remote, collaborative environment... 
    Remote work
    Medical
    Part time
    For contractors
    Freelance

    Turing

    Remote
    2 days ago
  •  ...Design evaluation methods to enhance AI clinical reasoning in real scenarios....  ...You'll design evaluation methods that test AI performance on real medical problems where clinical expertise...  ...Candidate Requirements: Licensed physician in active clinical practice (any specialty... 
    Remote work
    Medical
    Part time
    Flexible hours

    Turing

    Remote
    a month ago
  •  ...AfterQuery is searching for Medical MD Experts to design and evaluate complex medical scenarios that drive the training of our advanced AI systems. In this remote, flexible role, you will craft case studies and decision-making challenges that reflect real-world clinical and... 
    Remote work
    Medical
    Contract work
    Flexible hours

    AfterQuery

    Remote
    a month ago
  • $240 per day

     ...Fully remote, project-based work checking how well frontier AI does real clinical work, from assessment and planning to medications and handoffs. You write cases based on your own practice and grade...  ...For licensed clinicians (RN, NP, PA, MD or DO, PharmD, or allied health) in... 
    Remote work
    Medical
    Full time
    Contract work
    Part time
    For contractors
    10 hours per week

    Sovrano AI

    Remote
    a month ago
  • $40 - $125 per hour

     ...difficulty and specialization) Location: Remote · Type: Independent contractor ·...  ...· Payouts: Weekly Overview Medical questions are where model mistakes matter...  ...clinician would sign off on. As a Physician you'll evaluate AI clinical reasoning against current... 
    Remote work
    Medical
    Weekly pay
    Full time
    Contract work
    For contractors
    Flexible hours

    DataAnnotation

    Remote
    a month ago
  • $100 - $120 per hour

     ...History of Present Illness Past Medical History Rubric Inpatient/...  ...micro1 is a data lab that helps AI labs & Enterprises train foundational...  ...velocity. Role Title: Hospitalist Physician Role Type: Contractor Location: Remote micro1 is engaging Hospitalist... 
    Remote work
    Medical
    Part time
    For contractors

    micro1

    Remote
    a month ago
  • $187.5 per hour

     ...Job Summary The Medical Advisor provides independent clinical expertise...  ...Education Medical degree (MD or equivalent) from a recognized medical school. Licensed physician in good standing with the...  ...approved artificial intelligence (AI) tools to support day-to-day tasks... 
    Remote work
    Medical
    Hourly pay
    For contractors
    Local area
    Home office
    Night shift

    Foundever®

    Remote
    22 days ago
  • $50 - $150 per hour

     ...–$150/hr The work Clinicians review AI-written answers to health questions and clinical...  ...Who it suits A current licence as a physician, nurse practitioner, registered nurse or...  ...shows us you understand them. You work remotely, on your own schedule, and are paid for... 
    Remote work
    Medical
    Contract work

    HUMXN

    Remote
    a month ago
  • $100 per hour

     ...Requirements Must be eligible to work in Remote Fluent proficiency in English (Written & Verbal) Reliable...  ...in the relevant clinical/health field (MD, DO, RN, RD, NBHWC, etc.) Current knowledge of medical standards and practices How long hiring takes... 
    Remote work
    Medical
    Hourly pay

    Ethos

    Remote
    a month ago
  • $210 per day

     ...What You'll Do Review de-identified medical images across various clinical modalities...  ...as a radiologist or specialized imaging physician Extensive experience interpreting complex...  ...independent contractor. This is a fully remote opportunity that can be completed on your... 
    Remote work
    Medical
    Contract work
    For contractors

    Terac

    Remote
    16 days ago
  •  ...What You'll Do Evaluate de-identified medical images across multiple modalities Provide...  ...Practicing radiologist or specialized physician with imaging expertise Experience interpreting...  ...independent contractor. This is a fully remote opportunity that can be completed on your... 
    Remote work
    Medical
    Contract work
    For contractors

    Terac

    Remote
    16 days ago
  • $150 per hour

     ...Pay: up to $150/hour In this hourly, remote contractor role, you will work as a Medical Doctor (Specialist) Subject Matter Expert (SME) to review AI-generated clinical responses and/or...  ...stratification, red-flag recognition, test interpretation, and treatment planning;... 
    Remote work
    Medical
    Hourly pay
    Contract work
    For contractors
    Local area
    Immediate start

    SuperAnnotate

    Remote
    a month ago
  • $90 per hour

     ...Pay: up to $90/hour In this hourly, remote contractor role, you will review AI-generated medical responses and/or generate expert healthcare content, evaluating...  ...and contextual relevance. Ensure Model Integrity: Test AI models for potential inaccuracies or biases, validating... 
    Remote work
    Medical
    Hourly pay
    Contract work
    For contractors
    Immediate start

    SuperAnnotate

    Remote
    a month ago
  •  ...Key Responsibilities • Label medical images and diagnostic data • Ensure data quality and...  ...documentation Requirements • Medical degree (MD) or advanced healthcare qualification •...  ..., MRI, X-ray) • Understanding of medical AI applications • Attention to detail and... 
    Medical
    Contract work

    Rise Data Labs

    Remote
    a month ago
  • $24 - $26 per hour

     ...Job Description Job Description Remote Travel Advisor 100% Remote in Ontario | Permanent...  ...Advisors. This is an opportunity to put your travel knowledge and customer service...  ...based selling. ~ Comprehensive Benefits: Medical, dental, and vision coverage. ~ RRSP... 
    Remote job
    Hourly pay
    Permanent employment
    Full time
    Work from home
    Shift work
    Afternoon shift

    Aspire Lifestyles

    Remote
    6 days ago
  •  ...What You'll Do Review and analyze de-identified medical images Provide expert diagnostic reads for complex clinical cases Complete structured...  ...will be engaged as an independent contractor. This is a fully remote opportunity that can be completed on your own schedule.... 
    Remote work
    Medical
    Contract work
    For contractors

    Terac

    Remote
    16 days ago
  •  ...What You'll Do Review de-identified medical images and patient clinical cases Provide structured expert reads for assigned clinical scenarios...  ...Able to complete detailed analytical tasks independently in a remote setting Contract & Payment Terms You will be engaged as an... 
    Remote work
    Medical
    Contract work
    For contractors

    Terac

    Remote
    16 days ago
  •  ...About the Role: At TELUS Digital , we believe that when AI enters the healthcare space, accuracy is a matter of life and death. As a Clinical & Medical AI Expert , you are the guardian of medical truth. You will join an elite circle of healthcare professionals within... 
    Medical
    Part time

    TELUS International

    Remote
    a month ago
  • $125 per hour

     ...depends on task difficulty and specialization) Location: Remote · Type: Independent contractor · Hours: Flexible hours ·...  ...Overview We’re looking for an experienced Internal Medicine Physician to help train AI models. You’ll measure how well the models answer clinical... 
    Remote work
    Weekly pay
    Full time
    For contractors
    Flexible hours

    DataAnnotation

    Remote
    1 day ago
  •  ...We are looking for an experienced Test Lead to drive the Quality and Validation strategy for a high-impact solution, in a highly regulated industry, involving AI components and Salesforce integration. In this role, you will own both definition and delivery of the Testing &... 
    Full time
    Contract work

    Digital Forms

    Remote
    4 days ago
  • $150 per hour

     ...and radiation sciences experts to help make AI models safer in their domain. You'll author and review evaluation tasks that test whether a model can tell a safe request from...  ...why. Requirements Master's degree or higher (MD, DO, MSc, or PhD) in radiology, medical physics,... 
    Remote work
    Medical
    Contract work
    Flexible hours

    AfterQuery

    Remote
    4 days ago
  •  ...healthcare platform for men. We are looking for an innovative part-time physician who lives in the province of Saskatchewan and specializes in...  ...consultation begins with the patient providing a comprehensive medical history through our intelligent and graded online treatment... 
    Medical
    Full time
    Part time
    Work at office

    Phoenix

    Remote
    4 days ago
  •  ...support the development and refinement of physician-focused healthcare market research...  ...survey questions for clinical accuracy, medical relevance, clarity, and appropriateness for...  ...Advanced clinical or scientific degree: MD, DO, PhD, PharmD, or equivalent qualification... 
    Medical
    Hourly pay
    Contract work

    Mercor

    Remote
    20 days ago
  •  ...Do you have a background in Medicine, Nursing, or Biology ? AI is being used more and more in healthcare, but it needs experts to...  ...and accurate. At TELUS Digital , we are looking for people with medical knowledge to check AI-generated health content. You don't need to... 
    Medical
    Part time
    Immediate start

    TELUS International

    Remote
    a month ago
  •  ...Job Application for Medical Specialist (Fluent in Arabic) - Freelance AI Trainer Project at Meridial Back to jobs Medical Specialist (Fluent in Arabic) - Freelance AI Trainer Project World Wide - Remote Are you a medical professional fluent in Arabic and eager... 
    Remote work
    Medical
    Hourly pay
    Contract work
    Part time
    For contractors
    Freelance
    Immediate start

    Invisible Technologies

    Remote
    19 days ago
  •  ...PA-C certified physician assistants to build the clinical reasoning scenarios frontier AI labs use to test model judgment. AfterQuery builds the expert evaluation data that frontier...  ...not negotiated down after you start. Fully remote, fully async. No fixed hours, it fits around... 
    Remote work
    Weekly pay
    Ongoing contract
    Contract work
    For contractors
    10 hours per week
    Shift work

    AfterQuery

    Remote
    a month ago
  •  ...Remote | United Kingdom | Project-Based Independent Contractor Compensation :...  ...requirements Help Shape the Future of AI in Healthcare What if the medical knowledge and clinical skills you have...  ...Medicine, Nurse Practitioner, or Physician Assistant pathways . This opportunity... 
    Remote work
    Medical
    Contract work
    For contractors
    Traineeship

    Volga Partners

    Remote
    3 days ago
  • $40 - $125 per hour

     ...difficulty and specialization) Location: Remote · Type: Independent contractor ·...  ...coder is who catches it. As a Medical Coder you'll evaluate AI coding output against real guidelines...  ...ll actually do Write prompts that test coding accuracy: ICD-10-CM, CPT, and HCPCS... 
    Remote work
    Medical
    Weekly pay
    Full time
    Contract work
    For contractors
    Flexible hours

    DataAnnotation

    Remote
    a month ago
  • $40 - $125 per hour

     ...difficulty and specialization) Location: Remote · Type: Independent contractor ·...  ...gap, and that is you. As a Health/Medical Expert you'll evaluate AI reasoning across clinical and health...  ...this fits Common backgrounds: Physician, Clinician, Medical Reviewer. What... 
    Remote work
    Medical
    Weekly pay
    Full time
    Contract work
    For contractors
    Flexible hours

    DataAnnotation

    Remote
    a month ago
  • $15 - $25 per hour

     ...who routinely contour organs at risk in radiotherapy workflows: medical dosimetrists, treatment planners, RTTs and medical physicists. You...  ...contouring product. First project: 10 to 40 hours over 2 to 4 weeks, remote, on your own schedule, with further projects for contourers who... 
    Remote work
    Medical
    Contract work
    For contractors
    Flexible hours

    YPAI

    Remote
    a month ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Physician (MD/DO) - Put Medical AI to the Test, Remote. Be the first to apply!