Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Medical Evaluation Specialist

$40 - $90 per hour
Full-time

RemoteJobsOne

IMPORTANT: after you apply please check your email. We send you a link to complete your application it is not considered until that last step is done. If you do not see it check your spam folder.

This is a fully remote position open to candidates based in Canada.

Pay: $40$90/hr

Role Title: Medical Evaluation Specialist (students residents physicians)

Role Type: Contractor

Location: Remote

We are engaging Medical Evaluation Specialists (students residents physicians) to contribute advanced clinical insights to a cutting-edge customer this role youll apply your expertise to help train next-generation AI systems. Your work will shape how models learn reason and perform through high-quality real-world input. No prior experience in AI is required your domain knowledge is what matters. As a contributor you will design and verify high-quality challenging medical questions and answers that probe the depth and reliability of todays frontier AI focusing on the places where nuanced medical reasoning makes the difference.

Scope of Work

  1. Craft original high-difficulty medical question-and-answer pairs covering diagnosis mechanisms pharmacology guidelines and clinical judgment.
  2. Source and verify answers using primary literature clinical guidelines and authoritative references; thoroughly document rationale and citations.
  3. Develop questions requiring synthesis and genuine clinical reasoning rather than simple fact recall.
  4. Evaluate AI model responses to your questions refine those that are too straightforward and enhance complexity while ensuring clinical accuracy.
  5. Ensure all questions and answers are written with clarity precision and defensibility avoiding ambiguity.
  6. Incorporate reviewer feedback and align deliverables with evolving project guidelines and standards.

Preferred Qualifications

  1. Current medical training or clinical practice as a student resident or physician or equivalent biomedical expertise.
  2. Demonstrated ability to locate interpret and synthesize information from primary literature and clinical guidelines.
  3. Exceptional attention to detail and proficiency in written English (native fluency not required).
  4. Talent for creating engaging well-structured and clinically nuanced questions.
  5. Self-motivated and reliable in managing independent remote contributions.
  6. Experience with medical question writing peer review or evaluative projects is an advantage.
  7. Familiarity with AI/ML systems or model evaluation is a plus but not essential.

Compensation Structure

Compensation is output-based; experts are paid per task that meets the project specifications. The time required to complete work may vary depending on the experts experience and workflow. Minimum submission requirements apply. Experts must submit a minimum of tasks per week.

Start Timeline & Availability

We typically fill roles within 48 hours and are looking for experts ready to jump in right away. If selected we expect you to start your first tasks within 2448 hours of completing onboarding.

How to apply

Apply here with your profile. We will then email you a link to complete your application on the hiring platform your application is only considered once that final step is done.

Vacancy posted 3 hours ago
Similar jobs that could be interesting for youBased on the Medical Evaluation Specialist in Toronto, ON vacancy
  • $40 - $90 per hour

     ...spam folder. This is a fully remote position open to candidates based in Canada. Pay: $40$90/hr Role Title: Research Evaluation Specialist (PhD / Researcher / Professor) Role Type: Contractor Location: Remote We are engaging Research Evaluation Specialists... 
    Suggested
    Full time
    For contractors
    Remote work

    RemoteJobsOne

    Toronto, ON
    3 hours ago
  • $15 - $20 per hour

     ...include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Video Evaluation Generalist Type: Contract Compensation: $15–$20/hour Location: Remote Commitment: 20 hours/... 
    Suggested
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $80 - $160 per hour

     ...Provide structured feedback to enhance document relevance and impact. Utilize strong analytical skills to contribute to high-impact evaluation projects. Collaborate with teams to improve document standards and guidelines. Work independently and asynchronously to... 
    Suggested
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $80 - $150 per hour

     ..., and Jack Dorsey . Position: MS Excel / Google Sheets Evaluator Type: Contract Compensation: $80–$150/hour...  ...relevance. Provide structured feedback to enhance document evaluation processes. Utilize MS Excel and Google Slides to perform... 
    Suggested
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $90 per hour

     ...impact. ~ Professional written English for task justification. Preferred Prior RLHF , preference-labeling, model-evaluation, or structured code-review work. Pixel-perfect design-to-code experience. Accessibility expertise ( ARIA , semantic... 
    Suggested
    Full time
    Contract work
    Summer work
    Local area
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $80 - $120 per hour

     ..., General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: General Sales / GTM Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate... 
    Full time
    Contract work
    Summer work
    Work at office
    Remote work
    Flexible hours

    Mercor

    Toronto, ON
    1 day ago
  • $80 - $120 per hour

     ...Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Nonprofit / philanthropy / community programs Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate... 
    Full time
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $14 - $42 per hour

     ...Location: Remote Duration: Up to 6 months Commitment: 20+ hours/week Role Responsibilities Evaluate AI-generated music across various genres. Rate against detailed quality standards. Compare AI-generated lyrics with published songs... 
    Full time
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $70 - $100 per hour

     ...Compensation: $70–$100/hour Location: Remote Commitment: 20–40 hours/week Role Responsibilities Evaluate AI-generated office artifacts in Excel , Word , and PowerPoint formats. Perform side-by-side comparisons of two candidate... 
    Full time
    Contract work
    Summer work
    Work at office
    Immediate start
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $25 - $45 per hour

     ...25–$45/hour Location: Remote Role Responsibilities Evaluate AI-generated written conversations using predefined rubrics to enhance performance. Draft and refine evaluation rubrics for generalist writing tasks. Apply structured scoring... 
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $70k - $73.85k per year

     ...Posting will be closed on July 29, 2026 at 5:00 pm and this posting is for an existing vacancy.   Objective:   The Research and Evaluation Specialist will lead the analysis and development of sector-based program and policy research and evaluation that will enhance Friendship... 
    Full time
    Internship

    Ofifc

    Toronto, ON
    1 day ago
  • $80 - $120 per hour

     ...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Process improvement / SOPs Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate... 
    Full time
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $15 - $20 per hour

     ...Responsibilities Conduct fact-checking using trusted public sources and external tools . Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies. Assess reasoning quality, clarity, tone... 
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $20 - $60 per hour

     ...This is a fully remote position , open to candidates based in Canada. Pay: $20–$60/hr Role Title: Evaluation Specialist/Recent Grad Role Type: Contractor Location: Remote We are engaging Evaluation Specialists/Recent Grads to contribute to a high-impact project... 
    Remote job
    For contractors

    RemoteJobsOne

    Toronto, ON
    2 days ago
  • $120 - $170 per hour

     ...comfortable having your judgment reviewed and calibrated against peers. Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20–30 mins to complete) Submit your resume or relevant technical... 
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $94 - $119 per hour

     ...Jack Dorsey . Position: Applied Health & Medicine Benchmark Specialist Type: Contract Compensation: $94–$119/hour...  ...in a specific subdomain. Strong command of graduate-level medical knowledge, clinical reasoning, and biomedical research methodology... 
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $80 - $160 per hour

     ...Provide structured feedback to enhance document relevance and impact. Utilize strong analytical skills to contribute to high-impact evaluation projects. Collaborate with teams to improve document standards and guidelines. Work independently and asynchronously to... 
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    Toronto, ON
    20 days ago
  • $60 - $80 per hour

     ...investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Medical Secretaries and Administrative Assistants Type: Contract Compensation: $60–$80/hour Location: Remote... 
    Full time
    Contract work
    Summer work
    Immediate start
    Remote work
    Flexible hours

    Mercor

    Toronto, ON
    1 day ago
  • $50k - $70k per year

     ...who want meaningful work and real exposure to healthcare operations. You will speak directly with patients, collect and organize medical histories, and help ensure medical records are accurate and complete. The work is structured, repeatable, and supported by templates... 
    Full time
    Work at office
    2 days per week
    1 day per week

    Stealth Hc

    Toronto, ON
    1 day ago
  • $85 - $93 per hour

     ..., Larry Summers , and Jack Dorsey . Position: Basic Medical Sciences Expert Type: Contract Compensation:...  ...Role Responsibilities Author and review technical tasks to evaluate AI model performance on basic medical sciences problems.... 
    Hourly pay
    Full time
    Contract work
    For contractors
    Summer work
    Remote work
    Flexible hours

    Mercor

    Toronto, ON
    1 day ago
  • $65.91k - $82.39k per year

     ...Full-Time Closing Date: August 28, 2026 Duties Facilitate and coordinate the submission of application forms for access to medication assistance for outpatients Serve as the centralized point of contact for physicians, nurses, social workers and pharmacists, and,... 
    Remplacement
    Permanent employment
    Full time
    Internship
    Flexible hours

    University Health Network

    Toronto, ON
    1 day ago
  • $70 - $100 per hour

     ...Compensation: $70–$100/hour Location: Remote Commitment: 10–20+ hours/week Role Responsibilities Evaluate AI-generated office artifacts across Excel , Word , and PowerPoint formats. Perform pairwise comparisons of candidate responses... 
    Remote job
    Contract work
    Summer work
    Work at office
    Immediate start

    Mercor

    Toronto, ON
    a month ago
  • $70 - $100 per hour

     ...PowerPoint (.pptx) formats. Perform side-by-side pairwise comparisons of two candidate responses and select the stronger one. Evaluate both the content and rendering of each document—layout, formatting, visual structure, formulas, and overall presentation against the... 
    Remote job
    Contract work
    Summer work
    Work at office
    Immediate start

    Mercor

    Toronto, ON
    a month ago
  • $70 - $100 per hour

     ...Compensation: $70–$100/hour Location: Remote Commitment: 10–20+ hours/week Role Responsibilities Evaluate and compare AI-generated office artifacts in Excel , Word , and PowerPoint formats. Perform side-by-side comparisons of... 
    Remote job
    Contract work
    Summer work
    Work at office
    Immediate start

    Mercor

    Toronto, ON
    a month ago
  • $80 - $120 per hour

     ...and Jack Dorsey . Position: Humanities / arts / culture Evaluator Type: Contract Compensation: $80–$120/hour...  ...Work independently and asynchronously to ensure high-quality evaluations. Collaborate with AI research teams to enhance training... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    a month ago
  • $50k per year

     ...resources and expertise of a company that serves over 40% of practitioners nationwide.  The opportunity DoctorCare is hiring a Medical Billing Specialist to join the Customer Operations team. As our new Medical Billing Specialist, you will play a key role in helping physician... 
    Long term contract
    Full time
    Work at office
    Shift work

    WELLSTAR

    Toronto, ON
    8 days ago
  • $80 - $120 per hour

     ...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    Toronto, ON
    1 day ago
  • $18.5 per hour

    Overview Languages English Education Other trades certificate or diploma Medical office management/administration Medical office assistant/specialist Medical reception/receptionist Medical administrative/executive assistant and medical secretary... 
    Permanent employment
    Part time
    Work at office
    Remote work

    Compasscare Stockyards

    Toronto, ON
    4 days ago
  • $25 - $40 per hour

     ...candidates based in Canada. Pay: $25$40/hr Role Title: Medical Coder Role Type: Contractor Location: Remote We are engaging...  ...of ICD-10 and CPT codes based on documentation. # Apply Evaluation & Management (E&M) leveling using official guidelines to validate... 
    Full time
    For contractors
    Remote work

    RemoteJobsOne

    Toronto, ON
    3 hours ago
  • $30 - $60 per hour

     ...to candidates based in Canada. Pay: $30$60/hr Role Title: Medical Writer (Pharmaceutical) Role Type: Contractor Location:...  ...what matters. Scope of Work # Author and review realistic evaluation tasks based on Clinical Study Reports DSURs PSURs/PBRERs and related... 
    Full time
    For contractors
    Remote work

    RemoteJobsOne

    Toronto, ON
    3 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Medical Evaluation Specialist. Be the first to apply!