Medical Evaluation Specialist
$40 - $90 per hourRemoteJobsOne
IMPORTANT: after you apply please check your email. We send you a link to complete your application it is not considered until that last step is done. If you do not see it check your spam folder.
This is a fully remote position open to candidates based in Canada.
Pay: $40$90/hr
Role Title: Medical Evaluation Specialist (students residents physicians)
Role Type: Contractor
Location: Remote
We are engaging Medical Evaluation Specialists (students residents physicians) to contribute advanced clinical insights to a cutting-edge customer this role youll apply your expertise to help train next-generation AI systems. Your work will shape how models learn reason and perform through high-quality real-world input. No prior experience in AI is required your domain knowledge is what matters. As a contributor you will design and verify high-quality challenging medical questions and answers that probe the depth and reliability of todays frontier AI focusing on the places where nuanced medical reasoning makes the difference.
Scope of Work
- Craft original high-difficulty medical question-and-answer pairs covering diagnosis mechanisms pharmacology guidelines and clinical judgment.
- Source and verify answers using primary literature clinical guidelines and authoritative references; thoroughly document rationale and citations.
- Develop questions requiring synthesis and genuine clinical reasoning rather than simple fact recall.
- Evaluate AI model responses to your questions refine those that are too straightforward and enhance complexity while ensuring clinical accuracy.
- Ensure all questions and answers are written with clarity precision and defensibility avoiding ambiguity.
- Incorporate reviewer feedback and align deliverables with evolving project guidelines and standards.
Preferred Qualifications
- Current medical training or clinical practice as a student resident or physician or equivalent biomedical expertise.
- Demonstrated ability to locate interpret and synthesize information from primary literature and clinical guidelines.
- Exceptional attention to detail and proficiency in written English (native fluency not required).
- Talent for creating engaging well-structured and clinically nuanced questions.
- Self-motivated and reliable in managing independent remote contributions.
- Experience with medical question writing peer review or evaluative projects is an advantage.
- Familiarity with AI/ML systems or model evaluation is a plus but not essential.
Compensation Structure
Compensation is output-based; experts are paid per task that meets the project specifications. The time required to complete work may vary depending on the experts experience and workflow. Minimum submission requirements apply. Experts must submit a minimum of tasks per week.
Start Timeline & Availability
We typically fill roles within 48 hours and are looking for experts ready to jump in right away. If selected we expect you to start your first tasks within 2448 hours of completing onboarding.
How to apply
Apply here with your profile. We will then email you a link to complete your application on the hiring platform your application is only considered once that final step is done.
$40 - $90 per hour
...spam folder. This is a fully remote position open to candidates based in Canada. Pay: $40$90/hr Role Title: Research Evaluation Specialist (PhD / Researcher / Professor) Role Type: Contractor Location: Remote We are engaging Research Evaluation Specialists...SuggestedFull timeFor contractorsRemote work$15 - $20 per hour
...include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Video Evaluation Generalist Type: Contract Compensation: $15–$20/hour Location: Remote Commitment: 20 hours/...SuggestedFull timeContract workSummer workRemote work$80 - $160 per hour
...Provide structured feedback to enhance document relevance and impact. Utilize strong analytical skills to contribute to high-impact evaluation projects. Collaborate with teams to improve document standards and guidelines. Work independently and asynchronously to...SuggestedFull timeContract workSummer workRemote work$80 - $150 per hour
..., and Jack Dorsey . Position: MS Excel / Google Sheets Evaluator Type: Contract Compensation: $80–$150/hour... ...relevance. Provide structured feedback to enhance document evaluation processes. Utilize MS Excel and Google Slides to perform...SuggestedFull timeContract workSummer workRemote work$90 per hour
...impact. ~ Professional written English for task justification. Preferred Prior RLHF , preference-labeling, model-evaluation, or structured code-review work. Pixel-perfect design-to-code experience. Accessibility expertise ( ARIA , semantic...SuggestedFull timeContract workSummer workLocal areaRemote work$80 - $120 per hour
..., General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: General Sales / GTM Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote workFlexible hours$80 - $120 per hour
...Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Nonprofit / philanthropy / community programs Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote work$14 - $42 per hour
...Location: Remote Duration: Up to 6 months Commitment: 20+ hours/week Role Responsibilities Evaluate AI-generated music across various genres. Rate against detailed quality standards. Compare AI-generated lyrics with published songs...Full timeContract workSummer workImmediate startRemote work$70 - $100 per hour
...Compensation: $70–$100/hour Location: Remote Commitment: 20–40 hours/week Role Responsibilities Evaluate AI-generated office artifacts in Excel , Word , and PowerPoint formats. Perform side-by-side comparisons of two candidate...Full timeContract workSummer workWork at officeImmediate startRemote work$25 - $45 per hour
...25–$45/hour Location: Remote Role Responsibilities Evaluate AI-generated written conversations using predefined rubrics to enhance performance. Draft and refine evaluation rubrics for generalist writing tasks. Apply structured scoring...Full timeContract workSummer workRemote work$70k - $73.85k per year
...Posting will be closed on July 29, 2026 at 5:00 pm and this posting is for an existing vacancy. Objective: The Research and Evaluation Specialist will lead the analysis and development of sector-based program and policy research and evaluation that will enhance Friendship...Full timeInternship$80 - $120 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Process improvement / SOPs Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote work$15 - $20 per hour
...Responsibilities Conduct fact-checking using trusted public sources and external tools . Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies. Assess reasoning quality, clarity, tone...Full timeContract workSummer workRemote work$20 - $60 per hour
...This is a fully remote position , open to candidates based in Canada. Pay: $20–$60/hr Role Title: Evaluation Specialist/Recent Grad Role Type: Contractor Location: Remote We are engaging Evaluation Specialists/Recent Grads to contribute to a high-impact project...Remote jobFor contractors$120 - $170 per hour
...comfortable having your judgment reviewed and calibrated against peers. Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20–30 mins to complete) Submit your resume or relevant technical...Full timeContract workSummer workRemote work$94 - $119 per hour
...Jack Dorsey . Position: Applied Health & Medicine Benchmark Specialist Type: Contract Compensation: $94–$119/hour... ...in a specific subdomain. Strong command of graduate-level medical knowledge, clinical reasoning, and biomedical research methodology...Full timeContract workSummer workRemote work$80 - $160 per hour
...Provide structured feedback to enhance document relevance and impact. Utilize strong analytical skills to contribute to high-impact evaluation projects. Collaborate with teams to improve document standards and guidelines. Work independently and asynchronously to...Full timeContract workSummer workRemote work$60 - $80 per hour
...investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Medical Secretaries and Administrative Assistants Type: Contract Compensation: $60–$80/hour Location: Remote...Full timeContract workSummer workImmediate startRemote workFlexible hours$50k - $70k per year
...who want meaningful work and real exposure to healthcare operations. You will speak directly with patients, collect and organize medical histories, and help ensure medical records are accurate and complete. The work is structured, repeatable, and supported by templates...Full timeWork at office2 days per week1 day per week$85 - $93 per hour
..., Larry Summers , and Jack Dorsey . Position: Basic Medical Sciences Expert Type: Contract Compensation:... ...Role Responsibilities Author and review technical tasks to evaluate AI model performance on basic medical sciences problems....Hourly payFull timeContract workFor contractorsSummer workRemote workFlexible hours$65.91k - $82.39k per year
...Full-Time Closing Date: August 28, 2026 Duties Facilitate and coordinate the submission of application forms for access to medication assistance for outpatients Serve as the centralized point of contact for physicians, nurses, social workers and pharmacists, and,...RemplacementPermanent employmentFull timeInternshipFlexible hours$70 - $100 per hour
...Compensation: $70–$100/hour Location: Remote Commitment: 10–20+ hours/week Role Responsibilities Evaluate AI-generated office artifacts across Excel , Word , and PowerPoint formats. Perform pairwise comparisons of candidate responses...Remote jobContract workSummer workWork at officeImmediate start$70 - $100 per hour
...PowerPoint (.pptx) formats. Perform side-by-side pairwise comparisons of two candidate responses and select the stronger one. Evaluate both the content and rendering of each document—layout, formatting, visual structure, formulas, and overall presentation against the...Remote jobContract workSummer workWork at officeImmediate start$70 - $100 per hour
...Compensation: $70–$100/hour Location: Remote Commitment: 10–20+ hours/week Role Responsibilities Evaluate and compare AI-generated office artifacts in Excel , Word , and PowerPoint formats. Perform side-by-side comparisons of...Remote jobContract workSummer workWork at officeImmediate start$80 - $120 per hour
...and Jack Dorsey . Position: Humanities / arts / culture Evaluator Type: Contract Compensation: $80–$120/hour... ...Work independently and asynchronously to ensure high-quality evaluations. Collaborate with AI research teams to enhance training...Remote jobContract workSummer workWork at office$50k per year
...resources and expertise of a company that serves over 40% of practitioners nationwide. The opportunity DoctorCare is hiring a Medical Billing Specialist to join the Customer Operations team. As our new Medical Billing Specialist, you will play a key role in helping physician...Long term contractFull timeWork at officeShift work$80 - $120 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Contract workSummer workWork at officeRemote work$18.5 per hour
Overview Languages English Education Other trades certificate or diploma Medical office management/administration Medical office assistant/specialist Medical reception/receptionist Medical administrative/executive assistant and medical secretary...Permanent employmentPart timeWork at officeRemote work$25 - $40 per hour
...candidates based in Canada. Pay: $25$40/hr Role Title: Medical Coder Role Type: Contractor Location: Remote We are engaging... ...of ICD-10 and CPT codes based on documentation. # Apply Evaluation & Management (E&M) leveling using official guidelines to validate...Full timeFor contractorsRemote work$30 - $60 per hour
...to candidates based in Canada. Pay: $30$60/hr Role Title: Medical Writer (Pharmaceutical) Role Type: Contractor Location:... ...what matters. Scope of Work # Author and review realistic evaluation tasks based on Clinical Study Reports DSURs PSURs/PBRERs and related...Full timeFor contractorsRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Medical Evaluation Specialist. Be the first to apply!
- medical administrative specialist Toronto, ON
- laboratory technologist Toronto, ON
- medical technologist Toronto, ON
- medical billing specialist Toronto, ON
- medical reimbursement specialist Toronto, ON
- clinical scientist Toronto, ON
- trainee biomedical scientist Toronto, ON
- medical information specialist Toronto, ON
- travel medical technologist Toronto, ON
- medical imaging technologist Toronto, ON
