Chemistry PhD LLM Evaluation Specialist
$73 per hourFull-time
Mercor
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Chemistry AI Evaluator
Type: Contract
Compensation: $73/hour
Location: Remote
Role Responsibilities
- Write and refine prompts to guide model behavior in chemistry contexts.
- Evaluate LLM-generated responses to chemistry-related queries for scientific accuracy and mechanistic correctness.
- Conduct fact-checking using authoritative public sources and domain knowledge.
- Annotate model responses by identifying strengths, areas of improvement, and factual or conceptual inaccuracies.
- Assess clarity, structure, and appropriateness of explanations for different audiences.
- Apply consistent evaluation standards by following clear taxonomies, benchmarks, and detailed evaluation guidelines.
Qualifications
Must-Have
- PhD in Chemistry or a closely related field
- Deep expertise in Organic & Biological Chemistry , Inorganic & Materials Chemistry , Physical & Theoretical Chemistry , or Analytical & Instrumental Chemistry
- Significant experience using large language models (LLMs)
- Excellent writing skills and ability to clearly explain complex chemistry concepts
- Strong attention to detail
- Experience reviewing or editing technical or academic writing
Preferred
- Prior experience with RLHF , model evaluation, or data annotation work
- Experience teaching, mentoring, or explaining chemistry concepts to non-expert audiences
- Familiarity with evaluation rubrics, benchmarks, or structured review frameworks
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on jobs.jobcopilot.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 23 hours ago
Similar jobs that could be interesting for youBased on the Chemistry PhD LLM Evaluation Specialist in Toronto, ON vacancy
$73 per hour
...Dorsey . Position: Life Sciences AI Specialist Type: Contract... ...behavior in the life sciences context. Evaluate LLM-generated responses to biology-related queries... ...Qualifications Must-Have PhD in Biology or a closely related life sciences...For phdFull timeContract workSummer workRemote work$15 - $20 per hour
...Responsibilities Conduct fact-checking using trusted public sources and external tools . Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies. Assess reasoning quality, clarity, tone...SuggestedFull timeContract workSummer workRemote work$73 per hour
...Position: Mathematics AI Evaluator Type: Full-time or Part... ...mathematical contexts. Evaluate LLM-generated responses to... ...Qualifications Must-Have PhD in Mathematics or a closely... ...with RLHF , model evaluation, or data annotation work....For phdFull timeContract workPart timeSummer work$73 per hour
.... Position: Physics AI Evaluator Type: Full-time or Part... ...contexts. Evaluate LLM-generated responses to physics... ...audiences. Apply consistent evaluation standards by following clear... ...Qualifications Must-Have PhD in Physics or closely related...For phdFull timeContract workPart timeSummer work$70 per hour
...include Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey . Position: Mathematics PhD Coding Experts Type: Contract Compensation: $70/hour Location: Remote Duration: 6 weeks...For phdFull timeContract workSummer workImmediate startRemote work$55 - $80 per hour
...Jack Dorsey . Position: Phylogenetics PhDs & Domain Experts — Zion (Arches)... ...language models using domain expertise. Evaluate and improve AI model outputs through rigorous... ...Qualifications Must-Have PhD or advanced degree in Evolutionary Biology...For phdFull timeContract workSummer workRemote work$66 - $84 per hour
...Jack Dorsey . Position: Applied Computer Science Benchmark Specialist Type: Contract Compensation: $66–$84/hour... ...precision, or solvability. Qualifications Must-Have PhD or doctoral candidate in Computer Science , Electrical...For phdFull timeContract workSummer workRemote work$61 - $77 per hour
...and Jack Dorsey . Position: Applied Mathematics Benchmark Specialist Type: Contract Compensation: $61–$77/hour... ...justify any edits made. Qualifications Must-Have PhD or doctoral candidate in Mathematics , Applied Mathematics ,...For phdFull timeContract workSummer workRemote work$60 - $80 per hour
...and Jack Dorsey . Position: STEM PhDs Type: Contract Compensation... ...to enhance AI model training . Evaluate and improve AI model outputs through rigorous... ...Must-Have STEM PhD from a US, UK, Canadian, or European university...For phdFull timeContract workSummer workRemote work$50 - $60 per hour
...communications effectively. Demonstrate high agency and strong organizational skills. Apply an analytical mindset to improve LLM training projects . Own unfamiliar problems from start to finish. Qualifications Must-Have Strong...Full timeContract workSummer workRemote work$77 - $98 per hour
...Dorsey . Position: Applied Economics & Finance Benchmark Specialist Type: Contract Compensation: $77$98/hour... ...justify any edits made. Qualifications Must-Have PhD or doctoral candidate in Economics Finance or a closely related...For phdFull timeContract workSummer workRemote work$80 - $120 per hour
...Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Nonprofit / philanthropy / community programs Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote work$80 - $120 per hour
..., General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: General Sales / GTM Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote workFlexible hours$100 per hour
...$100–$500/hour Location: Remote Role Responsibilities Build and operate LLM agents in production environments. Ensure agent reliability and evaluate performance improvements. Manage long-running processes like scheduled jobs and cloud sandboxes...Remote jobContract workSummer work$80 - $150 per hour
..., and Jack Dorsey . Position: MS Excel / Google Sheets Evaluator Type: Contract Compensation: $80–$150/hour... ...relevance. Provide structured feedback to enhance document evaluation processes. Utilize MS Excel and Google Slides to perform...Full timeContract workSummer workRemote work$90 per hour
...impact. ~ Professional written English for task justification. Preferred Prior RLHF , preference-labeling, model-evaluation, or structured code-review work. Pixel-perfect design-to-code experience. Accessibility expertise ( ARIA , semantic...Full timeContract workSummer workLocal areaRemote work$70k - $73.85k per year
...Posting will be closed on July 29, 2026 at 5:00 pm and this posting is for an existing vacancy. Objective: The Research and Evaluation Specialist will lead the analysis and development of sector-based program and policy research and evaluation that will enhance Friendship...Full timeInternship$80 - $120 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Process improvement / SOPs Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote work$14 - $42 per hour
...Location: Remote Duration: Up to 6 months Commitment: 20+ hours/week Role Responsibilities Evaluate AI-generated music across various genres. Rate against detailed quality standards. Compare AI-generated lyrics with published songs...Full timeContract workSummer workImmediate startRemote work$70 - $100 per hour
...Compensation: $70–$100/hour Location: Remote Commitment: 20–40 hours/week Role Responsibilities Evaluate AI-generated office artifacts in Excel , Word , and PowerPoint formats. Perform side-by-side comparisons of two candidate...Full timeContract workSummer workWork at officeImmediate startRemote work$25 - $45 per hour
...25–$45/hour Location: Remote Role Responsibilities Evaluate AI-generated written conversations using predefined rubrics to enhance performance. Draft and refine evaluation rubrics for generalist writing tasks. Apply structured scoring...Full timeContract workSummer workRemote work$25 - $35 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: AI Model Evaluation Contractor Type: Contract Compensation: $25–$35/hour Commitment: 20 hours/week Role Responsibilities...Full timeContract workFor contractorsSummer work$70 - $100 per hour
...Compensation: $70–$100/hour Location: Remote Commitment: 10–20+ hours/week Role Responsibilities Evaluate AI-generated office artifacts across Excel , Word , and PowerPoint formats. Perform pairwise comparisons of candidate responses...Remote jobContract workSummer workWork at officeImmediate start$91.27k - $97.17k per year
...Position Title: Curriculum Specialist (Intermediate/Senior Physics & Chemistry) Employment Type: Full-time Contract (12-Month) | Union Salary range: $91,270.73 - $97,165.35 Division: Education Reports To: Manager, Curriculum Location: Toronto, Yonge...Full timeContract workImmediate start$70 - $100 per hour
...PowerPoint (.pptx) formats. Perform side-by-side pairwise comparisons of two candidate responses and select the stronger one. Evaluate both the content and rendering of each document—layout, formatting, visual structure, formulas, and overall presentation against the...Remote jobContract workSummer workWork at officeImmediate start$120 - $170 per hour
...comfortable having your judgment reviewed and calibrated against peers. Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20–30 mins to complete) Submit your resume or relevant technical...Remote jobContract workSummer work$70 - $100 per hour
...Compensation: $70–$100/hour Location: Remote Commitment: 10–20+ hours/week Role Responsibilities Evaluate and compare AI-generated office artifacts in Excel , Word , and PowerPoint formats. Perform side-by-side comparisons of...Remote jobContract workSummer workWork at officeImmediate start$80 - $120 per hour
...and Jack Dorsey . Position: Humanities / arts / culture Evaluator Type: Contract Compensation: $80–$120/hour... ...Work independently and asynchronously to ensure high-quality evaluations. Collaborate with AI research teams to enhance training...Remote jobContract workSummer workWork at office$70 - $90 per hour
...Preferred Experience across multiple computational domains or specialized software packages. Familiarity with benchmark or evaluation design. Background in scientific teaching or exam/problem-set design. Experience with computational reproducibility and...For phdRemote jobContract workSummer work$85 per hour
...against three frontier language models until at least one fails. Evaluate the accuracy and depth of AI-generated content in aerodynamics... ...model performance . Qualifications Must-Have ~ PhD in Aerospace Engineering or Mechanical Engineering with a...For phdFull timeContract workSummer workRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Chemistry PhD LLM Evaluation Specialist. Be the first to apply!
Related searches
- crisis intervention specialist Toronto, ON
- leasing specialist Toronto, ON
- visual specialist Toronto, ON
- child life specialist Toronto, ON
- lead generation specialist Toronto, ON
- spécialiste en amélioration continue Toronto, ON
- spécialiste paie Toronto, ON
- literacy specialist Toronto, ON
- building performance specialist Toronto, ON
- employment specialist Toronto, ON