Analytical Evaluator - AI Feedback Specialist

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Generalist Expert

Type: Contract

Compensation: $60–$200/hour

Location: Remote

Role Responsibilities

• Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs .
• Provide clear, structured feedback to improve training data quality and downstream performance.
• Develop evaluation rubrics and benchmarks for assessing AI-generated tasks .
• Work independently and asynchronously to meet deadlines while improving AI model performance .

Qualifications

Must-Have

• Bachelor's degree from a top-100 globally ranked university preferred.
• Strong analytical and written communication skills.
• Ability to work independently and follow detailed task guidelines.
• Strong critical reading skills with the ability to identify nuance and gaps in reasoning.
• Ability to write clear, precise, and well-evidenced written rationales.
• Consistent and honest judgment with critical assessment skills.
• Strict attention to detail and accurate application of structured evaluation guidelines.
• Ability to work entirely without AI writing tools .
• Native English fluency required.

Preferred

• Candidates based in the United States , United Kingdom , Canada , Australia , or New Zealand .

Application Process (Takes 20–30 mins to complete)

• Upload resume
• AI interview based on your resume
• Submit form

Resources & Support

• For details about the interview process and platform information, please check:
• For any help or support, reach out to: Show email

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.