Generalist Expert AI Evaluator

Evaluate AI-generated responses remotely for clarity, accuracy, and human reasoning while developing prompts across generalist subject areas. Work asynchronously as an independent contractor.

Mercor connects creative and technical talent with leading AI research labs and is seeking a Generalist Expert. The assignment lasts approximately two weeks, with potential extension, and requires ten to twenty-five hours weekly. You will benchmark model responses, identify failure points, and maintain consistent quality across summaries, Q&A, and synthesis tasks. Research, writing, editing, education, or related analytical experience supports this work.

Key Responsibilities:

• Review and revise responses for clarity and accuracy
• Create prompts spanning a range of generalist topics
• Benchmark responses by identifying model failure points
• Compare variations in length, style, and system prompts
• Meet deadlines while improving AI model performance

Requirements:

• Two or more years of relevant analytical experience
• Strong ability to explain complex topics concisely
• Careful attention to detail and critical thinking
• Comfort working independently as guidelines evolve
• Familiarity with LLMs preferred

Bring research judgment, concise communication, and response evaluation skills to Mercor’s AI benchmarking and prompt-development work.