Rubric Evaluation Specialist - Writing
$25 - $45 per hourFull-time
Mercor
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Generalist Writer
Type: Contract
Compensation: $25–$45/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated written conversations using predefined rubrics to enhance performance.
- Draft and refine evaluation rubrics for generalist writing tasks.
- Apply structured scoring criteria to assess clarity, coherence, accuracy, and style.
- Review edge cases and ambiguous outputs to ensure consistent grading standards.
- Work independently and asynchronously to meet deadlines and improve AI model performance .
Qualifications
Must-Have
- Experience with rubric-based grading , editing, or content evaluation at Mercor .
- Strong generalist writing skills across multiple domains (e.g., business, academic, creative).
- Excellent attention to detail and consistency in applying guidelines.
- Ability to interpret nuanced criteria and make sound judgment calls.
Preferred
- Background in English , journalism, communications, or a related field.
Compensation & Legal
- Opportunities for expanded reviewer/lead responsibilities.
Application Process (Takes 20–30 mins to complete)
- Submit a resume or writing sample to get started.
- Selected applicants will be contacted within a few days regarding next steps.
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on jobs.jobcopilot.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 7 hours ago
Similar jobs that could be interesting for youBased on the Rubric Evaluation Specialist - Writing in Toronto, ON vacancy
$35 per hour
...Angelo , Larry Summers , and Jack Dorsey . Position: Writing Expert Type: Freelance Compensation: $35/hour... .... Collaborate in QA review processes to ensure prompt tasks and rubrics meet rigor, maintaining consistency and reliability before integration...SuggestedFull timeSummer workFreelanceRemote work$25 - $35 per hour
...Jack Dorsey . Position: AI Model Evaluation Contractor Type: Contract... .../week Role Responsibilities Write realistic prompts reflecting professional... ...and rank model responses using structured rubrics. Provide written justifications with...SuggestedFull timeContract workFor contractorsSummer work$73 per hour
...Position: Chemistry AI Evaluator Type: Contract... ...Role Responsibilities Write and refine prompts to guide model... ...audiences. Apply consistent evaluation standards by following clear taxonomies... ...Familiarity with evaluation rubrics, benchmarks, or structured...SuggestedFull timeContract workSummer workRemote work$75 per hour
...Writer Type: Contract Compensation: $75/hour Commitment: 15 hours/week Role Responsibilities Write realistic prompts for professional writing tasks in corporate settings, including memos, proposals, and stakeholder updates. Craft...SuggestedFull timeContract workSummer workRemote workFlexible hours$80 - $150 per hour
..., and Jack Dorsey . Position: MS Excel / Google Sheets Evaluator Type: Contract Compensation: $80–$150/hour... ...relevance. Provide structured feedback to enhance document evaluation processes. Utilize MS Excel and Google Slides to perform...SuggestedContract workSummer workRemote work$80 - $120 per hour
.... Position: Nonprofit / philanthropy / community programs Evaluator Type: Contract Compensation: $80–$120/hour... ...Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and presentation errors in outputs...Remote jobContract workSummer workWork at office$80 - $120 per hour
...and Jack Dorsey . Position: Process improvement / SOPs Evaluator Type: Contract Compensation: $80–$120/hour... ...Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and presentation errors....Remote jobContract workSummer workWork at office$55 - $75 per hour
...history, political science, or related discipline highly preferred. At least 3 years of experience in project management, technical writing, or an associated field. Proven mastery of technical, business, or procedural writing with a portfolio of high-impact...Full timeSummer workRemote work$20 - $25 per hour
...Always, we partner with clients to build the infrastructure that powers the future. Under the supervision of the Manager, Technical Writing, the Technical Writing Co-op supports the development of technical proposal content across electric, energy, and telecom...Hourly pay$80 - $120 per hour
...Dorsey . Position: Customer success / support operations Evaluator Type: Contract Compensation: $80–$120/hour... ...Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and presentation errors in documents...Full timeContract workSummer workWork at officeRemote work$80 - $120 per hour
.... Position: Healthcare / clinical Evaluator Type: Contract Compensation... ...against domain-specific quality rubrics. Identify factual, aesthetic, and presentation... ...asynchronously to ensure high-quality evaluations. Utilize Microsoft Office and...Remote jobContract workSummer workWork at office$68k - $72k per year
...work together as a community—and you can be a part of that. Our team is looking to fill a new position of Manager, Planning and Evaluation. This is a full-time position working 35 hours per week anywhere in Canada. Please apply by August 18th, 2026. Discover. Innovate...Full timeInternshipWork at officeFlexible hoursWeekend work$62.48 per hour
...Dates of appointment: January 1, 2026 – April 30, 2026 Hours: 43 Hours Salary: CUPE minimum salary rates are: Writing Instructor 1 - $56.46/hr plus 4% vacation pay; Writing Instructor 1 Long Term - $60.69/hr plus 4% vacation pay; Writing Instructor 2...Long term contractPermanent employmentFlexible hours- ...week Affiliation: L79 Full-time Number of Positions Open: 1 Posting Period: 06-Aug-2026 to 20-Aug-2026 Reporting to the Manager, Evaluation and Quality Improvement, the Program Evaluator works collaboratively on multidisciplinary teams to provide continuous quality...Hourly payFull timeTemporary workPart timeInternshipMonday to friday
$122k - $192k per year
...Resource Innovations is looking for an experienced DSM Evaluation Manager to join our expanding team. In this role, you will provide technical... ...savings estimation. Lead and contribute to technical report writing, including development of findings, conclusions, and actionable...Full timeInternshipWork at officeLocal area$50 per hour
...Position: Internet-Native Bilingual Evaluator Expert (French) Type: Contract... ...sarcasm, and code-switching. Build evaluation rubrics for bilingual internet communication and... ...culture across both languages. Strong writing, analytical, and critical thinking...Remote jobContract workSummer work10 hours per week$80 - $150 per hour
...Work independently and asynchronously to evaluate and improve AI model performance .... ...Design Systems Design , UX Lead , or UX Writing . ~ Deep fluency in interaction and... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...Remote jobContract workSummer work- ...Description: Course number and title: INI101H1S: Writing, Advocacy, and Social Media Course description: This course... ..., evidence of teaching in the relevant area (including student evaluations, if available), and the CUPE 3902 Unit 3 application form located...Long term contractWork at office
$80k - $180k per year
...re building the platforms that help regulated organizations adopt AI responsibly and at scale. What’s the Opportunity? The AI Evaluation Engineer ensures AI agent solutions are accurate, reliable, safe, and production-ready within regulated banking environments. The role...Full time$35 - $40 per hour
...Jack Dorsey . Position: Writing Expert Type:... ...instructions. Define and document evaluation standards for correct responses... ...contexts. Develop comprehensive rubrics. Conduct model testing... ...-minute written assessment on rubric writing. If selected, you...Full timeSummer workFreelanceRemote work$80 - $120 per hour
...Dorsey . Position: Market research / competitive intelligence Evaluator Type: Contract Compensation: $80–$120/hour... ...Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and presentation errors in...Remote jobContract workSummer workWork at officeFlexible hours- ...We are seeking Audio Evaluators to participate in an exciting project focused on evaluating... ...Pakistani) and evaluate them using a defined rubric. • Accurately identify target accents... ...and objective judgment based on provided evaluation guidelines. • Maintain high quality...Remote jobContract workFreelance
$37.85 - $47.31 per hour
...multidisciplinary staff. Duties Plan and execute technology evaluation studies using validated frameworks such as the Technology... ...written and verbal communication skills, including the ability to write for diverse audiences (academic, clinical, public). Strong organizational...Hourly payLong term contractRemplacementFull timeContract workTemporary workInternshipFlexible hours$85 per hour
...Compensation: $85/hour Location: Remote Role Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving cloud platforms , Kubernetes ,...Full timeContract workSummer workRemote work$80 - $120 per hour
...Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80–$120/hour... ...Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and presentation errors in documents...Remote jobContract workSummer workWork at office$73 per hour
...Position: Engineering AI Evaluator Type: Contract... ...Role Responsibilities Write and refine prompts to guide model... ...experience with RLHF , model evaluation, or data annotation work... ...Familiarity with evaluation rubrics or structured review frameworks...Full timeContract workSummer workRemote work$73 per hour
...Position: Mathematics AI Evaluator Type: Full-time or Part... ...Role Responsibilities Write and refine prompts to guide model... ...with RLHF , model evaluation, or data annotation work.... ...Familiarity with evaluation rubrics, benchmarks, or structured review...Full timeContract workPart timeSummer workRemote work$120 - $170 per hour
...comfortable having your judgment reviewed and calibrated against peers. Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20–30 mins to complete) Submit your resume or relevant technical...Remote jobContract workSummer work$50 per hour
...Position: Lithuanian Audio Generalist Evaluator Expert Type: Contract... ...when required. Develop detailed evaluation rubrics and grading guidelines in Lithuanian and... ...Qualifications Must-Have Strong writing, editing, and critical thinking skills....Full timeContract workSummer work10 hours per week$25 - $37 per hour
...and France . Establish evaluation standards for correct responses... ...in consumer contexts. Develop rubrics accounting for linguistic nuance... ..., or France . Strong reading/writing ability in English .... ...assessment focused on writing and rubric creation. Then, if selected...Full timeContract workSummer workRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Rubric Evaluation Specialist - Writing. Be the first to apply!
Related searches
- spécialiste service client Toronto, ON
- member banking specialist Toronto, ON
- member engagement specialist Toronto, ON
- spécialiste talent Toronto, ON
- order management specialist Toronto, ON
- solution specialist Toronto, ON
- building performance specialist Toronto, ON
- public health specialist Toronto, ON
- access control specialist Toronto, ON
- rbc credit specialist Toronto, ON