Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

User Research Expert - Evaluator

$80 - $120 per hour

Mercor

Toronto, ON
  • Remote job

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: User/customer research and feedback synthesis Evaluator

Type: Contract

Compensation: $80–$120/hour

Location: Remote

Role Responsibilities

  • Evaluate AI-generated artifacts against domain-specific quality rubrics.
  • Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks .
  • Provide clear, structured written feedback to improve AI model outputs .
  • Work independently and asynchronously to meet deadlines and enhance AI model performance .
  • Collaborate with subject matter experts to ensure consistency and quality across datasets.

Qualifications

Must-Have

  • 5+ years of relevant professional experience in User/customer research and feedback synthesis .
  • Native or professional fluency in English .
  • Highly proficient in Microsoft Office and Google Workspace , especially Slides .

Preferred

  • Advanced degree (Master's or higher) from a reputable institution.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information, please check:
  • For any help or support, reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted more than 2 months ago
Similar jobs that could be interesting for youBased on the User Research Expert - Evaluator in Toronto, ON vacancy
  • $80 - $120 per hour

     ...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Larry Summers , and Jack Dorsey . Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–... 
    Suggested
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    more than 2 months ago
  • $80 - $120 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...research / competitive intelligence Evaluator Type: Contract Compensation...  .... Collaborate with subject matter experts to ensure consistency and relevance... 
    Suggested
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    more than 2 months ago
  • $39 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...requests similar to those a real user would type. Collaborate with the AI assistant...  ...concise notes on performance. Evaluate AI model interactions to ensure alignment... 
    Suggested
    Remote job

    Mercor

    Toronto, ON
    10 days ago
  • $39 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors include...  ...Responsibilities Write short realistic requests similar to those a real user would type. Work with the AI assistant on tasks steering it... 
    Suggested
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    Toronto, ON
    6 days ago
  • $57 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ...Write short, realistic requests, similar to what a real user would type. Work with the AI assistant on tasks. Steer it, answer... 
    Suggested
    Remote job
    Hourly pay
    Contract work
    For contractors
    Summer work

    Mercor

    Toronto, ON
    16 days ago
  • $50 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  .... Position: French Linguistic Experts Type: Contract Compensation...  ...Remote Role Responsibilities Evaluate the quality and accuracy of LLM-... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    20 days ago
  • $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Dorsey . Position: Healthcare Expert Type: Contract...  ...hours/week Role Responsibilities Evaluate and compare AI-generated responses on challenging... 
    Remote job
    Contract work
    Summer work
    Immediate start

    Mercor

    Toronto, ON
    21 days ago
  • $140k - $175k per year

     ...Reuters Labs is seeking a Lead Research Engineer to lead the design...  ...artificial intelligence (AI) model evaluation and benchmarking. You will...  ...versions, validate them with users, and improve them through iterative...  ...Department at HR.Leave-Expert@thomsonreuters.com. Disability... 
    Full time
    Internship
    Work at office
    Local area
    Flexible hours

    Thomson Reuters

    Toronto, ON
    10 days ago
  • $120 per hour

     ...and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...Position: Market Research & Insights Experts Type: Contract...  ...marketing strategy recommendations. Evaluate outputs from an end-user perspective focusing on relevance, strategic... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    2 days ago
  •  ...for our customers. Cohere is a team of researchers, engineers, designers, and more, who are...  ...mission and shape the future! Why this role?Evaluation is critical to making progress in...  ...infrastructure to measure LLM progress. As a Senior Research Scientist, Model Evaluation, you will:... 
    Full time
    Work at office
    Remote work
    Flexible hours

    cohere

    Toronto, ON
    10 days ago
  • $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Dorsey . Position: Healthcare Expert Type: Contract...  ...hours/week Role Responsibilities Evaluate and compare AI-generated responses to challenging... 
    Remote job
    Contract work
    Summer work
    Immediate start
    Flexible hours

    Mercor

    Toronto, ON
    a month ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Dorsey . Position: Data Science Expert Type: Contract...  ...Remote Role Responsibilities Evaluate AI-generated artifacts for usability in... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    a month ago
  • $60 - $70 per hour

     ...and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...Dorsey . Position: Application Users - macOS Sonoma 14.5 Expert - Professionals Type:...  ...Remote Role Responsibilities Evaluate and utilize macOS Sonoma 14.5 as your... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  .... Position: Software Engineer Expert Type: Contract Compensation...  ...Remote Role Responsibilities Evaluate AI-generated artifacts for usability in... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    a month ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Jack Dorsey . Position: B2B Sales Expert Type: Contract...  ...~ Prior experience with AI training , evaluation, or human-data projects. Application... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ...Summers , and Jack Dorsey . Position: Healthcare operations Evaluator Type: Contract Compensation: $80–$120/hour... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    more than 2 months ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ..., and Jack Dorsey . Position: People ops / recruiting Evaluator Type: Contract Compensation: $80–$120/hour... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    more than 2 months ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ...Summers , and Jack Dorsey . Position: Legal / compliance Evaluator Type: Contract Compensation: $80–$120/hour... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    more than 2 months ago
  • $70 - $100 per hour

     ...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ..., and Jack Dorsey . Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) Type: Contract... 
    Remote job
    Contract work
    Summer work
    Internship

    Mercor

    Toronto, ON
    more than 2 months ago
  • $50 per hour

     ...technical talent with leading AI research labs. Headquartered in San...  ...Language and Literature Expert Type: Contract...  ...Role Responsibilities Evaluate the quality and accuracy of...  ...linguistic annotation, language evaluation, or NLP data work in English... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    a month ago
  • $14 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Dorsey . Position: Music Audio Expert - Telugu Type: Contract...  ...hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation... 
    Remote job
    Contract work
    Summer work
    Immediate start

    Mercor

    Toronto, ON
    more than 2 months ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  .... Position: Sales Engineering Expert Type: Contract Compensation...  ...~ Prior experience with AI training , evaluation, or human-data projects.... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ...Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80–$120/hour... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    more than 2 months ago
  • $50 - $70 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Dorsey . Position: Generalist Expert (US/Canada) Type: Contract...  ...Role Responsibilities Review and evaluate documents, slides, spreadsheets, and similar... 
    Remote job
    Contract work
    Summer work
    Flexible hours

    Mercor

    Toronto, ON
    more than 2 months ago
  • $80 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Jack Dorsey . Position: Design Expert Type: Contract...  ...~ Prior experience with AI training , evaluation, or human-data projects. Application... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $70 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ...Summers , and Jack Dorsey . Position: Mathematics PhD Coding Experts Type: Contract Compensation: $70/hour... 
    Remote job
    Contract work
    Summer work
    Immediate start

    Mercor

    Toronto, ON
    a month ago
  • $80 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Dorsey . Position: UI / UX Design Expert Type: Contract...  ...~ Prior experience with AI training , evaluation, or human-data projects. Application... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $120 - $170 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Dorsey . Position: Data Science Expert Type: Contract...  ...~ Prior experience with AI training , evaluation, or human-data projects. Application... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $120 - $170 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Dorsey . Position: Data Science Expert Type: Contract...  ...~ Prior experience with AI training , evaluation, or human-data projects. Application... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  .... Position: Machine Learning Expert Type: Contract Compensation...  ...of 8 attempts. Focus on inference, evaluation, and systems work over heavy model... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    22 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to User Research Expert - Evaluator. Be the first to apply!