AI Media Quality Evaluator
Scale AI
Evaluate AI-generated images, videos, and voice conversations remotely while working independently on your own schedule with Outlier. Strong English skills and careful observation are central.
Outlier, operated by Scale AI, seeks detail-oriented English speakers for AI training projects. You will assess model outputs, compare media against prompts, and explain findings in writing. The work supports feedback loops that help AI systems improve their real-world performance.
Key Responsibilities:
- Conduct live English conversations with AI voice models
- Compare generated images and videos against text prompts
- Inspect outputs for visual, motion, text, and audio issues
- Rate adherence, quality, realism, and naturalness
- Write specific evidence-based explanations in English
Requirements:
- Fluent English and clear written communication
- Strong attention to detail and patience
- Comfort with natural conversational situations
- Experience writing concise reviews or explanations
- Valid ID, mobile phone, resume, and LinkedIn profile
Apply your English communication, media judgment, and attention to detail to evaluate AI outputs for Outlier projects.
#J-18808-LjbffrVacancy posted 3 hours ago
Similar jobs that could be interesting for youBased on the AI Media Quality Evaluator in Toronto, ON vacancy
$126k - $176k per year
...Sr. Legal AI Prompter / Evaluator Job Description Sr. Legal AI Prompters / Evaluators are responsible... ...legal expertise into practical, high-quality AI outputs. Key Responsibilities Create... ..., compliance, government, and media. Our products combine highly specialized...QualityRemplacementFull timeWork at officeLocal areaFlexible hours- ...changelog . About the Role Rootly is building the AI-native future of incident management, and we need someone... ...our AI to its limits before our customers do. As our AI Quality Engineer, you'll own the evaluation and optimization of Rootly's agentic AI features -- designing...QualityFull timeInternshipHome office
- ...specialists with project-based AI opportunities for leading tech... ..., focused on testing, evaluating, and improving AI systems. This... ...EHR vignettes for a medical AI evaluation project. General and internal... ...Score the clinical reasoning quality of benchmark items: vignette accuracy...QualityPart timeFor contractorsRemote workFlexible hours
$30 - $50 per hour
...competitive advantage. Cresta’s unified AI platform combines... ...our AI Delivery team as AI Quality Assurance Interns . This is... ...hands-on role where you will evaluate the behavior, accuracy, and reliability... ...Intent recognition evaluation Conversation quality assessment...QualityHourly payInternshipWork at office$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ..., and technical reports. Evaluate AI-generated or human-... ...established criteria to ensure quality and accuracy. Provide detailed... ...written justifications for evaluations and scores to maintain transparency...QualityRemote jobContract workSummer work$30 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors... ...tasks guiding it answering questions and correcting errors. Evaluate the experience and deliverables providing concise feedback on performance...Hourly payWeekly payFull timeContract workFor contractorsSummer workRemote work$140k - $175k per year
...ready software for artificial intelligence (AI) model evaluation and benchmarking. You will build... ...compare large language models (LLMs), measure quality, cost, and latency, and make informed... ...accounting, compliance, government, and media. Our products combine highly specialized...QualityFull timeInternshipWork at officeLocal areaFlexible hours$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and Jack Dorsey . Position: Media / journalism / communications Evaluator Type: Contract... ...generated artifacts against domain-specific quality rubrics. Identify factual,...QualityRemote jobContract workSummer workWork at office$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and Jack Dorsey . Position: Media / journalism / communications Evaluator Type: Contract... ...artifacts against domain-specific quality rubrics. Identify factual, aesthetic...QualityRemote jobContract workSummer workWork at office$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Government / public administration Evaluator Type: Contract Compensation... ...generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic,...QualityRemote jobContract workSummer workWork at office$70 per hour
...technical talent with leading AI research labs. Headquartered in... ...Role Responsibilities Evaluate AI-generated responses to... ...feedback to improve training data quality and downstream performance.... ...application of structured evaluation guidelines. Work independently...QualityRemote jobContract workSummer work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Software / AI / IT / data Evaluator Type: Contract Compensation... ...generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic,...QualityRemote jobContract workSummer workWork at office$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...A/B test write-ups . Evaluate AI-generated or human-created... ...established criteria to ensure quality and consistency. Provide... ...detailed written justifications for evaluations and scores to maintain...QualityRemote jobContract workSummer work$39 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation... ...other standards in music. Score the quality of musical training data to enhance model...QualityRemote jobContract workSummer workImmediate start$20 - $160 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Generalist Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $20–$160/hour...QualityRemote jobContract workSummer work- ...Quality Engineer (QE) / SDET – AI Chatbot Validation Job Description: We are looking for a seasoned AI first Senior Quality Engineer who is... ...deterministic and non-deterministic AI behavior, including evaluation datasets, expected behavior, quality thresholds, and...QualityContract work
$70 per hour
...About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position...Remote jobContract workSummer workImmediate start$80 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex data engineering tasks. Review model-generated...Remote jobContract workSummer work$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-...Remote jobContract workSummer work$120 - $170 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20–30...Remote jobContract workSummer work$60 - $100 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Marketing Expert — Social Media (AI Agent Environments) Type: Contract... ...wrong ones. Review AI agent attempts and evaluate them against established standards. Diagnose...Remote jobContract workSummer work$52.38 - $57.39 per hour
...Branch of Parks, Forestry & Recreation, the Evaluator, Programs will help design, plan, and... ...the implementation of small and large evaluations, performance measurement frameworks, needs... ..., improvement, and communication of quality of life outcomes for the public. Major...QualityHourly payPermanent employmentFull timePart timeInternshipMonday to fridayShift work$45 - $65 per hour
...Quality Assurance Specialist - Accessibility Job Description Join our Enterprise Digital... .... Use of Artificial Intelligence (AI): We may use Artificial Intelligence (AI)... ...process, including sourcing, screening, and evaluating candidates. AI helps assess applications and...QualityContract work$60k - $65k per year
...effectively to optimizing daily media channel performance and... ...supports the measurement and evaluation of media performance, helping... ...media platforms. Monitor data quality by identifying discrepancies and... ...to all permanent employees. AI Disclosure: [ dentsu ] utilizes...QualityPermanent employmentFull timeContract workTemporary workInternship$30 - $40 per hour
...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...assigned queries each day to ensure timely evaluation. Provide short notes and annotations to enhance training data quality . Work independently and asynchronously...QualityDaily paidFull timeContract workSummer workRemote workFlexible hours- ...that celebrates authenticity, quality, and community. Inspired by the... ...local product inspections, evaluating TOP samples, reviewing overseas... ...Make an Impact TOP Samples Evaluation Evaluate TOP samples against... ...artificial intelligence (AI), to support parts of the recruitment...QualityPermanent employmentFull timeSeasonal workWork at officeLocal areaOverseas
$150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...hours/week Role Responsibilities Evaluate and compare AI-generated responses on challenging... ...on clinical accuracy, safety, and quality of medical reasoning. Provide written...QualityRemote jobContract workSummer workImmediate start- ...through world-class networks, AI-powered solutions, and digital... ...us? At Bell, you'll join the Media program and become part of a... ...end-to-end sales processes and evaluating performance to help us remain... ...news, and radio. Leadership qualities demonstrated through extracurricular...QualityInternshipWork at office3 days per week
$60k - $75k per year
...ROLE: MEDIA SPECIALIST TEAM: MEDIA LOCATION: TORONTO - HYBRID ONSITE 2X/WEEK COMPANY... ...people combine strategic thinking with AI-powered tools, automation and advanced... ...accountability of the media plan, regularly evaluating and monitoring execution to implement meaningful...Full timeSeasonal work$50 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Location: Remote Role Responsibilities Evaluate the quality and accuracy of LLM-generated French text across various use...QualityRemote jobContract workSummer work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Media Quality Evaluator. Be the first to apply!
Related searches
- admissions evaluator Toronto, ON
- évaluateur-estimateur Toronto, ON
- remote social media evaluator Toronto, ON
- estimateur - dessinateur Toronto, ON
- test evaluator Toronto, ON
- estimateur automobile Toronto, ON
- estimateur en bâtiment Toronto, ON
- estimateur structure Toronto, ON
- estimateur industriel Toronto, ON
- chargé de projet estimateur Toronto, ON