AI Media Evaluation Specialist
Scale AI
Use English communication and meticulous observation to evaluate AI-generated media remotely, choosing your own schedule on Outlier’s platform. Review voice, images, video, and audio outputs.
As an AI Media Evaluation Specialist, you will assess how generated content follows prompts and performs across quality dimensions. You will roleplay conversations with AI voice models, compare media pairs, inspect outputs closely, and explain findings in writing. Your feedback supports Outlier, Scale AI’s AI training platform, and its evaluation loop.
Key Responsibilities:
- Conduct scenario-based English conversations with AI voice models
- Rank speech for naturalness, usefulness, and clarity
- Compare images and videos against text prompts
- Inspect anatomy, motion, text, and audio-video alignment
- Write evidence-based explanations of observed issues
Requirements:
- Fluent English with clear written communication
- Strong attention to detail and patience
- Comfort discussing realistic situations naturally
- Experience writing concise, evidence-based reviews
- Ability to decide when both options have flaws
Apply your English communication, media judgment, and detailed feedback skills to help improve advanced AI systems through Outlier.
#J-18808-LjbffrVacancy posted 3 hours ago
Similar jobs that could be interesting for youBased on the AI Media Evaluation Specialist in Toronto, ON vacancy
$30 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors... ...tasks guiding it answering questions and correcting errors. Evaluate the experience and deliverables providing concise feedback on performance...SuggestedHourly payWeekly payFull timeContract workFor contractorsSummer workRemote work$60k - $75k per year
...ROLE: MEDIA SPECIALIST TEAM: MEDIA LOCATION: TORONTO - HYBRID ONSITE 2X/WEEK COMPANY OVERVIEW... ...people combine strategic thinking with AI-powered tools, automation and advanced... ...of the media plan, regularly evaluating and monitoring execution to implement meaningful...SuggestedFull timeSeasonal work- ...About the company Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This is a task... ...EHR vignettes for a medical AI evaluation project. General and internal medicine...SuggestedPart timeFor contractorsRemote workFlexible hours
$60k - $68k per year
...Katalyst, and btwelve), Klick Media, Klick Applied Sciences (including... ...-winning data science and AI capabilities. Founded in 1997,... ...Description The Media Planning Specialist supports the development and... ...reports and insights that evaluate campaign effectiveness and...SuggestedFull time$126k - $176k per year
...Sr. Legal AI Prompter / Evaluator Job Description Sr. Legal AI Prompters / Evaluators are responsible... ...datasets with SMEs, coordinate evaluations, and design evaluation schemes, including... ...accounting, compliance, government, and media. Our products combine highly...SuggestedRemplacementFull timeWork at officeLocal areaFlexible hours$140k - $175k per year
...software for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable... ...support ownership, and work with data specialists to implement access controls, permitted... ...accounting, compliance, government, and media. Our products combine highly...Full timeInternshipWork at officeLocal areaFlexible hours$50k - $65k per year
...ROLE: SOCIAL MEDIA SPECIALIST TEAM: CREATOR LOCATION: TORONTO (HYBRID) COMPANY OVERVIEW Salt is a North American marketing agency that... ...improve output. Explore emerging technologies, including AI-powered creative and production tools, to improve workflows and...Full timeSeasonal workWork at officeShift work2 days per week$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...pipelines , and technical reports. Evaluate AI-generated or human-created work against... ...Provide detailed written justifications for evaluations and scores to maintain transparency....Remote jobContract workSummer work$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ..., Larry Summers , and Jack Dorsey . Position: Media / journalism / communications Evaluator Type: Contract Compensation: $80–$120...Remote jobContract workSummer workWork at office$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ..., Larry Summers , and Jack Dorsey . Position: Media / journalism / communications Evaluator Type: Contract Compensation: $80–$120...Remote jobContract workSummer workWork at office$62.85k - $104.75k per year
...Reporting to the Director of Content, Brand, the Social Media Specialist is a key member of Manulife Canada’s Brand team. The role manages and... ...and intended audience. Look for opportunities to integrate AI tools that allow for efficiencies in workflows. Work with optimization...Long term contractTemporary workLocal areaFlexible hours- ...connected future through world-class networks, AI-powered solutions, and digital experiences... ...and ever-changing industries. On the Bell Media team you will be a part of creating and... ...for an experienced, ideally bilingual CRM Specialist to join our dynamic team at the leading...Long term contractFull timeWork at office3 days per week
$90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and CSS : semantic markup, flexbox, grid, media queries, and legacy float- and table-based... ...Prior RLHF , preference-labeling, model-evaluation, or structured code-review work. Pixel...Remote jobContract workSummer workLocal area$64k - $106k per year
...Strategic Sourcing, the Strategic Sourcing Specialist position is responsible for supporting... ...related to Technology and Marketing, including Media and Events. The Strategic Sourcing... ...methodology, including determination of the evaluation and negotiation strategies for assigned...Contract work- ...We are seeking a Freelance Social Media Specialist to to lead end-to-end strategy, content creation, and profile management across TikTok and Instagram. This role sits at the intersection of personal branding, short-form video, and organic audience growth. The ideal candidate...Part timeFreelanceFlexible hours
- ...We are looking for an experienced, hands-on Freelance Paid Media Specialist to manage and optimise paid social campaigns for an established B2B technology company. You will work closely with the Demand Generation team to improve campaign performance across the customer...Freelance
$115k - $165k per year
...growing. We are building a new AI engineering capability within... ...Distinguished AI Engineering Specialist, the most senior technical individual... ...and team members. Evaluate emerging tools and techniques... ...Demonstrated experience establishing evaluation, safety, testing, and...Full timeWork at office2 days per week$80 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: MS Excel / Google Sheets Evaluator Type: Contract Compensation... ...structured feedback to improve document evaluation processes. Utilize MS Excel and Google...Remote jobContract workSummer work$70 - $100 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...comparisons of two candidate responses and select the stronger one. Evaluate both the content and rendering of each document—layout,...Remote jobContract workSummer workWork at officeImmediate start$15 - $20 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: Video Evaluation Generalist Type: Contract Compensation: $15–$...Remote jobContract workSummer work- ...Communication & Social Media Specialist - VIE Contract ~ Location: Canada, Toronto ~ Target start date: 01/03/2027 iMove, the Sanofi... ...Ready to get started? About Sanofi We’re an R&D-driven, AI-powered biopharma company committed to improving people’s lives...Contract workWorldwideFlexible hours
$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Remote Role Responsibilities Evaluate AI-generated responses to strengthen reasoning... ...detail and accurate application of structured evaluation guidelines. Work independently and...Remote jobContract workSummer work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...pipelines , and A/B test write-ups . Evaluate AI-generated or human-created work against... ...detailed written justifications for evaluations and scores to maintain transparency and accountability...Remote jobContract workSummer work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: General Sales / GTM Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at officeFlexible hours$14 - $42 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Commitment: 20+ hours/week Role Responsibilities Evaluate AI-generated music across various genres. Rate against detailed quality...Remote jobContract workSummer workImmediate start- ...engineers, interior designers, sustainability specialists, and technologists are passionate about... ...across the Buildings business. The AI & Automation Solutions Specialist is a Hybrid... ...with business stakeholders to identify, evaluate, and prioritize high-value AI opportunities...Work at officeLocal areaRemote work
$70 - $100 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Commitment: 10–20+ hours/week Role Responsibilities Evaluate AI-generated office artifacts across Excel , Word , and PowerPoint...Remote jobContract workSummer workWork at officeImmediate start$39 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Commitment: 10+ hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation, and other standards...Remote jobContract workSummer workImmediate start
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Media Evaluation Specialist. Be the first to apply!
Related searches
- social media manager Toronto, ON
- remote social media marketing manager Toronto, ON
- work from home social media manager Toronto, ON
- social media marketing manager Toronto, ON
- social media manager no experience Toronto, ON
- gestionnaire des médias sociaux Toronto, ON
- social media community manager Toronto, ON
- social media manager
- remote social media marketing manager
- work from home social media manager