AI Model Evaluation Specialist
Temporary
Rise Data Labs
Key Responsibilities
• Design and implement evaluation metrics for AI models
• Conduct thorough testing of model performance across different scenarios
• Analyze model outputs for bias, fairness, and accuracy
• Collaborate with ML engineers to implement improvements
• Document findings and recommendations
Ideal Candidate:
• PhD or Masters in Computer Science, ML, or related field
• 5+ years experience in AI/ML model evaluation
• Strong background in statistical analysis
• Experience with evaluation frameworks and metrics
• Excellent communication skills
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the AI Model Evaluation Specialist in Remote vacancy
- ...Notez et comparez des réponses d'IA et justifiez vos évaluations, dans le cadre d'un travail rémunéré depuis chez vous. Ce poste convient... ...Vos notes et vos justifications écrites aident à entraîner des modèles d'IA et à tester leurs réponses. Le travail Notez et comparez...SuggestedContract workLocal area
$40 per hour
...Employers Job seekers Career centers Handshake AI Enterprise AI Research Log in Sign up FRENCH AI EVALUATION SPECIALIST (CANADA) Sign up now Up to $40/hr Depending on the... ...help evaluate and rank AI responses so future models can better tailor their answers to individual users...SuggestedPart timeRemote workFlexible hours- ...requirements Help Shape the Future of AI in Healthcare What if the... ...healthcare information, evaluates medical scenarios, and delivers... ...opportunities to contribute to the evaluation and improvement of advanced Artificial Intelligence (AI) models. We're seeking individuals currently...SuggestedContract workFor contractorsTraineeshipRemote work
- ...About the role AI assistants are becoming part of everyday life, helping people with... ...and select your preferred experience. Evaluate conversational quality, helpfulness,... ...simple testing guidelines and document your evaluations accurately. Who we're looking for Active...SuggestedHourly payTemporary workPart timeRemote work
- ...What You'll Do Review machine learning model outputs across a variety of research tasks Rate AI-generated responses for accuracy, relevance, and factual correctness... ...Provide clear written justifications for your evaluation scores Complete approximately five hours of...SuggestedContract workFor contractorsRemote work
- ...each one into a self-contained Task that an AI agent has to solve in a Linux terminal.... ...*Machine learning:** training, inference, evaluation, and kernels - **Security:** cryptography,... ...Repositories, schemas, and failing test suites - Model checkpoints, training code, and evaluation...Shift work
$20 - $60 per hour
...About micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to... ...and velocity. Role Title: Evaluation Specialist/Recent Grad Role Type: Contractor Location...Part timeFor contractorsRemote work$10 - $24 per hour
...Teach live online classes on grading one AI response against written criteria: accuracy, instruction following and safety, with a justification... .... What the work is for YPAI Learn trains people for AI evaluation work in live classes. Your classes take learners from a gut...Contract workRemote work$30 - $60 per hour
...verification Reasoning clarity Evaluation expertise About micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier... ...Title: Domain Contributor - Evaluation Specialist Role Type: Contractor (~15 hrs/week...Part timeFor contractorsRemote workFlexible hours- ...Key Responsibilities Train and improve AI models through effective data annotation and labeling. Review and validate AI-generated content for accuracy and relevance. Utilize Bash scripting to automate data processing tasks and improve efficiency. Provide domain...Hourly payContract workFlexible hours
$10 - $30 per hour
...constructive feedback digital delivery platforms ai-driven tools familiarity independent work adaptability... ...that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve LLM...Part timeFor contractorsRemote workShift work- ...About the company Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This is a task... ...EHR vignettes for a medical AI evaluation project. General and internal medicine...For contractorsRemote workFlexible hours
$22 per hour
...Job Application for 3D Modeling & Python Specialist - Freelance AI Trainer Project at Meridial Back to jobs 3D Modeling & Python Specialist - Freelance... ...for a specialized AI 3D asset collection project. As AI models increasingly generate and interact with 3D environments,...Hourly payPart timeFor contractorsFreelanceRemote work- ...AI Fact-Checking & Response Evaluation Reviewer (German) Mission of the Role Apply independent judgment to... ...Quality Rating, RLHF, LLM evaluation, AI model assessment, or human-in-the-loop... ...AI Quality, Fact-Checking, and QC Specialist roles ~ Help shape how AI systems...Long term contractContract workFreelanceRemote workFlexible hours
$22 - $28 per hour
...1 - 3 years Location: United States | Canada + 23 more AI Model Benchmarking Intern (Remote, Iowa-Based) Part-time, 15–20 hours... ...support SureHost's product use cases. Create representative evaluation datasets, test cases, scoring criteria, and repeatable...Contract workTemporary workPart timeInternshipRemote workFlexible hours- ...Project Are you fluent in French (FR) and interested in working on AI-related projects? We're currently looking for detail-oriented individuals to join our talent pool for an upcoming AI Response Evaluation and Fact-Checking project . This freelance, remote opportunity...Contract workFreelanceRemote workFlexible hours
- ...Curate and evaluate code in Rust to enhance AI training datasets. About Us Based in San Francisco, California... ..., and advancing large language models, collaborating closely with... ...communication skills for clear, structured evaluation rationales. Engagement Details...Part timeFor contractorsFlexible hours
- ...Job Application for Portuguese Audio Evaluations Specialist - Freelance AI Trainer Project at Meridial Back to jobs Portuguese Audio Evaluations... ...benchmark evaluation project assessing advanced agentic audio models. As AI models increasingly handle complex workflows in...Hourly payPart timeFor contractorsFreelanceRemote work
$30 - $60 per hour
...About micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to... .... Role Title: Research Evaluation Specialist (PhD or Masters / Researcher) Role Type:...For contractorsRemote workWorldwide$125 per hour
...Within 3 months ~80% Why this role $125/hr for Senior Machine Learning Engineer / Model Evaluations Expert work puts your Software Engineering expertise on the same footing as an AI lab's own research staff, without the overhead of running a consulting practice around...Hourly payContract workRemote work- ...steps. Our partner is looking for a Senior AI Content Specialist based in Canada. The Senior AI Content... ...training content and rigorous response evaluation. This role combines writing, editing,... ...to prompts, training data, and model behavior. The position requires a strong...Hourly payFull timeManual laborRemote work
usd16 per hour
...We are looking for AI Linguistic Evaluators to assess how effectively an AI application performs in Marathi for a short collaboration. Job Type... ...naturalness, and cultural appropriateness. - Providing structured evaluations and feedback following the project instructions and quality...Hourly payDaily paidFull timeFreelanceImmediate startRemote workFlexible hours$100 per hour
...chemical safety professionals to audit how AI models respond to sensitive questions in the... ...professionals; CWC or chemical-security specialists; hazmat, poison-control or chemical incident... ...subjects. Optimize AI Performance: Evaluate and rank AI responses to enhance the...Permanent employmentFull time- © 2026 Smart Ecosystems. All rights reserved.Contract work
- ...remote, hourly contractor role supporting AI data and language projects on a project-based... ...to support AI training datasets. LLM evaluation: reviewing AI-generated responses for... ...generation: writing high-quality prompts and model responses, or recording high-quality voice...Hourly payContract workFor contractorsRemote workFlexible hours
$100 per hour
...enterprise sales role selling open-weight AI models or the platforms built on them Direct... ...this role Few outlets pay Business specialists what the leading AI labs pay for direct... ...deliverables, you're given complex scenarios to evaluate: grading the AI's logic, correcting its...Hourly payContract workRemote work- ...interview Walk us through how you currently evaluate new artificial intelligence platforms... ...recent challenges you faced while vetting AI tools Who Should Apply Active role in... ...Comfortable discussing technical and business evaluation criteria in a 1:1 setting Contract &...Contract workFor contractors
usd16 - usd50 per hour
...We are looking for AI Data Specialists to support the improvement of AI-generated content in French... ...development and safety of today’s AI models? What You’ll Do The role involves performing... ...tasks, including: - Data collection, evaluation, and annotation. - Pairwise comparisons...Hourly payExtra incomeFull timePart timeFreelanceImmediate startRemote workWork from home10 hours per weekFlexible hours- ...the Role mpathic provides independent, multidisciplinary evaluations of healthcare AI technologies seeking deployment in real-world clinical settings... ...evidence supporting healthcare AI applications, including model performance, validation methodology, generalizability,...Contract workWork at officeShift work
- ...· Radiological Science · Health Physics · AI Safety · CBRN What this pays Rates are... ...behind. About the Role AI Safety Evaluation Project | Remote | Contract Locations: US,... ...Responsibilities •Tier 1: review AI prompts and model responses •Tier 1: classify content as...Hourly payContract workFor contractorsRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Model Evaluation Specialist. Be the first to apply!
