AI Model Evaluation Specialist
Temporary
Rise Data Labs
Key Responsibilities
• Design and implement evaluation metrics for AI models
• Conduct thorough testing of model performance across different scenarios
• Analyze model outputs for bias, fairness, and accuracy
• Collaborate with ML engineers to implement improvements
• Document findings and recommendations
Ideal Candidate:
• PhD or Masters in Computer Science, ML, or related field
• 5+ years experience in AI/ML model evaluation
• Strong background in statistical analysis
• Experience with evaluation frameworks and metrics
• Excellent communication skills
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the AI Model Evaluation Specialist in Remote vacancy
- ...Notez et comparez des réponses d'IA et justifiez vos évaluations, dans le cadre d'un travail rémunéré depuis chez vous. Ce poste convient... ...Vos notes et vos justifications écrites aident à entraîner des modèles d'IA et à tester leurs réponses. Le travail Notez et comparez...SuggestedContract workLocal area
$40 per hour
...Employers Job seekers Career centers Handshake AI Enterprise AI Research Log in Sign up FRENCH AI EVALUATION SPECIALIST (CANADA) Sign up now Up to $40/hr Depending on the... ...help evaluate and rank AI responses so future models can better tailor their answers to individual users...SuggestedPart timeRemote workFlexible hours- ...About the role AI assistants are becoming part of everyday life, helping people with... ...and select your preferred experience. Evaluate conversational quality, helpfulness,... ...simple testing guidelines and document your evaluations accurately. Who we're looking for Active...SuggestedHourly payTemporary workPart timeRemote work
- ...a competitive advantage. Cresta’s unified AI platform combines conversational AI agents... ...owner. That’s you. As our first AI Agent Evaluation Engineer, you define what quality means for... ...set and regression suite that every prompt, model, knowledge, and flow change is validated...SuggestedFull time
- ...a competitive advantage. Cresta’s unified AI platform combines conversational AI agents... .... That’s you. As our first AI Agent Evaluation Engineer, you define what quality means for... ...and regression suite that every prompt, model, knowledge, and flow change is validated against...SuggestedFull timeWork at officeRemote workHome officeFlexible hours
- ...requirements Help Shape the Future of AI in Healthcare What if the... ...healthcare information, evaluates medical scenarios, and delivers... ...opportunities to contribute to the evaluation and improvement of advanced Artificial Intelligence (AI) models. We're seeking individuals currently...Contract workFor contractorsTraineeshipRemote work
- ...each one into a self-contained Task that an AI agent has to solve in a Linux terminal.... ...*Machine learning:** training, inference, evaluation, and kernels - **Security:** cryptography,... ...Repositories, schemas, and failing test suites - Model checkpoints, training code, and evaluation...Shift work
- ...What You'll Do Review machine learning model outputs across a variety of research tasks Rate AI-generated responses for accuracy, relevance, and factual correctness... ...Provide clear written justifications for your evaluation scores Complete approximately five hours of...Contract workFor contractorsRemote work
$20 - $60 per hour
...About micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to... ...and velocity. Role Title: Evaluation Specialist/Recent Grad Role Type: Contractor Location...Part timeFor contractorsRemote work- ...Teach live online classes on grading one AI response against written criteria: accuracy, instruction following and safety, with a justification... .... What the work is for YPAI Learn trains people for AI evaluation work in live classes. Your classes take learners from a gut...Contract workRemote work
$30 - $60 per hour
...verification Reasoning clarity Evaluation expertise About micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier... ...Title: Domain Contributor - Evaluation Specialist Role Type: Contractor (~15 hrs/week...Part timeFor contractorsRemote workFlexible hours- ...About the company Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This is a task... ...EHR vignettes for a medical AI evaluation project. General and internal medicine...Part timeFor contractorsRemote workFlexible hours
$10 - $30 per hour
...constructive feedback digital delivery platforms ai-driven tools familiarity independent work adaptability... ...that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve LLM...Part timeFor contractorsRemote workShift work- ...Project Are you fluent in French (FR) and interested in working on AI-related projects? We're currently looking for detail-oriented individuals to join our talent pool for an upcoming AI Response Evaluation and Fact-Checking project . This freelance, remote opportunity...Contract workFreelanceRemote workFlexible hours
- ...Curate and evaluate code in Rust to enhance AI training datasets. About Us Based in San Francisco, California... ..., and advancing large language models, collaborating closely with... ...communication skills for clear, structured evaluation rationales. Engagement Details...Part timeFor contractorsFlexible hours
- ...AI Fact-Checking & Response Evaluation Reviewer (French) Mission of the Role Apply independent judgment to... ...Quality Rating, RLHF, LLM evaluation, AI model assessment, or human-in-the-loop... ...AI Quality, Fact-Checking, and QC Specialist roles ~ Help shape how AI systems...Long term contractFreelanceRemote workFlexible hours
- ...Key Responsibilities Train and improve AI models through effective data annotation and labeling. Review and validate AI-generated content for accuracy and relevance. Utilize Bash scripting to automate data processing tasks and improve efficiency. Provide domain...Hourly payContract workFlexible hours
$30 - $60 per hour
...About micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to... .... Role Title: Research Evaluation Specialist (PhD or Masters / Researcher) Role Type:...For contractorsRemote workWorldwideusd16 per hour
...We are looking for AI Linguistic Evaluators to assess how effectively an AI application performs in Marathi for a short collaboration. Job Type... ...naturalness, and cultural appropriateness. - Providing structured evaluations and feedback following the project instructions and quality...Hourly payDaily paidFull timeFreelanceImmediate startRemote workFlexible hours$125 per hour
...Within 3 months ~80% Why this role $125/hr for Senior Machine Learning Engineer / Model Evaluations Expert work puts your Software Engineering expertise on the same footing as an AI lab's own research staff, without the overhead of running a consulting practice around...Hourly payContract workRemote work- ...Job Application for French Audio Evaluations Specialist - Freelance AI Trainer Project at Meridial Back to jobs French Audio Evaluations Specialist... ...evaluation project assessing advanced agentic audio models. As AI models increasingly handle complex workflows in this...Hourly payPart timeFor contractorsFreelanceRemote work
- ...interview Walk us through how you currently evaluate new artificial intelligence platforms... ...recent challenges you faced while vetting AI tools Who Should Apply Active role in... ...Comfortable discussing technical and business evaluation criteria in a 1:1 setting Contract &...Contract workFor contractors
- ...remote, hourly contractor role supporting AI data and language projects on a project-based... ...to support AI training datasets. LLM evaluation: reviewing AI-generated responses for... ...generation: writing high-quality prompts and model responses, or recording high-quality voice...Hourly payContract workFor contractorsRemote workFlexible hours
- ...· Radiological Science · Health Physics · AI Safety · CBRN What this pays Rates are... ...behind. About the Role AI Safety Evaluation Project | Remote | Contract Locations: US,... ...Responsibilities •Tier 1: review AI prompts and model responses •Tier 1: classify content as...Hourly payContract workFor contractorsRemote work
- ...the Role mpathic provides independent, multidisciplinary evaluations of healthcare AI technologies seeking deployment in real-world clinical settings... ...evidence supporting healthcare AI applications, including model performance, validation methodology, generalizability,...Contract workWork at officeShift work
$22 - $28 per hour
...1 - 3 years Location: United States | Canada + 23 more AI Model Benchmarking Intern (Remote, Iowa-Based) Part-time, 15–20 hours... ...support SureHost's product use cases. Create representative evaluation datasets, test cases, scoring criteria, and repeatable...Contract workTemporary workPart timeInternshipRemote workFlexible hours- ...steps. Our partner is looking for a Senior AI Content Specialist based in Canada . As a Senior AI... ...practical expertise in large language models (LLMs) to create engaging, natural, and... ...reduce repetitive manual tasks. - Establish evaluation frameworks, quality standards, and...Full timeManual laborRemote work
$22 per hour
...Job Application for 3D Modeling & Python Specialist - Freelance AI Trainer Project at Meridial Back to jobs 3D Modeling & Python Specialist - Freelance... ...for a specialized AI 3D asset collection project. As AI models increasingly generate and interact with 3D environments,...Hourly payPart timeFor contractorsFreelanceRemote work$55 - $100 per hour
...Pay: $55–100/hour You'll evaluate AI-generated responses to legal questions — covering case analysis, statutory interpretation, procedural... ...litigation strategy. Your primary task is assessing whether a model's answer reflects sound legal reasoning, accurate doctrinal knowledge...Contract work- ...Contra Labs sits at the intersection of human creativity and AI. We partner with leading AI labs and product companies to improve... ..., specifically across desktop and mobile applications. Skills evaluation: You'll rate AI-generated outputs across multiple dimensions, provide...Full timeContract work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Model Evaluation Specialist. Be the first to apply!
