LLM Annotator - Master's Degree
Part-time
Turing
Evaluate AI responses for accuracy and improve model performance.
About Turing:
Based in San Francisco, California, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.
Role Overview
We are seeking highly motivated Team Management to support the evaluation and improvement of cutting-edge Large Language Models (LLMs). In this role, you will analyze structured data, create challenging prompts, evaluate AI-generated responses for factual accuracy and reasoning quality, and provide evidence-backed feedback to improve model performance.
We are looking for curious, detail-oriented professionals who enjoy solving complex problems and evaluating AI systems. The ideal candidate can analyze data, think critically, validate responses against evidence, and clearly articulate why a model's output is correct or incorrect. Experience working with AI models and creating challenging evaluation prompts is a strong advantage.
Key Responsibilities
- Create challenging prompts that evaluate an LLM's ability to retrieve, analyze, and reason over structured data.
- Assess AI-generated responses for factual accuracy, logical reasoning, and completeness.
- Identify model failures, inconsistencies, hallucinations, and reasoning gaps.
- Validate model outputs using provided datasets and supporting evidence.
- Document findings with clear, evidence-based explanations.
- Consistently follow annotation guidelines and maintain high-quality standards.
Minimum Qualifications
- Master's degree or higher in any discipline.
- Minimum 3 years of professional, research, or teaching experience .
- Strong analytical and critical thinking skills.
- Excellent written English communication skills.
- Exceptional attention to detail and ability to validate information against source data. Preferred Qualifications
- Experience working with Large Language Models (LLMs) or Generative AI.
- Familiarity with prompt engineering, AI evaluation, data annotation, or model testing.
- Experience working with structured datasets (CSV, Excel, databases, etc.).
- Ability to identify edge cases and design prompts that expose model limitations. Benefits
- Opportunity to work on cutting-edge AI projects.
- Competitive compensation.
- Flexible working hours and remote work environment.
Offer Details
- Commitments Required : 40, 30 or 20 hours per week with at least 4 hours PST overlap
- Employment type: Contractor assignment (no medical/paid leave)
- Duration of contract: 4 weeks Evaluation Process:
- Shortlisting based on qualifications and assessment scores.
Skills: Team Management
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the LLM Annotator - Master's Degree in Remote vacancy
- ...Overview We are seeking experienced professionals with strong LLM expertise, analytical skills, and language proficiency to... ...Candidates must meet all of the following requirements: Master’s degree or higher in any field. Minimum 3 years of professional experience...PostgraduatePart timeRemote workFlexible hours
$70 - $85 per hour
...What You’ll Do Create high-quality coding prompts and reference answers (benchmark-style, e.g. SWE-Bench-like problems). Evaluate LLM outputs for code generation, refactoring, debugging, and implementation tasks. Identify and document model failures, edge cases, and...SuggestedLong term contractContract workPart timeRemote work- ...of Freelancing With Turing: Work in a fully remote environment. Opportunity to work on cutting-edge AI projects with leading LLM companies. Potential for contract extension based on performance and project needs. Offer Details: Commitments Required:...SuggestedContract workPart timeFor contractorsFreelanceRemote work
- ...frontier evaluations and reinforcement learning environments to improve LLM capabilities, as well as contextual evaluations to monitor and... ...ensures both quality and velocity. Role Title: Video Data Annotator Role Type: Contractor Location: Remote micro1 is engaging...SuggestedPart timeFor contractorsRemote work
- ...of software development Job Requirements: Bachelor’s/Master’s degree in Engineering, Computer Science (or equivalent experience)... ...to have candidates who have used Github in the past. Train LLM models with high-quality, stable, and scalable back-end components...PostgraduateContract workPart timeFlexible hours
$18 - $28 per hour
Pay: $18–28/hour We hire game annotators to label gameplay footage and input data for AI training. You will watch recorded gameplay sessions, identify actions and events, and tag them using our annotation platform. Some projects involve labeling object interactions, player...Contract work- ...to enhance AI training processes. Required Skills: Train LLM models with high-quality, stable, and scalable back-end components... ...annotated/classifiable code in the responses Bachelor’s/Master’s degree in Engineering, Computer Science (or equivalent experience)...PostgraduateContract workPart timeFor contractorsFreelanceRemote work
$35 - $55 per hour
Pay: $35–55/hour You'll spend your time evaluating model-generated solutions to math problems — ranging from algebra and calculus to combinatorics and proof-based reasoning. Your core task is to compare two or more responses side by side, rank them by correctness and clarity...Contract workFlexible hours- ...solutions with well-articulated reasoning. Collaborate with LLM researchers to align problems with evaluation goals,... ...Prefered Qualification: Candidates currently pursuing a Master’s/Ph.D./Postdoctoral degree in STEM, Applied Physics, or a related field are eligible...Contract workPart timeFor contractorsFreelanceRemote work
- ...Preferred Qualifications: Candidates pursuing a PhD/Postdoctoral degree in Chemistry, Chemical Engineering, Biochemistry, or a related... ...Opportunity to work on cutting-edge AI projects with leading LLM companies. Potential for contract extension based on performance...PostgraduateContract workPart timeFor contractorsFreelanceRemote work
$55 - $100 per hour
Pay: $55–100/hour You'll draw precise segmentation masks over anatomical structures in CT scans, MRI slices, X-rays, and histopathology slides using DICOM-compatible tools such as OHIF Viewer or ITK-SNAP. Work includes labeling organs, lesions, tumors, and tissue boundaries...Contract work$55 - $100 per hour
Pay: $55–100/hour You'll systematically probe large language models to find inputs that cause them to bypass safety guidelines and produce harmful, disallowed, or policy-violating outputs. Your work involves crafting adversarial prompts across defined harm categories — ...Contract work- ...resolution paths and rigid all-or-nothing grading rubrics featuring canonical answers and explicitly accepted variants for automated LLM evaluation. Quality Assurance & Verification: Ensure all benchmark deliverables pass automated validation checks, eliminating answer...Part time
- ...Lead teams to create and evaluate realistic LLM conversational data. About Turing Turing is a leading AI company accelerating the advancement and deployment of frontier AI systems. We work with the world’s top AI labs and enterprises to improve capabilities in reasoning...Contract workPart timeFreelanceRemote work
- This is a remote, project-based role for experienced practitioners across any professional domain who can apply their real-world expertise to data annotation tasks. You will review, label, and evaluate data samples — including text, documents, model outputs, and domain-specific...Contract workRemote work
- .... Preferred Qualifications: Candidates pursuing a Master’s/Ph.D./Postdoctoral degree in Mathematics, Applied Mathematics, Statistics, or a related... ...to work on cutting-edge AI projects with leading LLM companies. Potential for contract extension based on performance...Contract workPart timeFor contractorsFreelanceRemote work
- ...solve mission-critical priorities for companies. Role Overview: This position is within a project with one of the foundational LLM companies. The goal is to assist these foundational LLM companies in enhancing their Large Language Models. One way we help these companies...Contract workPart timeFor contractorsFreelanceRemote work
- Evaluate and refine AI-generated code for efficiency and reliability. About Turing Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing...Full timePart timeFor contractorsFlexible hours
$70 - $110 per hour
...Scrum masters to build the agile delivery scenarios frontier AI labs use to test model judgment. $70 to $110/hr, remote, async, paid weekly. AfterQuery builds the expert evaluation data that frontier AI labs use to measure and improve their models. We're hiring scrum masters...Weekly payOngoing contractContract workFor contractorsRemote work10 hours per week- ...Reliable, detail-oriented, and able to work independently and meet deadlines in a remote setting. Preferred Qualifications Prior LLM research or evaluation projects. Experience building or testing developer tools or automation agents. Open-source contribution or...Hourly payFor contractorsRemote work
- ...and rubrics for code-quality assessment. Collaborate with researchers and engineering teams to design coding benchmarks and improve LLM evaluation methodologies. Required Qualifications: ~7+ years of professional software engineering experience. ~ Strong proficiency...Contract workPart timeFor contractors
- ...Analyze GitHub issues and evaluate LLM performance on real code. Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering...Contract workPart timeFor contractorsFreelanceManual laborRemote work
- This is a remote, project-based role for currently enrolled graduate students across academic disciplines, to contribute to an AI evaluation research project. You will apply the kind of expertise that comes from years of coursework, lab work, and field research to assess, write...Contract workRemote work10 hours per week
- ...Please state your availability clearly in your resume (Start date, End date). Responsibilities Research and develop state-of-the-art LLM and/or video or multimodality models to accelerate open-source progress in the domains. Implement the application of large models in...Permanent employmentFull timeInternshipImmediate startRemote workWorldwide
- ...or JAX profiler); debugging distributed or accelerator-bound workloads; serving large language models at scale (vLLM, SGLang, TensorRT-LLM, Ray Serve, KV cache, paged attention, continuous batching). Working production experience with JAX and/or PyTorch. Framework-level...Full timeContract workPart timeWeekday work
- ...produce good generators quickly and at small parameter counts. LLM Post-Training & Behavioral Robustness Hands-on experience with... ...experience with PyTorch, JAX, TensorFlow, or similar ML frameworks. Degree from a top-100 university, experience at a FAANG or comparable AI...Part timeFlexible hours
- ...This is a remote, project-based role for a PhD student or strong Masters-level researcher with hands-on CRISPR and gene editing experience to contribute to a short-term AI safety evaluation project. Ideal candidates have familiarity with dual-use research of concern (DURC)...Contract workTemporary workRemote work10 hours per week
- ...Mit Professor:innen, Lehrstühlen und Forschungsstellen sauber kommunizieren Was du mitbringst Hochschulabschluss (Bachelor oder Master) in Statistik, Psychologie (empirisch/quantitativ), Sozialwissenschaft (quant), Wirtschaftsmathematik, BWL mit Empirie-Schwerpunkt,...Permanent employmentFull timeRemote work
- ...used to calculate deposit insurance coverage and process payouts during bank failures Responsibilities Scrum master, BA, QA activities Skills Must have 10+ yrs testing/QA + 10+ yrs BA experience;...Remote job
- ...responses. Adhere to Project Guidelines: Master complex project instructions and apply... ...Proven experience as an AI Trainer, Data Annotator, or in a similar role involving interaction... ...and critical thinking skills with a high degree of attention to detail. A broad...Hourly payRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to LLM Annotator - Master's Degree. Be the first to apply!
