Legal Reasoning AI Response Evaluator
$55 - $100 per hourTemporary
Claru
Pay: $55–100/hour
You'll evaluate AI-generated responses to legal questions — covering case analysis, statutory interpretation, procedural questions, and litigation strategy. Your primary task is assessing whether a model's answer reflects sound legal reasoning, accurate doctrinal knowledge, and appropriate caveats. This isn't about copy-editing; you're judging whether the legal analysis would hold up in front of a practitioner, and flagging anything that could mislead a non-expert relying on the output.
Day-to-day work involves reading a legal prompt, reviewing the model's response, and scoring it across dimensions like analytical rigor, jurisdictional accuracy, and practical usefulness. For litigation-focused tasks, you'll also evaluate whether the model correctly identifies procedural posture, burden of proof, and viable arguments. Feedback is written in structured rubric fields, with an optional free-text section for nuanced critique. Tasks are asynchronous — you work batches on your own schedule through a browser-based annotation platform.
A minimum of 3 years of litigation or substantive legal practice is required. Familiarity with U.S. federal and state court procedure is expected; experience in additional jurisdictions is a bonus. Law professors and clerks are also strong candidates. You do not need to be currently practicing, but you must be able to credibly evaluate whether legal reasoning is sound, not just grammatically correct.
Skills & requirements
- Legal reasoning and case analysis
- Litigation and civil procedure
- Statutory and regulatory interpretation
- Analytical writing
- Jurisdictional knowledge (US federal/state)
Category: RLHF
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Legal Reasoning AI Response Evaluator in Remote vacancy
$10 - $30 per hour
...digital delivery platforms ai-driven tools familiarity independent... ...agents. We provide frontier evaluations and reinforcement learning... ...-value, specialized domains (legal, medical, finance, coding, etc... ...Role Title: Food Delivery AI Response Evaluator Role Type:...LegalPart timeFor contractorsRemote workShift work$50 - $150 per hour
...hiring PhD-level experts to help train and evaluate advanced AI models. This is remote, flexible... ...directly shapes how cutting-edge models reason, answer, and improve. You'll review AI... ...depending on project needs. Key Responsibilities You may contribute your expertise by...SuggestedContract workTemporary workInternshipRemote workFlexible hours- ...Create and evaluate board game tasks to enhance AI reasoning skills. About Turing Turing is one of the world’s fastest-growing AI companies, accelerating... ..., and rule-based systems. Evaluate AI-generated responses for correctness, consistency, and reasoning quality....SuggestedContract workPart timeFor contractorsFreelanceRemote work
- ...AI Fact-Checking & Response Evaluation Reviewer (French) Mission of the Role Apply independent judgment to identify AI blind spots, reasoning failures, factual inaccuracies, contradictory evidence, and quality gaps. Successful reviewers do not simply validate AI-generated...SuggestedLong term contractFreelanceRemote workFlexible hours
$80 - $150 per hour
...Pay: $80–$150/hr The work Read AI-written answers to legal questions and grade them for accuracy, completeness and the mistakes a client could act on. Where the answer is wrong or incomplete, you write the correct one. Who it suits A law degree and bar admission...LegalContract workRemote work- ...editing skills in your native language. Strong attention to detail and ability to follow written guidelines. Comfortable working with AI-generated content. Reliable internet connection. Ability to work remotely using a client-provided laptop. French (France)...Remote work
- ...Review and rate AI coding solutions for real software engineering... ...You’ll be at the forefront of evaluating how LLMs interact with real... ...support a frontier AI training and evaluation initiative focused on... ..., consistency, and quality of reasoning. Candidates will be onboarded...Part timeFor contractorsFreelanceRemote work
- ...Evaluate AI chatbot responses for small business scenarios in Korean. About Turing: Turing is one of the world’s fastest-growing AI companies... ...labs to advance frontier model capabilities in thinking, reasoning, coding, agentic behavior, multimodality, multilinguality,...Part timeFreelanceRemote workFlexible hours
- ...In this remote, hourly contractor role, you will apply professional legal judgement to evaluate how advanced AI models reason through legal problems, and build cases that test the limits of that… In this remote, hourly contractor role, you will apply professional legal judgement...LegalHourly payContract workFor contractorsRemote work
- ...Specialist in our Core Engineering & AI Development Community , you are the... ...Engineer. What You’ll Do (Key Responsibilities): Algorithmic Reasoning Validation: Audit and refine "Chain... ...Code Quality & Security Auditing: Evaluate AI-generated code for security vulnerabilities...Part time
- ...requirements Help Shape the Future of AI in Healthcare What if the... ...healthcare information, evaluates medical scenarios, and delivers more accurate, reliable responses? At Volga Partners, we're connecting... ...clinical training, and evidence-based reasoning to evaluate AI-generated...Contract workFor contractorsTraineeshipRemote work
- ...your own work and turn each one into a self-contained Task that an AI agent has to solve in a Linux terminal. For every Task, you'll... ...programming languages - **Machine learning:** training, inference, evaluation, and kernels - **Security:** cryptography, reverse engineering,...Shift work
$150 per hour
...procedure against what the law actually says; identify where its reasoning would lose a motion or expose a client; draft the answer a... ...antitrust, white-collar or appellate practice. Precise, fast legal writing. How it works Apply with your CV and the hours you...LegalContract workRemote work$40 per hour
...seekers Career centers Handshake AI Enterprise AI Research Log in Sign up FRENCH AI EVALUATION SPECIALIST (CANADA) Sign up now... ...based in Canada to help make AI responses more personalized. This is not a... ...sharing your honest preferences and reasoning Provide clear written feedback...LegalPart timeRemote workFlexible hours- ...Subject Matter Experts from legal practice, restructuring and insolvency... ...in Excel. You have used AI tools on your own work and... ...of a Legal Professional. Responsibilities Writing realistic Excel... ...this by building high-quality evaluation and training environments that...LegalContract workFor contractorsImmediate startRemote work
usd16 per hour
...We are looking for AI Linguistic Evaluators to assess how effectively an AI application performs in... ...Evaluating the quality of AI-generated responses according to project guidelines. - Assessing... .... - Providing structured evaluations and feedback following the project instructions...Hourly payDaily paidFull timeFreelanceImmediate startRemote workFlexible hours- ...About the Role: Do you have a background in Physics or Mechanical Engineering ? AI models are getting smarter, but they often struggle with "common sense" physics—like how gravity works, how objects move, or how much force is needed to lift something. At TELUS Digital...Part time
- ...Curate and evaluate code in Rust to enhance AI training datasets. About Us Based in San Francisco, California... ...in software engineering, logical reasoning, STEM, multilinguality,... ...communication skills for clear, structured evaluation rationales. Engagement Details...Part timeFor contractorsFlexible hours
- ...specialists with project-based AI opportunities for leading tech... ..., focused on testing, evaluating, and improving AI systems. This... ...EHR vignettes for a medical AI evaluation project. General and internal... ...systems; Score the clinical reasoning quality of benchmark items: vignette...For contractorsRemote workFlexible hours
- ...a fully remote, hourly contractor role supporting AI data and language projects on a project-based, flexible... ...content to support AI training datasets. LLM evaluation: reviewing AI-generated responses for accuracy, reasoning quality, coherence, and cultural/linguistic...Hourly payContract workFor contractorsRemote workFlexible hours
- ...a global community of experts to evaluate, challenge and improve advanced AI systems . We are looking for... ...concepts and technical outputs Model reasoning and performance Data and... ...Hallucinations and factual accuracy LLM responses and instruction following...Part time
- ...Job Application for Geospatial Reasoning Specialist (Senior) - Freelance AI Trainer Project at Meridial Back to... ...reasoning agent focused on Crisis Response—helping AI systems understand, assess... ...expertise will guide model development and evaluation, laying the foundation for future...Hourly payContract workPart timeFor contractorsFreelanceRemote work
$70 - $140 per hour
...RealTrainAI is recruiting experts for its Medical knowledge & clinical reasoning AI training project. Experts help train and evaluate AI models by reviewing, rating and writing responses in their field of expertise. The work is remote and paid hourly. Pay: $70–$140/hr...Hourly payContract workRemote work- © 2026 Smart Ecosystems. All rights reserved.Contract work
- ...About the Role As a Legal & Compliance AI Specialist, you will directly improve the accuracy, safety... ...professional judgment will teach AI to reason about statutes, case law, and the... ...Reviewing AI-Generated Legal Summaries: Evaluating AI’s interpretation of statutes, case opinions...LegalHourly payContract work
- ...interview Walk us through how you currently evaluate new artificial intelligence platforms... ...recent challenges you faced while vetting AI tools Who Should Apply Active role in... ...Comfortable discussing technical and business evaluation criteria in a 1:1 setting Contract &...Contract workFor contractors
- ...Key Responsibilities • Design and implement evaluation metrics for AI models • Conduct thorough testing of model performance across different scenarios • Analyze model outputs for bias, fairness, and accuracy • Collaborate with ML engineers to implement improvements...Contract work
- ...Join a leading AI lab's cutting-edge GenAI team to be at the... ...improving how frontier AI models reason about real financial work.... ...assistance is not provided. # Key Responsibilities Data QA and reviews: Vet... ...finance tasks and evaluation sets, and help build finance-...LegalFull timeContract workPart timeFreelanceLive InManual laborRelocationRelocation package
- ...What You'll Do Apply your subject-matter expertise to advanced AI training and evaluation work Develop or evaluate challenging problems within your area of expertise Assess AI-generated reasoning for technical accuracy and identify subtle errors or failure modes...Contract workImmediate startRemote workFlexible hours
- ...Radiological Science · Health Physics · AI Safety · CBRN What this pays Rates... ...stand behind. About the Role AI Safety Evaluation Project | Remote | Contract Locations: US... ...reviewing AI-generated prompts and responses, evaluating safety risks, identifying dual...Hourly payContract workFor contractorsRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Legal Reasoning AI Response Evaluator. Be the first to apply!
