Legal Reasoning AI Response Evaluator
$55 - $100 par heureIntérimaire
Claru
Pay: $55–100/hour
You'll evaluate AI-generated responses to legal questions — covering case analysis, statutory interpretation, procedural questions, and litigation strategy. Your primary task is assessing whether a model's answer reflects sound legal reasoning, accurate doctrinal knowledge, and appropriate caveats. This isn't about copy-editing; you're judging whether the legal analysis would hold up in front of a practitioner, and flagging anything that could mislead a non-expert relying on the output.
Day-to-day work involves reading a legal prompt, reviewing the model's response, and scoring it across dimensions like analytical rigor, jurisdictional accuracy, and practical usefulness. For litigation-focused tasks, you'll also evaluate whether the model correctly identifies procedural posture, burden of proof, and viable arguments. Feedback is written in structured rubric fields, with an optional free-text section for nuanced critique. Tasks are asynchronous — you work batches on your own schedule through a browser-based annotation platform.
A minimum of 3 years of litigation or substantive legal practice is required. Familiarity with U.S. federal and state court procedure is expected; experience in additional jurisdictions is a bonus. Law professors and clerks are also strong candidates. You do not need to be currently practicing, but you must be able to credibly evaluate whether legal reasoning is sound, not just grammatically correct.
Skills & requirements
- Legal reasoning and case analysis
- Litigation and civil procedure
- Statutory and regulatory interpretation
- Analytical writing
- Jurisdictional knowledge (US federal/state)
Category: RLHF
L'offre d'emploi a été publiée Il y a un mois
Des emplois similaires qui pourraient vous intéresserBasé sur l'offre Legal Reasoning AI Response Evaluator à à domicile
$10 - $30 par heure
...digital delivery platforms ai-driven tools familiarity independent... ...agents. We provide frontier evaluations and reinforcement learning... ...-value, specialized domains (legal, medical, finance, coding, etc... ...Role Title: Food Delivery AI Response Evaluator Role Type:...JuridiqueTemps partielPour les contractantsTravail à distanceTravail posté$50 - $150 par heure
...hiring PhD-level experts to help train and evaluate advanced AI models. This is remote, flexible... ...directly shapes how cutting-edge models reason, answer, and improve. You'll review AI... ...depending on project needs. Key Responsibilities You may contribute your expertise by...SuggéréСontratTravail temporaireStageTravail à distanceHoraires flexibles- ...Create and evaluate board game tasks to enhance AI reasoning skills. About Turing Turing is one of the world’s fastest-growing AI companies, accelerating... ..., and rule-based systems. Evaluate AI-generated responses for correctness, consistency, and reasoning quality....SuggéréСontratTemps partielPour les contractantsFreelanceTravail à distance
- ...AI Fact-Checking & Response Evaluation Reviewer (French) Mission of the Role Apply independent judgment to identify AI blind spots, reasoning failures, factual inaccuracies, contradictory evidence, and quality gaps. Successful reviewers do not simply validate AI-generated...SuggéréContrat Longue DuréeFreelanceTravail à distanceHoraires flexibles
$80 - $150 par heure
...Pay: $80–$150/hr The work Read AI-written answers to legal questions and grade them for accuracy, completeness and the mistakes a client could act on. Where the answer is wrong or incomplete, you write the correct one. Who it suits A law degree and bar admission...JuridiqueСontratTravail à distance- ...editing skills in your native language. Strong attention to detail and ability to follow written guidelines. Comfortable working with AI-generated content. Reliable internet connection. Ability to work remotely using a client-provided laptop. French (France)...Travail à distance
- ...Evaluate AI chatbot responses for small business scenarios in Korean. About Turing: Turing is one of the world’s fastest-growing AI companies... ...labs to advance frontier model capabilities in thinking, reasoning, coding, agentic behavior, multimodality, multilinguality,...Temps partielFreelanceTravail à distanceHoraires flexibles
- ...Review and rate AI coding solutions for real software engineering... ...You’ll be at the forefront of evaluating how LLMs interact with real... ...support a frontier AI training and evaluation initiative focused on... ..., consistency, and quality of reasoning. Candidates will be onboarded...Temps partielPour les contractantsFreelanceTravail à distance
- ...Specialist in our Core Engineering & AI Development Community , you are the... ...Engineer. What You’ll Do (Key Responsibilities): Algorithmic Reasoning Validation: Audit and refine "Chain... ...Code Quality & Security Auditing: Evaluate AI-generated code for security vulnerabilities...Temps partiel
$150 par heure
...procedure against what the law actually says; identify where its reasoning would lose a motion or expose a client; draft the answer a... ...antitrust, white-collar or appellate practice. Precise, fast legal writing. How it works Apply with your CV and the hours you...JuridiqueСontratTravail à distance- ...In this remote, hourly contractor role, you will apply professional legal judgement to evaluate how advanced AI models reason through legal problems, and build cases that test the limits of that… In this remote, hourly contractor role, you will apply professional legal judgement...JuridiqueTaux horaireСontratPour les contractantsTravail à distance
- ...requirements Help Shape the Future of AI in Healthcare What if the... ...healthcare information, evaluates medical scenarios, and delivers more accurate, reliable responses? At Volga Partners, we're connecting... ...clinical training, and evidence-based reasoning to evaluate AI-generated...СontratPour les contractantsStage de formationTravail à distance
- ...your own work and turn each one into a self-contained Task that an AI agent has to solve in a Linux terminal. For every Task, you'll... ...programming languages - **Machine learning:** training, inference, evaluation, and kernels - **Security:** cryptography, reverse engineering,...Travail posté
$40 par heure
...seekers Career centers Handshake AI Enterprise AI Research Log in Sign up FRENCH AI EVALUATION SPECIALIST (CANADA) Sign up now... ...based in Canada to help make AI responses more personalized. This is not a... ...sharing your honest preferences and reasoning Provide clear written feedback...JuridiqueTemps partielTravail à distanceHoraires flexibles- ...Subject Matter Experts from legal practice, restructuring and insolvency... ...in Excel. You have used AI tools on your own work and... ...of a Legal Professional. Responsibilities Writing realistic Excel... ...this by building high-quality evaluation and training environments that...JuridiqueСontratPour les contractantsRecrutement immédiatTravail à distance
- ...About the Role: Do you have a background in Physics or Mechanical Engineering ? AI models are getting smarter, but they often struggle with "common sense" physics—like how gravity works, how objects move, or how much force is needed to lift something. At TELUS Digital...Temps partiel
usd16 par heure
...We are looking for AI Linguistic Evaluators to assess how effectively an AI application performs in... ...Evaluating the quality of AI-generated responses according to project guidelines. - Assessing... .... - Providing structured evaluations and feedback following the project instructions...Taux horaireTarif journalierTemps pleinFreelanceRecrutement immédiatTravail à distanceHoraires flexibles- ...specialists with project-based AI opportunities for leading tech... ..., focused on testing, evaluating, and improving AI systems. This... ...EHR vignettes for a medical AI evaluation project. General and internal... ...systems; Score the clinical reasoning quality of benchmark items: vignette...Pour les contractantsTravail à distanceHoraires flexibles
- ...Curate and evaluate code in Rust to enhance AI training datasets. About Us Based in San Francisco, California... ...in software engineering, logical reasoning, STEM, multilinguality,... ...communication skills for clear, structured evaluation rationales. Engagement Details...Temps partielPour les contractantsHoraires flexibles
- ...a fully remote, hourly contractor role supporting AI data and language projects on a project-based, flexible... ...content to support AI training datasets. LLM evaluation: reviewing AI-generated responses for accuracy, reasoning quality, coherence, and cultural/linguistic...Taux horaireСontratPour les contractantsTravail à distanceHoraires flexibles
- ...a global community of experts to evaluate, challenge and improve advanced AI systems . We are looking for... ...concepts and technical outputs Model reasoning and performance Data and... ...Hallucinations and factual accuracy LLM responses and instruction following...Temps partiel
$70 - $140 par heure
...RealTrainAI is recruiting experts for its Medical knowledge & clinical reasoning AI training project. Experts help train and evaluate AI models by reviewing, rating and writing responses in their field of expertise. The work is remote and paid hourly. Pay: $70–$140/hr...Taux horaireСontratTravail à distance- ...Job Application for Geospatial Reasoning Specialist (Senior) - Freelance AI Trainer Project at Meridial Back to... ...reasoning agent focused on Crisis Response—helping AI systems understand, assess... ...expertise will guide model development and evaluation, laying the foundation for future...Taux horaireСontratTemps partielPour les contractantsFreelanceTravail à distance
- © 2026 Smart Ecosystems. All rights reserved.Сontrat
- ...About the Role As a Legal & Compliance AI Specialist, you will directly improve the accuracy, safety... ...professional judgment will teach AI to reason about statutes, case law, and the... ...Reviewing AI-Generated Legal Summaries: Evaluating AI’s interpretation of statutes, case opinions...JuridiqueTaux horaireСontrat
- ...interview Walk us through how you currently evaluate new artificial intelligence platforms... ...recent challenges you faced while vetting AI tools Who Should Apply Active role in... ...Comfortable discussing technical and business evaluation criteria in a 1:1 setting Contract &...СontratPour les contractants
- ...Key Responsibilities • Design and implement evaluation metrics for AI models • Conduct thorough testing of model performance across different scenarios • Analyze model outputs for bias, fairness, and accuracy • Collaborate with ML engineers to implement improvements...Сontrat
- ...Join a leading AI lab's cutting-edge GenAI team to be at the... ...improving how frontier AI models reason about real financial work.... ...assistance is not provided. # Key Responsibilities Data QA and reviews: Vet... ...finance tasks and evaluation sets, and help build finance-...JuridiqueTemps pleinСontratTemps partielFreelanceLogé Sur PlaceTravail manuelRelocationProgramme de réinstallation
- ...Radiological Science · Health Physics · AI Safety · CBRN What this pays Rates... ...stand behind. About the Role AI Safety Evaluation Project | Remote | Contract Locations: US... ...reviewing AI-generated prompts and responses, evaluating safety risks, identifying dual...Taux horaireСontratPour les contractantsTravail à distance
- ...What You'll Do Apply your subject-matter expertise to advanced AI training and evaluation work Develop or evaluate challenging problems within your area of expertise Assess AI-generated reasoning for technical accuracy and identify subtle errors or failure modes...СontratRecrutement immédiatTravail à distanceHoraires flexibles
Voulez-vous recevoir plus d'offres d'emploi ?
S'abonner et recevoir des offres d'emploi similaires à Legal Reasoning AI Response Evaluator. Soyez parmi les premiers à postuler !
Recherches connexes