Senior Software Engineer - LLM Evaluation (US/Canada/WEU based)
Part-time
Turing
Evaluate and refine AI-generated code for efficiency and reliability.
About Us
Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in software engineering, logical reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.
As a Software Engineering evaluator, you will create cutting-edge datasets for training, benchmarking, and advancing large language models, collaborating closely with researchers. This includes curating code examples, providing precise solutions, and making corrections in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go; evaluating and refining AI-generated code for efficiency, scalability, and reliability; and working with cross-functional teams to enhance enterprise-level AI-driven coding solutions. What Does a Typical Day Look Like?
- Working on AI model training initiatives by curating code examples, building solutions, and correcting code in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go.
- Evaluate and refine AI-generated code to ensure that it is efficient, scalable, and reliable.
- Collaborate withcross-functionalteamsto enhance AI-driven coding solutions against industry performance benchmarks.
- Build agents that can verify the quality of the code and identify error patterns.
- Hypothesize on steps in the software engineering cycle (prototyping, architecture design, API design, production implementation, launch, experiments, monitoring, operational maintenance) and evaluate model capabilities on them
- Design verification mechanisms that can automatically verify a solution to a software engineering task.
Required Skills
- Several years of software engineering experience (3 years or more)
- Strong expertise in building full-stack applications and deploying scalable, production-grade software using modern languages and tools.
- Deep understanding of software architecture, design, development, debugging, and code quality/review assessment.
- Excellent oral and written communication skills for clear, structured evaluation rationales.
Engagement Details
- Commitment : flexible engagement, minimum 10 hrs/week, up to 40 hrs/week
- Type : Contractor (no medical/paid leave)
- Duration : 1 month (potential extensions based on performance and fit)
- Location: candidates must be based out of US, Canada or WEU countries (Austria, Belgium, France, Germany, ...)
Evaluation Process
- The application process takes 15-30 minutes.
- Completion of an AI video interview is required.
Skills: Python, JavaScript, Java, C++, Rust
Vacancy posted 12 days ago
Similar jobs that could be interesting for youBased on the Senior Software Engineer - LLM Evaluation (US/Canada/WEU based) in Remote vacancy
- ...Analyze GitHub issues and evaluate LLM performance on real code. Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering...SeniorContract workPart timeFor contractorsFreelanceManual laborRemote work
- ...About the Role We are looking for software engineers to validate real-world repositories used to evaluate large language models. You will triage issues, set up reproducible... ...setup. Ability to navigate complex code bases and run real-world projects locally. Fluent written...SuggestedHourly payFor contractorsRemote work
- ...Lead teams to create and evaluate realistic LLM conversational data. About Turing Turing is a... ...Role Overview We are looking for an Engineering Manager to lead teams creating high-quality... ...and AI assistants, including tool-based workflows involving calendars, email, maps...SuggestedContract workPart timeFreelanceRemote work
- ...with frontier AI labs to accelerate model development through high-quality training data, evaluations, and engineering talent. About the Role We are looking for senior software engineers with 7+ years of industry experience to support the training and evaluation of large...SeniorContract workPart timeFor contractors
$70 - $85 per hour
...Location: United States | Canada + 21 more This is a contracting... .... Location: Paris-based preferred; alternatively Europe... ...candidates We are building and evaluating state-of-the-art large language... ...and are looking for experienced software engineers to join our...SeniorLong term contractContract workPart timeRemote work- ...About the role Swedish Language Expert (US/Canada Based) is a remote evaluation track for reviewing swedish language evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback...Hourly payContract workFor contractorsRemote work10 hours per week
$232k - $379k per year
...considered a security risk. About Us Quanata is on a mission to... ...a better world through context-based insurance solutions. We are an... ...innovation company that engineers advanced risk prediction and prevention... ...impact. The Role As a Senior Application Security Engineer...SeniorLong term contractExtra incomeInternshipLocal areaRemote workWork from homeHome officeFlexible hoursusd15 per hour
...on behalf of RWS, you will test and evaluate how our clients’ search engine responds to everyday searches from everyday... ...and French proficiency. Location: Canada (Work from Home). Schedule: E. g. 8-... ...connection. - Reliable antivirus software to protect your computer as you surf...Weekly payFull timePart timeFreelanceSecond jobRemote workWork from homeHome officeFlexible hours$149.6k - $199.6k per year
...About us Coalition is the world's first Active Insurance provider designed to help prevent digital risk before it strikes... ...happening daily at Coalition. About the role We are hiring a Senior Software Engineer to lead the development of our internal AI platform and...SeniorFull timeFixed term contractWork at officeRemote workHome officeFlexible hours$90k - $120k per year
...Job Responsibility: Benefits: Bonus based on performance Competitive salary Home office stipend Paid time off Training... ...Overview We are seeking a resourceful and talented Software Engineer to join our team and contribute to the ongoing growth and development...Full timeFor contractorsRemote workHome office$75 - $90 per hour
...week Experience: 5 - 10 years Location: United States | Canada + 21 more Help a top AI lab evaluate and improve large language models through security-focused coding tasks. Bring your software-engineering judgment and hands-on security experience to work involving...Long term contractContract workRemote work$65k per year
...winning culture . Get to know more about us at foundever.com and connect with us on... ...role We are currently looking for a Senior Network Engineer to join the Foundever family! 80%... ...Day to Day Work on assigned tickets based on a ticket queue and priority Work on...SeniorLocal areaRemote workWork from homeShift work- ...You'll use your finance expertise to help evaluate and improve advanced AI systems on... ...and professional quality. Develop clear evaluation rubrics and scoring criteria, identifying... ...materials as needed. Refine your work based on testing, feedback, and project guidelines...Contract work
- ...and Lead Edge Capital. Learn more at grafana. com and follow us on LinkedIn and X . We’re scaling fast and staying true to what... ...interested in applicants from Canadian EST time zones only. Senior Software Engineer - Session Replay The Opportunity: The Session Replay squad is...SeniorFull timeRemote work
- ...production-ready builds of all the open source software engineers and AI agents rely on, Chainguard helps... ...to startups and SMBs. Join us as we help organizations secure their software... ...little manual intervention as possible. As a Senior Engineer, you'll be a hands-on contributor...SeniorFull time
- ..., and clean, trustworthy records come out. We’re looking for the engineer who will own that transformation. At DistantJob, we’re searching for a Senior Full-Stack Software Engineer for one of our U.S.-based clients. They’re building a platform that cleans, matches and unifies...SeniorLong term contractFull timeRemote work
- .... Requirements ~5+ years of backend engineering experience with a bachelor’s degree, or an... ...within Ontario or British Columbia, Canada. Annual bonuses for eligible employees... ...stipend. Family-forming benefits and generous parental leave base-salary top-up....SeniorRemote workFlexible hours
- ...S. , U. K. , Finland, India, Singapore, Canada, and Ireland. Come join us! About the Role AlphaSense is seeking an exceptional engineer to own our Excel plugin strategy, bringing... ...architectures - Experience with enterprise software deployment, updates, and version management...SeniorFull timeWork at officeLocal areaWorldwide
- ...training frontier models and evaluating AI agents. Experts contribute... ...as finance, healthcare, STEM engineering, and more. micro1 transforms... ...in identifying and resolving software bugs and system bottlenecks.... ...Structure Compensation is output-based; experts are paid per task...SeniorPart timeFor contractorsInternshipRemote workFlexible hours
$90 - $100 per hour
...: 40 hours / week Experience: 5 - 10 years Location: United States | Canada + 6 more Open to candidates based in the US, Canada, and LATAM. Role Overview We are hiring a senior independent contractor to design and ship production-grade AI agents that automate...SeniorFull timeContract workFor contractorsImmediate startRemote work- ...needed to welcome a billion customers. Spotify is looking for a Senior Software Engineer to join the Authorization team within Platform. Our team... ...a critical path for Spotify's creator ecosystem. You'll help us evolve these systems as they grow in scale and complexity, bringing...SeniorLong term contractFull timeWork from home
- ...is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Software Engineer (Senior, Staff) based in Canada. This is a high-impact senior engineering opportunity focused on building consumer AI products used by millions...SeniorFull timeInternshipWorldwide
$100 per hour
...for more details: We’re looking for highly experienced software engineer (SR+) to help evaluate the quality of interactions with modern coding agents... ...judgments, and explaining them clearly. WHO YOU ARE • Senior, Staff or Principal-level engineer (or equivalent...SeniorFull timeContract workRemote workWorldwideFlexible hours- ...We are hiring skilled software engineers with strong full-stack experience. The ideal candidate is comfortable architecting and working across multi-file codebases, structuring realistic development environments, and building reproducible setups supported by clear documentation...Contract work
- ...applications from candidates based in France, Italy, or Germany for... ...aims to create a thriving engineering organization that supports the... ...is also of great importance to us that working at MetaMask is an... ...into the MetaMask platform. As a Senior Software Engineer on this team...SeniorRemote workShift work
$153k - $213k per year
...builds the high-throughput decision engine at the heart of Affirm’s card... ...financial brand—come join us. It’s going to be an amazing ride... ...for - 5+ years of experience in software development or engineering,... ...transparent pay structure which is based on a variety of factors,...SeniorLong term contractFull timeRemote work- ...S. , U. K. , Finland, India, Singapore, Canada, and Ireland. Come join us! About the Role AlphaSense is seeking an exceptional engineer to own our Excel plugin strategy, bringing... ...Who You Are: - 5+ years building production software, with significant experience in desktop...SeniorFull timeWork at officeWorldwide
$154.5k - $225.45k per year
...performance. Join a talented, product focussed engineering team committed to solving some of the... ...About the Role We’re looking for a Senior Software Engineer who thrives in the earliest... ...excel here. Remote: United States or Canada What you’ll do: Build 0→1 products...SeniorLong term contractInternshipRemote workFlexible hours$153k - $213k per year
...closely with other teams and engineers across our mobile and web technologies... ...1.5+ years of experience as a software engineer. You have... ...comfortable navigating a large code base, debugging others’ code, and... ...$213,000 Location - Remote Canada This remote role is open only...SeniorWork at officeRemote workFlexible hours$142.8k - $184.8k per year
...firmware teams, and full-stack engineers. Requirements ~8+ years... ...professional industry experience in software development. ~ Experience... ...C/C++ or Golang, Linux-based embedded systems on complex SoCs... ...open to candidates residing in Canada; relocation assistance is not...SeniorFull timeRemote workRelocation packageFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Software Engineer - LLM Evaluation (US/Canada/WEU based). Be the first to apply!
Related searches
- software developer no experience Remote
- junior software developer Remote
- software developer entry level Remote
- software developer intern Remote
- software developer internship no experience Remote
- software developer Remote
- software development engineer Remote
- part time software developer Remote
- ingénieur logiciel Remote
- software design engineer Remote