AI Code Reviewer & Systems Evaluation Engineer 1
Lifted, an Upwork Company
- Contract
Company Description
An enterprise client is currently seeking experienced software engineers to contribute to improving advanced AI systems through human feedback. This work supports leading AI organizations in training large language models to better understand software development practices, debugging, and code quality.
This is part of a cutting‑edge initiative focused on enhancing how AI systems write, review, and optimize code in real‑world scenarios. You’ll play a key role in shaping how AI models evaluate performance, detect issues, and generate reliable outputs.
Job Description
This opportunity is ideal for engineers who enjoy analyzing systems, improving code quality, and working on complex technical challenges. You will contribute to AI training projects by evaluating outputs, refining logic, and identifying potential vulnerabilities.
What You’ll Do:
- Develop objective, verifiable evaluation criteria (rubrics) for system performance
- Review system logs and execution paths to improve reliability and code quality
- Refactor code and optimize system behavior toward ideal outcomes
- Test systems for vulnerabilities, including data exposure and edge‑case failures
- Provide detailed, high‑quality feedback on system performance and outputs
Qualifications
Requirements:
- 2+ years of experience in backend engineering, AI automation, or systems integration
- Experience working with SQL databases
- Proven ability to build and maintain production‑grade systems
- Experience working in live (non‑mocked) environments with multi‑step interactions
- Strong analytical skills and attention to detail
Nice to Haves:
- Experience with multi‑stage system workflows and coordination tasks
- Familiarity with integrating tools such as APIs, databases, or external platforms
- Understanding of system vulnerabilities (e.g., privacy leaks, prompt injection, access escalation)
- Experience working with AI systems or agent‑based workflows
- Comfort working with persistent state tracking or similar frameworks
Additional Information
- Fully remote and flexible work schedule
- Project-based engagement with no guaranteed hours
- Work on tasks based on availability and project assignment
- Payment is based on completed tasks only
- Must accept project invitations before beginning work
- No guaranteed workload; volume may vary weekly
Vacancy posted 8 days ago
Similar jobs that could be interesting for youBased on the AI Code Reviewer & Systems Evaluation Engineer 1 in Toronto, ON vacancy
- ...innovative company is seeking a skilled Software Engineer to help train AI chatbots. This exciting remote role allows... ...You'll be responsible for creating diverse coding challenges, writing high-quality code snippets, and evaluating AI-generated code for accuracy and...SuggestedRemote work
- Elevate your AI career as an Applied AI Engineer with B3 Systems. You'll develop cutting-edge agentic workflows, collaborating with product teams to craft high-impact AI solutions. B3 Systems is seeking an Applied AI Engineer with over 5 years of experience in Machine Learning...Suggested
- Senior Applied AI Engineer – GenAI Systems (Toronto) Manulifes Group Functions AI team is scaling AI and advanced analytics capabilities across Finance... ...approaches including workflow, data flow, modeling, evaluation, and operational controls. Apply strong design thinking:...SuggestedLong term contractTemporary work
- Elevate your career with Thomson Reuters as an AI Systems Lead Engineer! Take the reins in developing advanced AI solutions that shape the future... ...into production environments • Oversee proof-of-concept evaluations and feasibility studies • Establish best practices for AI system...SuggestedFlexible hours
- ...career with the company Canada ULC as an Embedded Software Engineer focused on low-power AI solutions. Ideal for professionals skilled in machine... ...requires deep knowledge of C/C++ programming and embedded system design. Collaborate across teams to ensure optimized execution...Suggested
- ...as a Member of Technical Staff 1. Drive advancements in search solutions while embracing AI tools for development. This role... ..., work across product and engineering to enhance relevance and performance... ...and developer efficiency across systems • Drive continuous improvement in...
$85 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Backend Engineer (Coding Agent Experience) Type: Contract... ...frontier AI coding agents to complete and evaluate complex engineering tasks. Review model...Remote jobContract workSummer work$85 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Security Engineer (Coding Agent Experience) Type: Contract... ...frontier AI coding agents to complete and evaluate complex security engineering tasks ....Remote jobContract workSummer work$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type: Contract... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review...Remote jobContract workSummer work$85 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... .... Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience) Type: Contract... ...frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks...Remote jobContract workSummer work$142.4k - $190.1k per year
...We are seeking a MTS1 Software Engineer to join our eBay Search, Recs and... ...technologies like Generative AI. You’ll be part of a collaborative... ..., extensible, maintainable code. Be responsible for subsystems... ...technical guidance, code review for immediate team and other teams...Immediate startFlexible hoursShift work$85 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Frontend Engineer (Coding Agent Experience) Type: Contract... ...frontier AI coding agents to complete and evaluate complex engineering tasks. Review model...Remote jobContract workSummer work- Utilize your coding expertise as a Coding Analyst II at UHN, focusing on the Discharge Abstract... ...and coding using ICD-10-CA and CCI systems. This position highlights your analytical... ...patient conditions relevant for diagnoses • Review and code clinical interventions from documentation...
- The bet This is the most senior engineering hire Bold AI will make, this week. The person in this role... ...who has built enterprise agentic systems before, knows how to architect for the... ...handling, and human-in-the-loop checkpoints. Evaluate and integrate orchestration frameworks...InternshipRemote work
- ...About the Role As anElectrical & Embedded Systems Engineer, youwill play a critical role in... ...Write clean, well-documented, and testable code suitable for eventual flight qualification... ..., characterization, and flight readiness review. Define control and monitoring requirements...
$85 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and Jack Dorsey . Position: iOS Engineer (Coding Agent Experience) Type: Contract... ...frontier AI coding agents to complete and evaluate complex engineering tasks. Review model...Remote jobContract workSummer work- VP, AI Software Engineering Mastercard is looking for a talented VP, AI Software Engineering to join the Mastercard Foundry Research and Development... ...‑native SDLC, including design, architecture, development, code reviews, testing, deployment, and release. Coaching teams on...
$123k - $180.4k per year
...Autodesk is evolving its productivity engineering ecosystem by embedding AI‑powered engineering into its... ...orchestrate, and operationalize agent‑based systems that improve engineering workflows... ...optimization using intelligent agents Evaluate and integrate emerging AI frameworks...Local area$130.19k - $179.02k per year
Description Piping Engineering & Analysis Synopsis The Piping Engineering & Analysis discipline performs evaluations for piping systems in accordance with ASME Boiler and Pressure Vessel Code, ASME B31.1, ASME B31.7, and other codes as applicable to the specific evaluation being...Hourly payWork at officeRemote work3 days per week- ...developed in partnership with Nebius AI. Our team is fully remote and globally... ...re launching a program for experienced engineers transitioning into system architecture roles. The curriculum is... ...for students: hosts the live sessions, reviews the work, runs the architecture review...Work at officeRemote workFlexible hours
- ...innovative technology company is seeking a Power Architect to lead architectural strategy and design decisions for high-performance AI compute systems. This hybrid role involves optimizing power usage in product design and requires extensive experience with power estimation...
- Elevate your career as a Contracts Review Specialist with Stantec in Charlottetown, Canada. This role involves drafting and negotiating... ...proofreading skills Utilize your expertise in contract management to contribute to Stantec's mission of sustainable engineering. #J-18808-LjbffrContract workWork at office
- Become a Senior Manager focused on Streaming AI Data Systems at Autodesk, where you will spearhead the personalization initiative by leveraging advanced machine learning techniques. Engage a diverse engineering team to create value-driven AI solutions. Your leadership in...
$81.28k - $124.18k per year
Description This is a senior engineering position that offers a variety... ...clients. You will prepare and review design calculations, develop specifications... ...Appendix A to Part 810 of the Code of Federal Regulations. Please... ...'s needs at the time. AI & Automation at Sargent & Lundy...Hourly payWork at officeRemote work3 days per week- A leading AI solutions provider in Toronto is looking for a Senior AI Engineer to leverage expertise in agentic AI systems and cloud deployment. The ideal candidate has significant experience in building multi-agent architectures and RAG pipelines on AWS. Key responsibilities...
- Get notified about new Information Technology Business Analyst jobs in United States . 3,000+ Information Technology Business Analyst Jobs in United States IT Business Analyst - Tools Rationalization IT Healthcare Consultant – Business Analyst (Advanced) Technical Business...Full timeRemote work
- PrecisionERP/IT is seeking a Generative AI Engineer to design, build, and scale AI systems and intelligent agents. This is a contract position for one year in downtown Toronto beginning early July. Candidate must be able to work onsite at least 3 days per week. Requirements:...Contract work3 days per week
$140.6k - $190.6k per year
The Lead AI Forward Engineer designs and guides the delivery of AI‑powered solutions that reduce operational toil and accelerate... ..., governance, privacy, and auditability. AI System Design and Technology Strategy Evaluate and recommend AI/ML technologies and platforms (LLM...Manual laborWork at officeLocal areaFlexible hours2 days per week3 days per week$165.31k - $222.84k per year
Amendment Type: #1 Amendment Date: 06/19/2026 Reason for Amendment... ...Extended Job ID: 64580 Job Category: Engineering & Technical Division & Section: Development Review, Engineering Review Work... ...impacts on transportation and utility systems. Ensures development activity is...Permanent employmentFull timeInternshipRemote workMonday to fridayFlexible hoursShift work- ...is seeking a highly skilled and hands-on AI Engineer with proven experience in deploying AI/ML... ...techniques for improved accuracy. Prototype and evaluate AI solutions using proofs of concept and... ...design and cloud-native ML system deployment. Ability to design and automate...Remote workFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Code Reviewer & Systems Evaluation Engineer 1. Be the first to apply!
