Python Developer-AI Evaluation

Freelancing Opportunity: We are hiring Senior Software Engineers – Python / TypeScript (USA) for a short-term AI Engineering and Evaluation project focused on improving AI models through high-quality training data, preference data evaluation, and engineering workflows.

Candidates must be based in Canada.

Role: Python Developer-AI Evaluation

Experience: 4+ Years

Location: Canada

Mode of Work: Remote

Engagement: Freelance / Project-Based

Mandatory Eligibility Criteria

Technical Qualification:

• Experience building, maintaining, and improving software pipelines and infrastructure.
• Hands-on experience with preference data collection and/or rubric creation.
• Ability to review technical data and evaluate quality using defined criteria.
• Strong analytical, problem-solving, and technical judgment skills.
• Experience working with large codebases, such as VS Code, is an advantage.
• Ability to explain why a technical solution or output is good, accurate, and effective.
• Strong written and verbal communication skills.

Educational Qualification:

• Bachelor’s or Master’s degree in a relevant field, including BCA, B.Tech, B.E., MCA, or M.Tech.

Experience:

• Candidates from Tier 1 companies are preferred.
• Candidates must not currently be working with Microsoft.

Availability:

• Minimum 4 hours per day and 20 hours per week.
• Minimum 4 hours of overlap with Pacific Standard Time (PST) working hours.
• Immediate availability preferred.
• Ability to commit to the full 3-month contract.

Others:

• An updated LinkedIn profile is mandatory for candidate review. Please include the LinkedIn profile URL in your resume.
• Ability to work independently in a fully remote environment.
• Strong attention to detail and commitment to high-quality deliverables.
• Ability to work in a fast-paced environment and deliver results within deadlines.
• Good communication and collaboration skills to work with researchers and engineering teams.

What You’ll Work On

• Design, develop, and improve evaluation rubrics and criteria for preference data.
• Review and label data while ensuring high standards of quality and accuracy.
• Identify subtle issues and inconsistencies in data and evaluation results.
• Build and maintain pipelines and infrastructure for data generation, collection, and evaluation.
• Analyze technical findings and prepare clear reports, summaries, and recommendations.
• Collaborate with researchers and engineers to convert qualitative judgments into scalable processes.
• Support AI training and evaluation workflows to improve the quality of AI model outputs.
• Communicate technical findings clearly and recommend improvements to project teams.

Evaluation Process

• AI Interview
• Technical and Cultural Interview – Approximately 30 minutes.

Perks of Freelancing

• Fully remote work environment.
• Opportunity to work on cutting-edge AI projects with leading AI companies.
• Opportunity to contribute to AI model training, evaluation, and engineering initiatives.
• Competitive hourly compensation based on candidate quality and interview feedback.