Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Engineer, Trust & Evaluation

Equinix

Who are we?

Equinix is the world’s digital infrastructure company®, shortening the path to connectivity to enable the innovations that enrich our work, life and planet.

A place where tech thinkers and future builders turn bold ideas into breakthrough experiences, we welcome your unique perspective. Help us challenge assumptions, uncover bias, and remove barriers—because progress starts with fresh ideas. You’ll find belonging, purpose, and a team that welcomes you—because when you feel valued, you’re empowered to do your best work.

Job Summary

Designs, develops and deploys the evaluation and trust capabilities of Equinix's GenAI platform. Builds platform features that let teams evaluate their agents at scale: golden datasets, synthetic test data, scoring with LLM-as-judge, and monitoring of agent quality in production. Builds and tests the trust controls that agents run on, including agent identity, access, guardrails and data boundaries. Traces agent failures to their root cause and fixes them in the platform. Works in a team with senior engineers and with product, platform and security partners to turn defined problems into robust, secure, observable and performant solutions.

Knowledge and Application

Applies solid knowledge of software engineering practice and of LLM and agent concepts (prompting, retrieval, tool and function calling, orchestration, MCP). Understands evaluation methods (code-based checks, LLM-as-judge, human review, golden datasets, regression suites) and evaluation statistics (sampling, variance across runs, agreement between reviewers). Understands AI security risks such as prompt injection and data leakage, and identity concepts such as OAuth and per-user access. Follows established standards and suggests improvements.

Complexity and Problem Solving

Works on problems of defined scope where analysis requires evaluation of identifiable factors. Traces an agent failure to its cause (retrieval, tool call, prompt, model, data or infrastructure) using logs and traces, and follows it until it is fixed. Judges whether a metric reflects real quality. Escalates unusual or high-impact issues. Work is reviewed at defined checkpoints.

Collaboration and Interaction

Works with AI engineers, product managers, forward-deployed engineers, platform and security teams. Turns the needs of agent builders into platform features. Explains results, defects and risks to partners who are not familiar with the subject. Asks for guidance early and shares knowledge with the team.

Responsibilities

Evaluation platform (primary focus)

  • Build platform features for evaluation: dataset creation and versioning, golden datasets, synthetic test data and reusable metric templates
  • Build scoring for agents: code-based checks, LLM-as-judge, tool-call accuracy and grounding checks
  • Build human-review features that compare judge scores against human reviewers, and recalibrate when agreement is low
  • Run evaluations at scale: before release, on every change to a prompt, model or tool, and on agents in production
  • Build evaluation playbooks so agent builders can evaluate their agents without specialist help
  • Keep results traceable to the agent, model, prompt and dataset versions tested, with a record of who approved each release

Trust and safety

  • Build and test the platform's trust controls: agent identity, per-user access to connected systems, tool permissions, guardrails and data boundaries
  • Run adversarial tests and red-teaming (prompt injection, out-of-scope requests, data-boundary tests), and turn findings into fixes and regression tests
  • Work with security and governance teams to turn policy requirements into automated checks

Root-cause analysis and quality

  • Investigate agent failures and performance problems end to end, and follow them until the cause is fixed
  • Build test and performance tooling: end-to-end agent tests, load and latency tests, and failure diagnostics
  • Turn production failures and user feedback into regression tests, so the same defect does not return

Quality and responsible practice

  • Write tested, reviewed and documented code, and contribute to shared libraries and tooling
  • Test and build to security, data-privacy and responsible-AI requirements

Qualifications

Required

  • 1–4 years in AI/ML or software engineering (internships and applied research count)
  • Bachelor's degree in computer science, engineering, data science, mathematics or a related field, or equivalent experience
  • Proficiency in Python, with solid habits: version control, testing, code review. Experience building backend services or platform features
  • Hands-on experience building, not only using, evaluations or test systems for ML or LLM applications, including reading failures from logs or traces
  • Working knowledge of LLM applications (prompting, retrieval, agents)
  • Working knowledge of evaluation statistics (sampling, confidence intervals, variance across runs) and of measuring agreement between reviewers
  • Ability to write SQL and analyze results
  • Clear written and verbal communication
#J-18808-Ljbffr
Vacancy posted 5 hours ago
Similar jobs that could be interesting for youBased on the AI Engineer, Trust & Evaluation in Toronto, ON vacancy
  • $140k - $175k per year

     ...Reuters Labs is seeking a Lead Research Engineer to lead the design and delivery of...  ...software for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable...  ...actors, report the facts, and provide trusted, unbiased information to people all over... 
    Suggested
    Full time
    Internship
    Work at office
    Local area
    Flexible hours

    Thomson Reuters

    Toronto, ON
    10 days ago
  • $126k - $176k per year

     ...Sr. Legal AI Prompter / Evaluator Job Description Sr. Legal AI Prompters / Evaluators are responsible...  ...matter experts (SMEs), software engineers, and data scientists to make sure our products...  ...outputs that legal professionals can trust. As senior members of the team, they take... 
    Suggested
    Remplacement
    Full time
    Work at office
    Local area
    Flexible hours

    Thomson Reuters

    Toronto, ON
    6 days ago
  • $85 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Dorsey . Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience) Type: Contract...  ...AI coding agents to complete and evaluate complex infrastructure engineering tasks.... 
    Suggested
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  •  ...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This is a task-based...  ...clinical EHR vignettes for a medical AI evaluation project. General and internal medicine are the... 
    Suggested
    Part time
    For contractors
    Remote work
    Flexible hours

    Mindrift

    Toronto, ON
    3 days ago
  • $80 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type: Contract...  ...frontier AI coding agents to complete and evaluate complex data engineering tasks.... 
    Suggested
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $30 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors...  ...tasks guiding it answering questions and correcting errors. Evaluate the experience and deliverables providing concise feedback on performance... 
    Hourly pay
    Weekly pay
    Full time
    Contract work
    For contractors
    Summer work
    Remote work

    Mercor

    Toronto, ON
    6 days ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...pipelines , and technical reports. Evaluate AI-generated or human-created work against...  ...Provide detailed written justifications for evaluations and scores to maintain transparency.... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    26 days ago
  • $136.7k - $182.3k per year

     ...company, which means we pair our small, senior engineering team with the reach of one of the world's...  ...it globally. Because it runs generative AI in children's homes, safety isn't a...  ...core of the product. As our first dedicated Trust & Safety engineer, you'll own the systems... 
    Full time
    Flexible hours

    Spin Master

    Toronto, ON
    10 days ago
  • $156k - $188k per year

     ...Funding: $177M Series C  Employees: 200+ Trusted by tens of thousands of providers and...  ...successful launch and rollout. Partner with engineering leaders and product managers to generate,...  ...in your communication. Default trust As a Nexer, you do not have to earn trust... 
    Live In
    Remote work
    Flexible hours

    NexHealth

    Toronto, ON
    1 day ago
  • $72k - $138k per year

     ...ways of deploying capital, enables fair, trusted, and functioning social and economic...  ...-- We are looking for a passionate AI Research Engineer to join our team. You will work at the...  ...research and product engineering—designing, evaluating, and deploying generative AI (GenAI)... 
    Permanent employment
    Flexible hours

    Deloitte

    Toronto, ON
    6 days ago
  • $150k - $210k per year

     ...buying. Our first product is the AI media manager for Amazon DSP....  .... We're hiring an AI Agent Engineer to join our engineering team...  ...providers. Stay model-agnostic, evaluate new models within hours of...  ...improvements to production, stood up trusted evals on our core workflows,... 
    Long term contract
    Temporary work
    Summer work
    Internship
    Work at office

    gigi

    Toronto, ON
    4 days ago
  • $39 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation...  ...of experience as a music producer, audio engineer, and/or sound mixer. ~ College degree... 
    Remote job
    Contract work
    Summer work
    Immediate start

    Mercor

    Toronto, ON
    more than 2 months ago
  • $84k - $175k per year

     ...deploying capital, enables fair, trusted, and functioning social and...  ...design and implementation of AI-powered applications and agents...  ...solutions including:  o Prompt engineering o Retrieval-Augmented...  ...mitigation o Monitoring and evaluation of model outputs • Integrate... 
    Permanent employment
    Flexible hours

    Deloitte

    Toronto, ON
    7 days ago
  • $130k - $155k per year

     ...Our vision is to build an ML/AI first network of advertisers and...  ...hiring its first dedicated AI Engineer. Over the past year, AI at...  ...not the model. It is getting trusted data and reliable actions in front...  ...demo and a production system: evaluation, monitoring, cost, and a clear... 
    Full time
    Work at office
    Remote work

    Fluent, LLC

    Toronto, ON
    11 days ago
  • $70 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...hour Location: Remote Role Responsibilities Evaluate AI-generated responses to strengthen reasoning and rigor in model... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ..., and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    more than 2 months ago
  • $175k - $210k per year

     ...Guidepoint is investing in applied AI across its research platform....  ...for a hands-on Senior AI Engineer to take working prototypes into...  ...ownership across agent design, evaluation, APIs, and cloud deployment,...  ...providers. Ground responses in trusted data: Build retrieval and... 
    Full time
    Internship
    Work at office

    Guidepoint

    Toronto, ON
    1 day ago
  • $140k - $160k per year

     ...Join our Engineering team, where you'll help build our agentic future. Our Applied AI Engineers don't just call an LLM and hope it works...  ...next time, and raise the bar for evaluation across the team....  ...worse. Data residency and trust: Make sure AI features respect... 
    Manual labor
    Work at office
    3 days per week
    1 day per week

    ContactMonkey

    Toronto, ON
    1 day ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    more than 2 months ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    more than 2 months ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...pipelines , and A/B test write-ups . Evaluate AI-generated or human-created work against...  ...detailed written justifications for evaluations and scores to maintain transparency and accountability... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    more than 2 months ago
  • $130k - $165k per year

     ...NetBrain’s platform. What We Need We’re looking for a Senior AI Engineer to design and build production-grade agent and RAG systems that...  ...system-level thinking—owning everything from architecture and evaluation to scalability, observability, and reliability in production.... 

    NetBrain

    Toronto, ON
    1 day ago
  • $20 - $160 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Position: Generalist Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $20–$160/hour... 
    Remote job
    Contract work
    Summer work

    Mercor

    Toronto, ON
    a month ago
  • $70 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...peer-reviewed journals. Prior scientific software or research engineering experience. Start Date ~ Immediate Interview Process... 
    Remote job
    Contract work
    Summer work
    Immediate start

    Mercor

    Toronto, ON
    a month ago
  • $140.6k - $190.6k per year

     ...Job Description: Engineering Manager, Are you passionate about transforming...  ...future of legal research with AI? Join our Global Westlaw...  ...help legal professionals find trusted answers, uncover insights, and...  ...coding practices, and continuous evaluation of quality. Design and manage... 
    Full time
    Internship
    Work at office
    Local area
    Worldwide
    Flexible hours
    Shift work

    Thomson Reuters

    Toronto, ON
    10 days ago
  •  ...Director, Engineering, AI Job Description Are you passionate about transforming how legal professionals find trusted answers with AI? Join our Global Westlaw Engineering organization and...  ...and model integration through evaluation, deployment, and monitoring. Set the... 
    Long term contract
    Full time
    Contract work
    Work at office
    Local area
    Flexible hours
    Shift work

    Thomson Reuters

    Toronto, ON
    10 days ago
  • $173k - $197.4k per year

     ...Manager, AI Engineer About Capital One Canada For 30 years, we've been on a mission to change...  ...inference, similarity search, guardrails, model evaluation, experimentation, governance, and...  ...trajectory. Find your rhythm: We believe trust fosters the flexibility and autonomy... 
    Long term contract
    Full time
    Work at office
    Work from home

    Capital One

    Toronto, ON
    10 days ago
  •  ...connected future through world-class networks, AI-powered solutions, and digital experiences...  ...We are looking for a hands-on AI Engineer (CP2) to design, build, and deploy the next...  ...deployments. Reliability & Quality: Implement evaluation metrics, automated unit/integration tests,... 
    Full time
    Work at office
    3 days per week

    Bell

    Toronto, ON
    7 days ago
  • $135k - $210k per year

     ...Overview:   Guidepoint seeks an experienced AI Engineer as an integral member of the Toronto-based AI team. The Toronto Technology Hub...  ...establish and implement best practices for LLMOps, including automated evaluation using frameworks like LLM Judges or MLflow, AI observability,... 
    Internship

    Guidepoint

    Toronto, ON
    1 day ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Jack Dorsey . Position: Software Engineer Expert Type: Contract Compensation...  ...Remote Role Responsibilities Evaluate AI-generated artifacts for usability in... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    Toronto, ON
    a month ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Engineer, Trust & Evaluation. Be the first to apply!