Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Machine Learning Engineer, Evaluation and Quality, Creativity Apps

$119.1k - $230.5k per year

Apple, Inc.

This position requires local presence. Please view similar jobs below.

Machine Learning Engineer, Evaluation and Quality, Creativity Apps

Vancouver, British Columbia, Canada Software and Services

Imagine what you could do here. At Apple, new ideas have a way of becoming outstanding products, services, and customer experiences very quickly. Bring passion and dedication to your job and there's no telling what you could accomplish.The Creativity Apps team is looking for a Machine Learning Engineer focused on the quality and evaluation of ML models and AI features in our creativity applications. You will define how we measure whether the product and AI features are good enough to ship: designing the evaluation strategy and metrics, building the datasets and infrastructure behind them, and driving end-to-end quality from model development through to the feature in customers' hands. You will have the unique and rewarding opportunity to help shape upcoming products that will delight and inspire millions of creators every day.

Description


In this role, you will own the evaluation and quality of AI features across Creativity Apps. You will work closely with Machine Learning engineering, infrastructure, product, and design teams to understand each model and use case, and translate them into evaluation methodology, testing strategy, and quality metrics that reflect what creators actually experience.

You will:


  • * Define evaluation strategy and metrics for ML models, LLMs, and AI features, tailored to the model, the product, and its use cases.

  • * Curate and create evaluation datasets, measure and improve dataset quality, and run iterative improvement loops so that datasets stay representative of each product.

  • * Design, build, maintain, and optimize evaluation infrastructure at scale, with reproducible runs, tracked baselines, and comparisons between model versions.

  • * Apply ML, LLMs, and agentic workflows to the evaluation itself, including automated judging, test generation, and analysis, to make evaluation more efficient, more accurate, and more scalable.

  • * Drive end-to-end evaluation and quality of AI features, from model checkpoints to integrated, shipping features, including triaging issues and driving them to resolution.

  • * Report quality clearly, turning evaluation results into ship/no-ship risk and recommendations for both technical and non-technical audiences.

  • * Partner with and drive cross-functional teams, comfortable with ambiguity and making educated, critical judgments.

Your curiosity about the latest developments in ML and generative AI, and your drive to apply new technology to practical problems, will set you apart.

Minimum Qualifications



  • 5+ years of industry experience in machine learning or software engineering, with a focus on ML/AI model evaluation and quality.

  • Experience with evaluation methodology for ML models, LLMs, and AI products or features.

  • Strong understanding of the ML development cycle, from data and training to evaluation, deployment, and monitoring.

  • Solid machine learning fundamentals and LLM knowledge, including agentic workflows, and you keep up with the latest developments in the field.

  • Track record of applying LLMs, or agentic workflows to ML evaluation and quality.

  • Experience in data curation, dataset creation, and dataset quality, including iterative improvement workflows that deliver high-quality datasets tailored to specific products.

  • Experience designing, implementing, maintaining, or optimizing ML evaluation infrastructure at scale.

  • Experience designing evaluation metrics based on the model, the product, and its use cases.

  • Experience driving end-to-end evaluation and quality for AI products.

  • Fluency in quality reporting, presenting quality data to both technical and non-technical audiences.

  • Strong communication skills and experience working with and driving cross-functional partners.

  • Proficient in Python and/or Swift or comparable languages.

  • Bachelor's degree in Computer Science, Data Science, Machine Learning, or a related field, or equivalent experience.

Preferred Qualifications



  • Knowledge of the software development life cycle and software quality assurance methodologies.

  • Experience in objective image and video quality evaluation and metrics.

  • Knowledge of color science and/or experience with subjective image or video quality review.

  • Passion for creative applications; experience as a creator (photo, video, music, or design) is a plus.

  • Excitement about learning new technology and applying it to practical problems.

  • Familiarity with iOS and macOS.

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $119,100 and $230,500, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple Inc.’s discretionary employee stock programs. Employees are eligible for discretionary restricted stock unit award recommendations, and can purchase Apple Inc. stock at a discount if voluntarily participating in Apple Inc.’s Employee Stock Purchase Plan. Participation in Apple Inc.’s discretionary stock programs is governed by Apple Inc.’s stock plans and agreements and are not part of local employment contracts or compensation.

You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit and compensation programs are subject to eligibility requirements and other terms of the applicable plan or program.

Apple is an equal opportunity employer that is committed to inclusion and diversity. Apple provides reasonable accommodations to applicants with disabilities. Apple is a drug-free workplace.

At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.

Learn about accessibility in Apple’s workplace

#J-18808-Ljbffr
Vacancy posted 6 hours ago
Similar jobs that could be interesting for youBased on the Machine Learning Engineer, Evaluation and Quality, Creativity Apps in Vancouver, BC vacancy
  •  ...Responsibilities Define evaluation strategies and metrics for ML models, LLMs, and AI...  ...representative evaluation datasets and dataset-quality workflows. Design, build, maintain,...  ...with and drive cross-functional machine learning, infrastructure, product, and design teams... 
    Quality
    Full time

    Apple

    Vancouver, BC
    8 days ago
  • $165k - $200k per year

     ...Job Description As a Machine Learning Engineer, you'll design and implement scalable, high-performance machine learning systems to support DeviantArt...  ...solutions that enhance user experiences and foster creativity within a high-traffic environment. Develop and improve algorithms... 
    Suggested
    Full time
    Work at office
    1 day per week

    Wix

    Vancouver, BC
    10 days ago
  •  ..., and Okta, our team includes engineers from premier tech companies and...  ...at scale. Build agents that learn from the work they do. Ema...  ...data, data becomes training and evaluation, and better agents produce...  ...workflows, and quantify data quality, performance uplift and reliability... 
    Quality
    Full time

    ema

    Vancouver, BC
    7 days ago
  • $156k - $234k per year

     ...for seamlessly embedding cutting-edge machine learning, Generative AI, and autonomous agents directly...  ...of deep applied research and scalable engineering. Whether we are developing...  ...agents, advancing our next-generation AI engine (Workday Illuminate), or engineering highly... 
    Suggested
    Full time
    Work at office
    Remote work
    Home office
    Flexible hours

    Workday

    Vancouver, BC
    5 days ago
  •  ...and the ecosystem a billion machines will run on in the future. Very...  ...Simulation team (Simulation Evaluation & Validation), working on Gaia...  ..., Core Simulation Engineering, and Cloud/Infrastructure to...  ...prediction, scenario generation quality, failure-case coverage). Mentor... 
    Quality
    Long term contract
    Full time
    Work at office
    Remote work
    Visa sponsorship
    Relocation package
    Flexible hours
    Shift work

    Wayve

    Vancouver, BC
    10 days ago
  • $104.5k - $142.8k per year

     ...a community that connects across the globe. A place where creativity thrives, new perspectives are invited, and ideas matter. A...  ...the Senior Data Science Manager, we are seeking a full-stack Machine Learning Engineer to operationalize, deploy, and scale high-impact machine learning... 
    Full time
    Local area

    Electronic Arts

    Vancouver, BC
    10 days ago
  • $119.1k - $230.5k per year

     ...Software Development Engineer in Test, Full Stack, Creativity Apps Vancouver, British Columbia, Canada Software and Services Imagine what you could do...  ...applications, cloud-based frameworks, and APIs with the highest quality, validating new and innovative on-device ML and AI... 
    Quality
    Local area
    Relocation

    Apple Inc.

    Vancouver, BC
    7 days ago
  • $3600 - $4500 per month

     ...forefront of robotics, AI, and machine vision — recognized with the...  ...AI is seeking a Machine Learning Applied Scientist (Co-op) for...  ...applications. You’ll prototype, evaluate, and improve models that are...  ...Work with robotics and software engineers to understand deployment... 
    Long term contract
    Work at office
    Flexible hours

    Apera AI Inc

    Vancouver, BC
    1 day ago
  • $92.9k - $129.2k per year

     ...across the globe. A place where creativity thrives, new perspectives are...  ...and provide opportunities for learning and leading that allow our...  ...next revolution in games. Our Quality Verification and Standards (QVS...  ...join the EA Sports FC Companion App Quality Verification team. This... 
    Quality
    Full time
    Local area

    Electronic Arts

    Vancouver, BC
    10 days ago
  • $84k - $115.5k per year

     ...advertising and marketing channels. StackAdapt is seeking a talented Quality Engineer who is passionate about software quality and test automation...  ..., and transform our software development process.   Learn more about our engineering culture here:   StackAdapt is a... 
    Quality
    Local area
    Remote work
    Work from home
    Home office

    StackAdapt

    Vancouver, BC
    1 day ago
  • $85k - $95k per year

     ...flexibility. Enjoy weekly lunches, quality coffee, and regular social...  ..., the Quality Assurance Engineer will be responsible for delivering...  ...with engineering standards. Evaluate emerging AI technologies and identify...  ...is designed to help us learn more about your skills and experience... 
    Quality
    Permanent employment
    Full time
    Work at office
    Local area
    Worldwide
    Flexible hours
    3 days per week

    Trulioo

    Vancouver, BC
    7 days ago
  • $77.7k - $107.9k per year

     ...across the globe. A place where creativity thrives, new perspectives are invited...  ...makes play happen. As a Quality Designer supporting the Research & Engineering team you will test AI-enabled game...  ...potential, to creating space for learning and experimenting, we empower you... 
    Quality
    Full time
    Temporary work
    Local area

    Electronic Arts

    Vancouver, BC
    10 days ago
  • $113k - $153k per year

     ...apply today! Your Role in Supporting Our Members: As a Engineering Lead, Quality , you will play a senior technical role responsible for...  ...approach to measuring and improving quality ~ Ability to evaluate quality effectiveness by analyzing automation outcomes and leveraging... 
    Quality
    Permanent employment
    Full time
    Contract work
    Internship
    Work at office
    Immediate start
    Flexible hours
    Shift work

    Vancity

    Vancouver, BC
    6 hours ago
  • $85k - $120k per year

     ...looking for an Analyst, Analytics Quality to join our  Process Engineering (PE ) team. The ideal candidate is...  ...technology and are committed to continuous learning. You apply and integrate emerging...  ...problems with structure and creativity.  You can communicate complex concepts... 
    Quality
    Internship
    Flexible hours

    Connor, Clark & Lunn Financial Group Ltd.

    Vancouver, BC
    1 day ago
  • $120k - $180k per year

     ...data, signal processing, acoustics, sound engineering, sonar, or related scientific data....  ...intersection of signal processing, data science, machine learning and embedded product development. We...  ...within the signal. Design, evaluate and improve signal-processing methodologies... 
    Quality
    Long term contract
    Full time
    Remote work
    Flexible hours

    BLANKSLATE Partners

    Vancouver, BC
    more than 2 months ago
  • $115.5k - $132.75k per year

     ...across voice and digital. Our AI agents learn from your best human agents and improve...  ...Empathetic . Your role As an AI Evaluation Engineer, you'll be an integral part of our AI Evaluation...  ..., model evaluation, or applied ML quality for AI-driven products. ~ Experience designing... 
    Quality
    Internship
    Work at office

    Software Engineering, Data Science

    Vancouver, BC
    more than 2 months ago
  • $7.73k - $11.11k per year

     ...Job Title ORCA Learning Designer Job Category M&P - AAPS...  ...development, implementation, and evaluation of innovative assessment pedagogies...  ...and maintains scripts, apps, and web-based automations to...  ...Studios, LT Hub, etc.) to ensure quality instructor support. Disseminate... 
    Quality
    Work at office
    Local area
    Flexible hours

    UBC The University of British Columbia

    Vancouver, BC
    7 days ago
  • $66.4k - $94.62k per year

     ...and delivering innovative 3D machine vision solutions to OEMs and System...  .... What will you do as an Quality Systems Specialist? The Quality...  ...successful? Bachelor's degree in Engineering, Quality, or related fields....  ...is safe with us. To learn more about how we use your personal... 
    Quality
    Full time

    LMI Technologies

    Vancouver, BC
    6 days ago
  • $175.5k - $218.25k per year

     ...problems in real time across voice and digital. Our AI agents learn from your best human agents and improve with every...  ...We are seeking a talented, experienced Full-Stack Engineer passionate about building high-quality, scalable web applications using modern frontend and backend... 
    Quality
    Work at office

    Software Engineering

    Vancouver, BC
    more than 2 months ago
  • $67k - $92.7k per year

     ...across the globe. A place where creativity thrives, new perspectives are...  ...to the club! As an Associate Quality Designer, you will work with...  ...: You can partner with Engineering, Game Design, Analytics, and...  ...potential, to creating space for learning and experimenting, we empower... 
    Quality
    Full time
    Local area

    Electronic Arts

    Vancouver, BC
    11 days ago
  •  ...how to build brands people remember, lead creative teams people want to be part of, and turn...  ..., then we can’t wait to meet you. Qualities That Make You An Ideal Teammate  At AlgaeCal...  ...ahead of your own You’re hungry to learn more You’re people smart, in the sense... 
    Quality
    Full time
    Casual work
    Work at office
    Flexible hours

    AlgaeCal

    Vancouver, BC
    a month ago
  •  ...Staffmax is helping our client hire a skilled Machine Operator for a 5-month contract. This role is essential to production operations...  ...and make adjustments as needed · Inspect finished products for quality control · Perform routine maintenance and troubleshooting ·... 
    Quality
    Contract work
    Temporary work
    Manual labor
    Shift work

    Staffmax Staffing & Recruiting

    Vancouver, BC
    13 days ago
  •  ...buyer is not just a button pusher. They understand the full system: creative, offer, conversion rate, platform dynamics, competition, and what...  ...work on real brands with real budgets. We test aggressively. We learn quickly. We build frameworks that scale. You will grow into... 
    Full time

    Sweat Pants Agency

    Vancouver, BC
    18 days ago
  • $46.39 - $66.68 per hour

     ...Job Summary Come work as a Quality Leader, Clinical Quality and...  ...Team as required to support evaluation and decision-making. Maintain...  .../Training Facilitate learning and deliver educational sessions...  ...Utilizes independent thinking and creative problem-solving abilities.... 
    Quality
    Immediate start

    Vancouver Coastal Health

    Vancouver, BC
    7 days ago
  •  ...leading tech companies, focused on testing, evaluating, and improving AI systems. This is a...  ...clinical EHR vignettes for a medical AI evaluation project. General and internal medicine are...  ...; Score the clinical reasoning quality of benchmark items: vignette accuracy and... 
    Quality
    Part time
    For contractors
    Remote work
    Flexible hours

    Mindrift

    Vancouver, BC
    3 days ago
  • $70k - $85k per year

     ...Summary: As a Learning & Development Specialist in the IT industry, you will play a crucial role in enhancing the skills and knowledge...  ...programmes and workshops tailored to the needs of employees. Evaluate the effectiveness of training initiatives through feedback and performance... 
    Permanent employment
    Full time

    CA Demo

    Vancouver, BC
    6 hours ago
  • $175k - $200k per year

     ...THE ROLE We're looking for a Director of Creative to lead our Performance Design department...  ...& Output - Own the creative vision and quality standard for the Performance Design...  ...campaigns vs. ads - Ensure we are testing & learning - Partner with Directive senior leadership... 
    Quality
    Permanent employment
    Full time
    For contractors
    Remote work
    Home office

    Directive

    Vancouver, BC
    18 days ago
  •  ...Job Title Learning Initiatives Coordinator Job Category M&P - AAPS Job Profile AAPS Salaried - Student Management, Level...  ...plans, and other relevant materials required to deliver, implement, evaluate, and assess the efficacy of opt-in, low-barrier programs;... 
    Remplacement
    Temporary work
    Part time
    Fixed term contract
    Work at office
    Flexible hours
    Afternoon shift

    UBC The University of British Columbia

    Vancouver, BC
    7 days ago
  • $104.5k - $142.8k per year

     ...connects across the globe. A place where creativity thrives, new perspectives are invited...  ...play happen. Description In FC Quality Verification Engineering, our mission is to help our teams identify...  ...FC, including for the Companion App on iOS, Android, and web. Role summary... 
    Quality
    Full time
    Work at office
    Local area
    3 days per week

    Electronic Arts

    Vancouver, BC
    10 days ago
  •  ...with multilingual professionals to support AI development, data quality, and language-related projects. ## Job Overview We are seeking...  ...multilingual language specialists to join our Audio Caption Evaluation Project. Selected contributors will evaluate audio captions for... 
    Quality
    For contractors
    Remote work
    Monday to friday

    Agents Only Technologies

    Vancouver, BC
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Machine Learning Engineer, Evaluation and Quality, Creativity Apps. Be the first to apply!