Machine Learning Engineer, Evaluation and Quality, Creativity Apps
$119.1k - $230.5k per yearApple, Inc.
This position requires local presence. Please view similar jobs below.
Machine Learning Engineer, Evaluation and Quality, Creativity Apps
Vancouver, British Columbia, Canada Software and Services
Imagine what you could do here. At Apple, new ideas have a way of becoming outstanding products, services, and customer experiences very quickly. Bring passion and dedication to your job and there's no telling what you could accomplish.The Creativity Apps team is looking for a Machine Learning Engineer focused on the quality and evaluation of ML models and AI features in our creativity applications. You will define how we measure whether the product and AI features are good enough to ship: designing the evaluation strategy and metrics, building the datasets and infrastructure behind them, and driving end-to-end quality from model development through to the feature in customers' hands. You will have the unique and rewarding opportunity to help shape upcoming products that will delight and inspire millions of creators every day.
Description
In this role, you will own the evaluation and quality of AI features across Creativity Apps. You will work closely with Machine Learning engineering, infrastructure, product, and design teams to understand each model and use case, and translate them into evaluation methodology, testing strategy, and quality metrics that reflect what creators actually experience.
You will:
- * Define evaluation strategy and metrics for ML models, LLMs, and AI features, tailored to the model, the product, and its use cases.
- * Curate and create evaluation datasets, measure and improve dataset quality, and run iterative improvement loops so that datasets stay representative of each product.
- * Design, build, maintain, and optimize evaluation infrastructure at scale, with reproducible runs, tracked baselines, and comparisons between model versions.
- * Apply ML, LLMs, and agentic workflows to the evaluation itself, including automated judging, test generation, and analysis, to make evaluation more efficient, more accurate, and more scalable.
- * Drive end-to-end evaluation and quality of AI features, from model checkpoints to integrated, shipping features, including triaging issues and driving them to resolution.
- * Report quality clearly, turning evaluation results into ship/no-ship risk and recommendations for both technical and non-technical audiences.
- * Partner with and drive cross-functional teams, comfortable with ambiguity and making educated, critical judgments.
Your curiosity about the latest developments in ML and generative AI, and your drive to apply new technology to practical problems, will set you apart.
Minimum Qualifications
- 5+ years of industry experience in machine learning or software engineering, with a focus on ML/AI model evaluation and quality.
- Experience with evaluation methodology for ML models, LLMs, and AI products or features.
- Strong understanding of the ML development cycle, from data and training to evaluation, deployment, and monitoring.
- Solid machine learning fundamentals and LLM knowledge, including agentic workflows, and you keep up with the latest developments in the field.
- Track record of applying LLMs, or agentic workflows to ML evaluation and quality.
- Experience in data curation, dataset creation, and dataset quality, including iterative improvement workflows that deliver high-quality datasets tailored to specific products.
- Experience designing, implementing, maintaining, or optimizing ML evaluation infrastructure at scale.
- Experience designing evaluation metrics based on the model, the product, and its use cases.
- Experience driving end-to-end evaluation and quality for AI products.
- Fluency in quality reporting, presenting quality data to both technical and non-technical audiences.
- Strong communication skills and experience working with and driving cross-functional partners.
- Proficient in Python and/or Swift or comparable languages.
- Bachelor's degree in Computer Science, Data Science, Machine Learning, or a related field, or equivalent experience.
Preferred Qualifications
- Knowledge of the software development life cycle and software quality assurance methodologies.
- Experience in objective image and video quality evaluation and metrics.
- Knowledge of color science and/or experience with subjective image or video quality review.
- Passion for creative applications; experience as a creator (photo, video, music, or design) is a plus.
- Excitement about learning new technology and applying it to practical problems.
- Familiarity with iOS and macOS.
At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $119,100 and $230,500, and your base pay will depend on your skills, qualifications, experience, and location.
Apple employees also have the opportunity to become an Apple shareholder through participation in Apple Inc.’s discretionary employee stock programs. Employees are eligible for discretionary restricted stock unit award recommendations, and can purchase Apple Inc. stock at a discount if voluntarily participating in Apple Inc.’s Employee Stock Purchase Plan. Participation in Apple Inc.’s discretionary stock programs is governed by Apple Inc.’s stock plans and agreements and are not part of local employment contracts or compensation.
You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits
Note: Apple benefit and compensation programs are subject to eligibility requirements and other terms of the applicable plan or program.
Apple is an equal opportunity employer that is committed to inclusion and diversity. Apple provides reasonable accommodations to applicants with disabilities. Apple is a drug-free workplace.
At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.
Learn about accessibility in Apple’s workplace
#J-18808-Ljbffr- ...Responsibilities Define evaluation strategies and metrics for ML models, LLMs, and AI... ...representative evaluation datasets and dataset-quality workflows. Design, build, maintain,... ...with and drive cross-functional machine learning, infrastructure, product, and design teams...QualityFull time
$165k - $200k per year
...Job Description As a Machine Learning Engineer, you'll design and implement scalable, high-performance machine learning systems to support DeviantArt... ...solutions that enhance user experiences and foster creativity within a high-traffic environment. Develop and improve algorithms...SuggestedFull timeWork at office1 day per week- ..., and Okta, our team includes engineers from premier tech companies and... ...at scale. Build agents that learn from the work they do. Ema... ...data, data becomes training and evaluation, and better agents produce... ...workflows, and quantify data quality, performance uplift and reliability...QualityFull time
$156k - $234k per year
...for seamlessly embedding cutting-edge machine learning, Generative AI, and autonomous agents directly... ...of deep applied research and scalable engineering. Whether we are developing... ...agents, advancing our next-generation AI engine (Workday Illuminate), or engineering highly...SuggestedFull timeWork at officeRemote workHome officeFlexible hours- ...and the ecosystem a billion machines will run on in the future. Very... ...Simulation team (Simulation Evaluation & Validation), working on Gaia... ..., Core Simulation Engineering, and Cloud/Infrastructure to... ...prediction, scenario generation quality, failure-case coverage). Mentor...QualityLong term contractFull timeWork at officeRemote workVisa sponsorshipRelocation packageFlexible hoursShift work
$104.5k - $142.8k per year
...a community that connects across the globe. A place where creativity thrives, new perspectives are invited, and ideas matter. A... ...the Senior Data Science Manager, we are seeking a full-stack Machine Learning Engineer to operationalize, deploy, and scale high-impact machine learning...Full timeLocal area$119.1k - $230.5k per year
...Software Development Engineer in Test, Full Stack, Creativity Apps Vancouver, British Columbia, Canada Software and Services Imagine what you could do... ...applications, cloud-based frameworks, and APIs with the highest quality, validating new and innovative on-device ML and AI...QualityLocal areaRelocation$3600 - $4500 per month
...forefront of robotics, AI, and machine vision — recognized with the... ...AI is seeking a Machine Learning Applied Scientist (Co-op) for... ...applications. You’ll prototype, evaluate, and improve models that are... ...Work with robotics and software engineers to understand deployment...Long term contractWork at officeFlexible hours$92.9k - $129.2k per year
...across the globe. A place where creativity thrives, new perspectives are... ...and provide opportunities for learning and leading that allow our... ...next revolution in games. Our Quality Verification and Standards (QVS... ...join the EA Sports FC Companion App Quality Verification team. This...QualityFull timeLocal area$84k - $115.5k per year
...advertising and marketing channels. StackAdapt is seeking a talented Quality Engineer who is passionate about software quality and test automation... ..., and transform our software development process. Learn more about our engineering culture here: StackAdapt is a...QualityLocal areaRemote workWork from homeHome office$85k - $95k per year
...flexibility. Enjoy weekly lunches, quality coffee, and regular social... ..., the Quality Assurance Engineer will be responsible for delivering... ...with engineering standards. Evaluate emerging AI technologies and identify... ...is designed to help us learn more about your skills and experience...QualityPermanent employmentFull timeWork at officeLocal areaWorldwideFlexible hours3 days per week$77.7k - $107.9k per year
...across the globe. A place where creativity thrives, new perspectives are invited... ...makes play happen. As a Quality Designer supporting the Research & Engineering team you will test AI-enabled game... ...potential, to creating space for learning and experimenting, we empower you...QualityFull timeTemporary workLocal area$113k - $153k per year
...apply today! Your Role in Supporting Our Members: As a Engineering Lead, Quality , you will play a senior technical role responsible for... ...approach to measuring and improving quality ~ Ability to evaluate quality effectiveness by analyzing automation outcomes and leveraging...QualityPermanent employmentFull timeContract workInternshipWork at officeImmediate startFlexible hoursShift work$85k - $120k per year
...looking for an Analyst, Analytics Quality to join our Process Engineering (PE ) team. The ideal candidate is... ...technology and are committed to continuous learning. You apply and integrate emerging... ...problems with structure and creativity. You can communicate complex concepts...QualityInternshipFlexible hours$120k - $180k per year
...data, signal processing, acoustics, sound engineering, sonar, or related scientific data.... ...intersection of signal processing, data science, machine learning and embedded product development. We... ...within the signal. Design, evaluate and improve signal-processing methodologies...QualityLong term contractFull timeRemote workFlexible hours$115.5k - $132.75k per year
...across voice and digital. Our AI agents learn from your best human agents and improve... ...Empathetic . Your role As an AI Evaluation Engineer, you'll be an integral part of our AI Evaluation... ..., model evaluation, or applied ML quality for AI-driven products. ~ Experience designing...QualityInternshipWork at office$7.73k - $11.11k per year
...Job Title ORCA Learning Designer Job Category M&P - AAPS... ...development, implementation, and evaluation of innovative assessment pedagogies... ...and maintains scripts, apps, and web-based automations to... ...Studios, LT Hub, etc.) to ensure quality instructor support. Disseminate...QualityWork at officeLocal areaFlexible hours$66.4k - $94.62k per year
...and delivering innovative 3D machine vision solutions to OEMs and System... .... What will you do as an Quality Systems Specialist? The Quality... ...successful? Bachelor's degree in Engineering, Quality, or related fields.... ...is safe with us. To learn more about how we use your personal...QualityFull time$175.5k - $218.25k per year
...problems in real time across voice and digital. Our AI agents learn from your best human agents and improve with every... ...We are seeking a talented, experienced Full-Stack Engineer passionate about building high-quality, scalable web applications using modern frontend and backend...QualityWork at office$67k - $92.7k per year
...across the globe. A place where creativity thrives, new perspectives are... ...to the club! As an Associate Quality Designer, you will work with... ...: You can partner with Engineering, Game Design, Analytics, and... ...potential, to creating space for learning and experimenting, we empower...QualityFull timeLocal area- ...how to build brands people remember, lead creative teams people want to be part of, and turn... ..., then we can’t wait to meet you. Qualities That Make You An Ideal Teammate At AlgaeCal... ...ahead of your own You’re hungry to learn more You’re people smart, in the sense...QualityFull timeCasual workWork at officeFlexible hours
- ...Staffmax is helping our client hire a skilled Machine Operator for a 5-month contract. This role is essential to production operations... ...and make adjustments as needed · Inspect finished products for quality control · Perform routine maintenance and troubleshooting ·...QualityContract workTemporary workManual laborShift work
- ...buyer is not just a button pusher. They understand the full system: creative, offer, conversion rate, platform dynamics, competition, and what... ...work on real brands with real budgets. We test aggressively. We learn quickly. We build frameworks that scale. You will grow into...Full time
$46.39 - $66.68 per hour
...Job Summary Come work as a Quality Leader, Clinical Quality and... ...Team as required to support evaluation and decision-making. Maintain... .../Training Facilitate learning and deliver educational sessions... ...Utilizes independent thinking and creative problem-solving abilities....QualityImmediate start- ...leading tech companies, focused on testing, evaluating, and improving AI systems. This is a... ...clinical EHR vignettes for a medical AI evaluation project. General and internal medicine are... ...; Score the clinical reasoning quality of benchmark items: vignette accuracy and...QualityPart timeFor contractorsRemote workFlexible hours
$70k - $85k per year
...Summary: As a Learning & Development Specialist in the IT industry, you will play a crucial role in enhancing the skills and knowledge... ...programmes and workshops tailored to the needs of employees. Evaluate the effectiveness of training initiatives through feedback and performance...Permanent employmentFull time$175k - $200k per year
...THE ROLE We're looking for a Director of Creative to lead our Performance Design department... ...& Output - Own the creative vision and quality standard for the Performance Design... ...campaigns vs. ads - Ensure we are testing & learning - Partner with Directive senior leadership...QualityPermanent employmentFull timeFor contractorsRemote workHome office- ...Job Title Learning Initiatives Coordinator Job Category M&P - AAPS Job Profile AAPS Salaried - Student Management, Level... ...plans, and other relevant materials required to deliver, implement, evaluate, and assess the efficacy of opt-in, low-barrier programs;...RemplacementTemporary workPart timeFixed term contractWork at officeFlexible hoursAfternoon shift
$104.5k - $142.8k per year
...connects across the globe. A place where creativity thrives, new perspectives are invited... ...play happen. Description In FC Quality Verification Engineering, our mission is to help our teams identify... ...FC, including for the Companion App on iOS, Android, and web. Role summary...QualityFull timeWork at officeLocal area3 days per week- ...with multilingual professionals to support AI development, data quality, and language-related projects. ## Job Overview We are seeking... ...multilingual language specialists to join our Audio Caption Evaluation Project. Selected contributors will evaluate audio captions for...QualityFor contractorsRemote workMonday to friday
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Machine Learning Engineer, Evaluation and Quality, Creativity Apps. Be the first to apply!
- machine learning engineer Vancouver, BC
- junior machine learning developer Vancouver, BC
- stage qualité Vancouver, BC
- qualité agroalimentaire Vancouver, BC
- qualité santé Vancouver, BC
- quality manufacturing Vancouver, BC
- qualité Vancouver, BC
- superviseur qualité Vancouver, BC
- quality inspection Vancouver, BC
- qualité environnement Vancouver, BC
