ML Kernel Performance Engineer, Edge AI and Science (Vancouver)

Job Overview

Amazon Devices is an inventive research and development company that designs and engineers high‑profile consumer products like the Kindle family, Fire Tablets, Fire TV, Health & Wellness devices, Amazon Echo, and Astro. We are building the next generation of edge AI capabilities through our advanced compression platform and custom neural accelerator silicon. Within Edge AI & Science, the AI Platform team builds a compression platform—the first of its kind—enabling 20‑100x neural network compression for edge and cloud deployment.

We are looking for an ML Kernel Performance Engineer to work at the hardware-software boundary of this platform, crafting high-performance CUDA and Triton kernels that make our compression algorithms run at peak efficiency during training, fine‑tuning, and inference.

Key Responsibilities

• Design and implement high-performance CUDA and Triton kernels for quantization‑aware training, sparse matrix operations, and low‑bit inference on modern GPU accelerators.
• Analyze and optimize kernel‑level performance for compression training workloads, conducting detailed performance analysis using profiling tools to identify and resolve bottlenecks that slow model training from days to weeks.
• Implement kernel‑level optimizations such as operator fusion, tiling, memory access pattern optimization, and scheduling for compression‑specific compute patterns.
• Build a kernel development harness that enables any team member to profile kernel performance, test forward/backward accuracy, and validate at production scale.
• Maintain and extend the team’s training kernels library with clean interfaces, CI, and examples that enable scientists to contribute kernel improvements alongside platform engineers.
• Collaborate closely with Applied Scientists, compiler engineers, and hardware architects to co‑design ML‑centric solutions that unify software and hardware for both cloud and edge deployment.
• Develop inference kernels for cloud deployment (custom backends for quantized models that keep weights packed in memory and reconstruct on the fly for compute).
• Build and maintain performance regression tests and benchmarking infrastructure that track kernel efficiency as models scale from billions to hundreds of billions of parameters.

A Day in the Life

A scientist files a ticket: “QAT training on our large model is 4x slower than expected.” The engineer pulls up the profiler, identifies that a custom quantizer kernel is thrashing shared memory at scale, writes a Triton replacement that tiles correctly for the layer shapes at that model size, validates accuracy in the test harness, and pushes it to the kernels repo. By the end of the day, the training run that was taking four days now takes one.

About the Team

The AI Platform team builds Amazon’s neural network compression platform. We compress models using knowledge distillation, network restructuring, and advanced quantization to achieve 20‑100x compression while preserving model quality. Our platform packages these into automated pipelines that deploy to both custom edge silicon and GPU‑based cloud inference.

Basic Qualifications

• 3+ years of non‑internship professional software development experience.
• 2+ years of non‑internship design or architecture of new and existing systems.
• Experience with CUDA kernels or ML/low‑level kernels, or experience in developing and deploying LLMs in production on GPUs, Neuron, TPU or other AI acceleration hardware.
• Experience with programming languages such as Python, Java, C++.

Preferred Qualifications

• Bachelor’s degree in computer science or equivalent.
• 3+ years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations.
• Experience with GPU kernel optimization and GPGPU computing (CUDA, Triton, SYCL, or ROCm).
• Proficiency in low‑level performance optimization for GPUs.
• Understanding of GPU memory hierarchies and optimization strategies.
• Experience developing high‑performance libraries for ML or HPC applications.
• Knowledge of ML frameworks (PyTorch, TensorFlow) and their GPU backends.
• Experience implementing custom PyTorch operators (torch.autograd.Function, C++ extensions).
• Experience with parallel programming and optimization techniques.
• Background in neural network compression (quantization, pruning, knowledge distillation, low‑rank factorization).
• Knowledge of mixed‑precision training and inference (FP16, BF16, FP8, INT8, INT4).
• Experience with inference optimization (TensorRT, ONNX Runtime, vLLM, or similar).
• Familiarity with Transformer architectures, attention mechanisms, and their compute/memory profiles.
• Experience with AWS Trainium/Inferentia or the Neuron Kernel Interface (NKI).
• Experience with edge deployment, model compilation, or hardware‑aware optimization.

Compensation & Benefits

Base salary range: 114,800.00 – 191,800.00 CAD annually (CAN, BC, Vancouver). Compensation may also include sign‑on payments and restricted stock units (RSUs). Amazon offers comprehensive benefits including health insurance (medical, dental, vision, prescription, basic life & AD&D insurance), Registered Retirement Savings Plan (RRSP), Deferred Profit Sharing Plan (DPSP), paid time off, and other resources to improve health and well‑being.

Equal Opportunity Employer

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.