AI Engineer V (Principal)

Overview

The AI Engineer V (Principal) is the most senior individual-contributor technical authority for artificial intelligence and machine learning within AssistRx Engineering. Reporting to the Vice President, Engineering, this role defines and drives the company’s AI technical strategy and leads the design, build, and deployment of production-grade AI solutions across the specialty medication access ecosystem, including large language model applications, predictive models, intelligent document processing, and agentic automation that improve speed, accuracy, and scale in patient onboarding, benefits verification, prior authorization, document intake, and patient and provider engagement.

The AI Engineer V (Principal) shapes AI direction at the enterprise level, solves the company’s most complex and ambiguous technical problems, and sets the architecture, evaluation standards, and responsible AI practices for all AI work at AssistRx. The role serves as a trusted advisor to engineering and executive leadership, represents AssistRx’s AI capabilities with clients, partners, and auditors, and develops the next generation of AI technical leaders. The Principal AI Engineer partners closely with Software Engineering, Product, Architecture, Data, Security, Compliance, and Client Services, and is accountable for the quality, performance, cost efficiency, and safe and compliant use of AI with protected health information across the enterprise.

Responsibilities

AI Strategy & Solution Design

• Define the multi-year AI technical strategy and roadmap in partnership with the Vice President, Engineering, Product, and Architecture, aligning AI investments to business value, feasibility, and risk
• Lead the design of the most complex enterprise-scale AI and machine learning solutions, including generative AI, predictive models, and intelligent document processing that span multiple platforms, products, and clients
• Architect large language model applications using retrieval augmented generation, prompt engineering, structured outputs, function and tool calling, and agent-based orchestration
• Set the patterns for agentic workflows, including task decomposition, tool and function calling, memory and state management, routing, clear boundaries, human review checkpoints, and fallback paths for high-impact decisions such as benefits verification and prior authorization
• Identify new AI opportunities and emerging technologies that create competitive advantage, and lead proofs of concept that move proven pilots to scale
• Determine the right approach for each problem, including prompting, retrieval augmentation, tool use, task-specific models, and conventional software or deterministic logic when a model is not the right answer
• Set the standard for design documentation of AI features, covering intended behavior, context and retrieval design, prompt and agent structure, evaluation approach, and fallback behavior

AI Foundations & Platform

• Own the enterprise AI reference architecture, platform strategy, and engineering standards, including model access, prompt and version management, and retrieval services, and ensure their adoption across all engineering teams
• Lead evaluation of commercial models, AI platforms, and vendor tools, and make build-versus-buy recommendations to executive leadership based on quality, cost, security, strategic fit, and time to value
• Guide vendor and cloud provider relationships from a technical perspective, including model selection, capacity, pricing, and risk
• Anticipate scaling, reliability, and cost challenges in AI systems, and lead architectural investments that address them before they affect clients

Model Development, Evaluation & Optimization

• Set organization-wide targets for accuracy, groundedness, hallucination rate, task completion, latency, and cost, and guide model selection, tuning, and evaluation against them
• Define the evaluation strategy, benchmark datasets, and automated test frameworks used across the organization to measure output quality and detect regressions before each release
• Guide optimization of retrieval pipelines, embeddings, chunking strategies, and inference performance to improve quality and reduce cost per transaction at scale
• Direct error analysis and human-in-the-loop review programs with operations and clinical subject matter experts, and ensure findings drive improvement of prompts, models, and guardrails
• Establish the experimentation approach, including A/B testing across prompts, retrieval configurations, model selection, and workflow design
• Define and enforce quality gates and regression thresholds so AI features must meet defined accuracy and safety criteria before release

Data Engineering & Integration

• Set the technical direction for the pipelines that prepare structured and unstructured healthcare data, such as enrollment forms, payer documents, and clinical notes, for training, retrieval, and inference
• Partner with Data Engineering and Architecture to define enterprise data contracts, feature sets, vector stores, and lineage for AI workloads
• Define enterprise standards for de-identification, masking, and data minimization when working with protected health information

Machine Learning Operations & Production Support

• Define enterprise standards for CI/CD, source control, containerization, and infrastructure-as-code practices for AI workloads
• Establish versioning, promotion, and rollback practices for prompts, agents, and model configurations, treating them as versioned artifacts alongside application code
• Set the observability strategy for accuracy drift, groundedness, task completion, token consumption, latency, cost, and reliability, including alerting and incident response practices
• Own the strategy for model and inference spend, applying caching, routing, and model right-sizing to keep AI solutions cost-effective at scale
• Serve as the final technical escalation point for critical production issues and client-reported defects in AI-enabled features, and drive systemic improvements that prevent recurrence

Responsible AI, Security, Privacy & Compliance

• Lead the company’s responsible AI approach and ensure solutions satisfy healthcare privacy, data security, and regulatory requirements, including HIPAA and HITRUST
• Partner with Security, Compliance, and Legal to address AI-specific risks such as prompt injection, data leakage, and insecure output handling, and to confirm vendor agreements (including Business Associate Agreements) are in place before PHI is processed
• Define guardrails, content filtering, human oversight, and audit trails appropriate to the risk profile of each use case
• Contribute to internal AI governance, and represent AssistRx’s AI practices in audits, client due diligence, and security reviews
• Track emerging AI regulation and industry guidance, and translate it into engineering policy and practice

Technical Leadership, Influence & Enablement

• Serve as a trusted technical advisor to the Vice President, Engineering and executive leadership on AI strategy, risk, investment, and capability
• Provide technical leadership across the engineering organization, influencing architecture and engineering direction at the enterprise level without direct authority
• Mentor and develop Staff, Senior, and mid-level engineers, build the AI technical bench, and raise AI fluency across engineering and the broader company
• Lead cross-functional technical initiatives and resolve complex technical disagreements across teams
• Work with Product Management, Client Services, and business leaders to identify high-value AI use cases and define clear success measures before build begins
• Communicate AI capabilities, limitations, risks, and results in plain language to technical and non-technical audiences, including executives, clients, and external partners
• Contribute to hiring strategy, interviewing, and role definition for AI and related engineering roles
• Advance AI-assisted and agentic development practices across the engineering organization, including specification-driven workflows and agentic coding tools, and share what works with engineering peers
• Participate in agile team activities as needed, including sprint reviews and retrospectives, and conduct regular self-guided study to stay at the forefront of a fast-moving AI landscape

Qualifications

• Bachelor’s degree in Computer Science, Engineering, Data Science, or related technical field (or equivalent experience); Master’s degree preferred
• 10+ years of software engineering experience, including 6+ years building and deploying AI or machine learning solutions in production
• Track record of leading the strategy and delivery of enterprise-scale AI solutions used by real customers or operations teams, with significant, measurable business outcomes
• Expert programming skills in Python, along with experience in .NET/C#/Java or a comparable enterprise language, including building backend services and REST APIs (for example FastAPI, Flask, or ASP.NET)
• Deep, hands‑on expertise with large language models and generative AI patterns, including retrieval augmented generation, prompt engineering, embeddings, vector and hybrid search, reranking, structured outputs, function and tool calling, context window management, and model evaluation
• Extensive experience architecting and operating production AI workloads on cloud-based AI and machine learning platforms (AWS preferred, including services such as Amazon Bedrock and Amazon SageMaker; equivalent Azure experience with Azure AI Foundry, Azure OpenAI Service, or Azure Machine Learning also acceptable)
• Expert knowledge of APIs, microservices, distributed systems, SQL, and modern development practices including Git, CI/CD, containerization, and automated testing
• Demonstrated ability to deliver in an environment where data privacy, security, and regulatory requirements shape solution design
• Demonstrated enterprise-level technical leadership, including setting technical strategy, driving architectural decisions across teams, and developing other technical leaders
• Ability to work independently on highly ambiguous, high-impact problems, set direction, and make sound tradeoffs with limited precedent
• Hands‑on experience building software with agentic coding tools such as Claude Code, GitHub Copilot agents, Codex, or Cursor, including specification-driven workflows and multi‑step agent execution
• Deep practical understanding of how large language models behave in production, including hallucination, prompt sensitivity, non‑determinism, and latency and cost tradeoffs
• Expert knowledge of evaluation methods for LLM systems, including automated scoring, LLM‑as‑judge techniques, and human review workflows
• Exceptional written and verbal communication skills, with the ability to influence executives, clients, and engineers and to represent the company’s AI approach externally

Preferred Qualifications

• Master’s or doctoral degree in Computer Science, Machine Learning, Data Science, or related field
• Experience in healthcare technology, specialty pharmacy, HUB services, ePA, or EHR/EMR integrations
• Experience with intelligent document processing, OCR, and data extraction from unstructured enrollment or clinical documents
• Applied machine learning or data science background, including experiment design, model evaluation, and error analysis
• Experience with Docker, Kubernetes, or infrastructure-as-code at scale
• Experience with agent and orchestration frameworks such as LangChain, LangGraph, LlamaIndex, Semantic Kernel, Strands, or Model Context Protocol
• Experience with machine learning operations tooling, model registries, feature stores, and LLM observability or evaluation platforms
• Experience with model fine‑tuning or smaller task‑specific models for classification and extraction
• Familiarity with AI security guidance such as the OWASP Top 10 for LLM Applications
• Familiarity with HIPAA, HITRUST, or responsible AI frameworks such as the NIST AI Risk Management Framework
• Prior experience establishing an AI function or serving as the senior AI technical leader for an organization
• Experience representing technical capabilities in client due diligence, security reviews, or external audits

COMPETENCIES

• Technical Depth
• Strategic Thinking
• Analytical Rigor & Problem Solving
• Experimentation & Measurable Outcomes
• Accuracy & Quality
• Sound Judgment & Risk Awareness
• Ownership & Accountability
• Pragmatism & Bias for Delivery
• Communication
• Collaboration & Influence
• Technical Leadership & Talent Development
• Curiosity & Learning Agility
• Adaptability
• Initiative & Innovation
• Ethics & Integrity

Pay Range

USD $162,239.00 - USD $204,048.00 /Yr.