Ai Systems Engineer Rag Evaluation

Shape reliable AI systems through retrieval, RAG, evaluation, and monitoring for an energy platform supporting complex portfolio decisions. Use Python or Node.Js daily.As a mid-to-senior Applied AI Engineer, you will improve how AI quality, cost, and latency are measured in production. You will build evaluation sets, automated graders, human review processes, and monitoring. The platform uses Ruby on Rails, Node.Js/TypeScript, Python, PostgreSQL, Snowflake, and AWS.Key Responsibilities:
Build evals tracking quality, cost, and latencyImprove systems using evaluation resultsOwn retrieval and RAG across company dataDefine measurable success for AI applicationsSupport customer-facing AI feature refinementRequirements:
Production experience delivering LLM features or agentsPython preferred, or Node.Js/TypeScript production experienceExperience with representative eval sets and automated gradersHuman review and production AI monitoring experienceEnterprise cloud experience, preferably AWS BedrockBring evaluation, monitoring, retrieval, and RAG expertise to strengthen production AI quality across energy products.
#J-18808-Ljbffr