Distinguished Engineer, AI Engineering (Agents, Harnesses & Evals)

Datavant is seeking a highly experienced engineer to help shape how AI is applied in healthcare.

Our Life Sciences technology powers clinical research, real-world evidence generation, pharmacovigilance, and secure data collaboration. As a Distinguished Engineer, you will build intelligent systems on top of high-quality, privacy-preserving healthcare data-supporting pharmaceutical companies, biotech innovators, and research organizations in making better decisions.

In this role, you will design and deploy agentic systems, LLM harnesses, and evaluation workflows that integrate models, structured data, and production services into real-world applications. This is a senior individual contributor role for an engineer who combines deep technical judgment, hands-on execution, and broad organizational influence.

You will serve as a senior technical leader for AI engineering, defining architecture, establishing engineering patterns, and directly building critical systems. This role combines deep hands-on execution with broad technical influence across AI, data, and product.

You will operate with a high degree of autonomy and end-to-end ownership, leading work from concept through production. The focus is on building reliable, scalable AI systems and creating the harnesses, evaluation approaches, and engineering practices that make them effective in real-world healthcare applications.

• Set technical direction for agentic systems, harness engineering, and LLM evaluation
• Establish architectural patterns, engineering standards, and best practices for AI systems in production
• Guide complex technical decisions across AI, data, and product domains

• Design and implement agentic workflows including tool use, retrieval, and multi-step reasoning
• Build AI-driven applications that integrate LLMs with APIs, data pipelines, and analytics systems
• Develop reusable abstractions for context management, tool use, and orchestration

• Design and build LLM harnesses that structure how agents operate in production
• Create frameworks for prompt management, context injection, execution loops, and agent control flow
• Standardize patterns for integrating models with internal systems and external APIs

• Use and extend evaluation frameworks to measure correctness, reliability, and safety
• Define metrics for model performance within real workflows
• Build datasets and test cases to validate agent and system behavior over time

• Integrate AI capabilities into production backend services
• Design systems combining retrieval, structured data, and model reasoning
• Improve latency, cost, scalability, logging, tracing, and monitoring for AI-powered workflows
• Debug and improve multi-step system performance and failure modes

• Partner with Product, Data, and Science leaders to identify and deliver high-value AI opportunities
• Mentor engineers and raise the bar for AI engineering practices across the organization
• Help shape Datavant's long-term AI engineering direction through architecture and hands-on execution

• Practice compound engineering: treat each system as an opportunity to improve how future systems are built
• Work in iterative loops across humans and agents, moving from planning and implementation to evaluation, refinement, and codification
• Capture learnings from failures, evaluations, and edge cases, and encode them into prompts, tools, workflows, and reusable patterns
• Build durable assets such as agent patterns, harness components, and evaluation utilities that improve engineering velocity and system quality over time
• Operate with a high degree of autonomy and broad ownership, using AI tools and strong engineering judgment to deliver high-impact outcomes with speed and focus

• 10+ years of software engineering experience, ideally spanning backend, platform, or data-intensive systems
• Significant experience as a senior/staff/principal-level individual contributor
• Experience building and shipping applications with LLMs or agent frameworks
• Experience designing systems that integrate models, APIs, data systems, and production services
• Strong understanding of system design, testing, observability, and production reliability
• Track record of setting technical direction and influencing engineering outcomes across teams
• Strong hands-on engineering ability and comfort operating with a high degree of autonomy

• Experience building LLM harnesses or agent frameworks
• Experience using evaluation frameworks for LLM systems
• Familiarity with retrieval systems, vector databases, and RAG patterns
• Experience with cloud platforms such as AWS, GCP, or Azure
• Experience in healthcare or other regulated environments
• Experience defining technical standards in emerging areas with rapidly evolving tooling and practices

• Operate as a top-level individual contributor in an emerging area of healthcare technology
• Help define how AI systems are built, evaluated, and trusted in regulated environments
• Work on real-world applications that combine LLMs, healthcare data, and production infrastructure
• Shape Datavant's AI engineering direction through both architecture and direct execution