AI EvalsJobs
Hiring?
All roles / Hippocratic AI / Machine Learning Engineer

Machine Learning Engineer

Evals & Benchmarks2w ago
Source verified: this exact posting was present on Hippocratic AI’s careers feed on . View employer source →

What the posting asks for

Names
Python
Doctorate
Mentioned, without saying whether it is needed

Read out of the employer's own description. Absence means the posting does not say, not that the answer is no.

The role

About the role

We are building a recursive self-improvement system — a machine learning system that iteratively improves itself through feedback, evaluation, and automated learning loops. You will help build the engineering pipeline that keeps these loops fast, reliable, and trustworthy: the training and evaluation pipelines, the reward and feedback signals, and the safeguards that prevent a self-improving system from silently degrading or gaming its objectives.

This is an engineering-first role with deep reinforcement learning requirements. You should be equally comfortable writing robust production ML code and reasoning about reward design, credit assignment, and why feedback-driven systems become unstable.

What you'll do

  • Build and maintain the training, evaluation, and deployment loops at the core of the self-improvement system, with a strong emphasis on reproducibility and reliability.

  • Design and implement reward and feedback signals; investigate and mitigate reward hacking, specification gaming, and distribution drift.

  • Build evaluation harnesses and metrics before models — because a self-improving system is only as safe as its measurement of “better.”

  • Own data pipelines and automated data flywheels that feed the learning loop.

  • Debug subtle model-quality regressions and stabilize training and feedback loops that go non-stationary.

  • Collaborate with research and product to turn methods into robust, shippable systems.

What we're looking for

Must-Haves:

Strong MLE fundamentals (non-negotiable)

  • Excellent Python and clean, well-tested ML training code.

  • Solid grasp of data pipelines, distributed / large-scale training, and experiment tracking.

  • The instinct and skill to debug why a model silently got worse — not just why it crashed.

Hands-on experience with a feedback or learning loop (at least one).

  • Built or owned part of a feedback loop — a reward model, an evaluation harness, or the data pipeline for an RLHF/RLAIF or active-learning system.

  • Ran a retraining or continual-learning pipeline where a model consumed its own predictions or production data (e.g. ranking, recommendations, fraud, spam).

  • Fine-tuned LLMs with human or AI feedback, or built agentic evaluation harnesses.

Reinforcement learning foundations and curiosity.

  • Working knowledge of reward modeling, on-policy vs. off-policy tradeoffs, and credit assignment (does not need to be a research-level RL expert).

  • Has seen — or can reason clearly about — feedback-system failure modes: reward hacking, specification gaming, feedback loops amplifying errors.

  • Comfortable evaluating non-stationary systems (systems whose behavior and data distribution change over time).

Systems and evaluation instinct.

  • Builds the eval before the model; treats measurement as a first-class deliverable.

  • Has shipped an ML system into production and kept it healthy over time.

Strongest signal

Bonus, not required:

The ideal candidate has built or shipped a full system that improved from its own outputs or feedback end to end. This is rare at this level, so treat it as a standout differentiator rather than a filter. Examples:

  • RLHF / RLAIF pipelines

  • Self-play systems

  • Active-learning loops

  • Automated data flywheels

  • Agentic evaluation harnesses

Nice to Have:

  • PhD or MS in RL / ML paired with real production experience (either the science or the engineering half alone is fine if the other is strong).

  • Experience at a lab or company doing RLHF, agents, or large-scale ML infrastructure.

  • Familiarity with LLM fine-tuning, evaluation frameworks, or agent orchestration.

Why Join Hippocratic AI

Reinvent healthcare with AI that puts safety first. We’re building the world’s first healthcare‑only, safety‑focused LLM — a breakthrough platform designed to transform patient outcomes at a global scale. This is category creation.

Work with the people shaping the future. Hippocratic AI was co‑founded by CEO Munjal Shah and a team of physicians, hospital leaders, AI pioneers, and researchers from institutions like El Camino Health, Johns Hopkins, Washington University in St. Louis, Stanford, Google, Meta, Microsoft, and NVIDIA.

Backed by the world’s leading healthcare and AI investors. We recently raised a $126M Series C at a $3.5B valuation, led by Avenir Growth, bringing total funding to $404M with participation from CapitalG, General Catalyst, a16z, Kleiner Perkins, Premji Invest, UHS, Cincinnati Children’s, WellSpan Health, John Doerr, Rick Klausner, and others.

Build alongside the best in healthcare and AI. Join experts who’ve spent their careers improving care, advancing science, and building world‑changing technologies — ensuring our platform is powerful, trusted, and truly transformative.

Equal Opportunity

Hippocratic AI is an equal opportunity employer. We do not discriminate on the basis of race, color, religion, national origin, sex, age, disability, sexual orientation, gender identity or expression, genetic information, military or veteran status, or any other characteristic protected by applicable law. We are committed to building a team that reflects the patients we serve. We actively encourage applications from candidates of all backgrounds. If you require accommodations during the hiring process, please contact people@hippocraticai.com.

Please be aware of recruitment scams impersonating Hippocratic AI. All recruiting communication will come from @hippocraticai.com email addresses. We will never request payment or sensitive personal information during the hiring process.

Published by Hippocratic AI on their own careers page and reproduced here unedited. Read it at Hippocratic AI.

Apply at Hippocratic AI → Applications go directly to Hippocratic AI. This board does not sit in between, take a fee from you, or see your application.

What this listing does not tell you

Listed 16 days. Of the 114 evals and benchmarks roles this board has watched from listing to removal, 24% were gone from their employer's careers page by day 16, and the median came down after 56 days. That is a description of other listings that have already ended, not a prediction about this one: this board records when a listing disappears, never why, and a posting still up is not on a clock it can see.

Pay is not confirmed for this role. Check the employer’s posting for current compensation. Explore published bands from other roles →

Hippocratic AI has 4 roles open on this board, 2 of them in evals and benchmarks.

Get the weekly AI Evals Jobs briefNew roles and board updates, with published pay where available. This is the general weekly brief. Or browse them all now.

More roles like this

Matched by discipline, title, listed location and work arrangement.

Same discipline: Evals & Benchmarks · Shared listed location

Same discipline: Evals & Benchmarks · Shared listed location

Same discipline: Evals & Benchmarks · Shared listed location

Same discipline: Evals & Benchmarks · Shared listed location

Same discipline: Evals & Benchmarks · Shared listed location

Privacy · Terms