What programming language do AI evals jobs require?

Python is named in 531 of 996 parsed postings, far ahead of anything else.

What eval frameworks do employers actually name?

Named tooling specific to evaluation appears rarely: Inspect in 31, Label Studio in 20, LangSmith in 13, Weights & Biases in 2, Braintrust in 1, Ragas in 1. Most postings describe the work without naming a framework.

Which tools go with the highest paying roles?

Postings naming Rust have a median published midpoint of $311k; postings naming TypeScript sit at $245k. That gap tracks the kind of work rather than the tool itself.

Recomputed every morning · 2026-10-20

What tools do these jobs actually ask for?

Name: Tools named in AI evaluation and post-training job postings
Creator: AI Evals Jobs
License: https://creativecommons.org/licenses/by/4.0/

Counted from 996 postings whose requirements this board has parsed. Python leads at 531, and the interesting part is how thin the eval-specific tail is.

Named in the posting

Only tools appearing in five or more postings are listed. A median is shown only where five or more published bands sit behind it, from at least three different employers, with none of them supplying half. The employer column is there so you can see why a row is blank.

Tool	Postings	Share	Median pay	Bands	Employers
Python	531	53%	$284k	301	67
PyTorch	262	26%	$281k	137	48
SQL	109	11%	$265k	69	23
JAX	102	10%	$310k	58	23
Kubernetes	94	9%	$298k	56	25
TensorFlow	66	7%	$281k	33	13
Databricks	58	6%	—	50	8
TypeScript	49	5%	$245k	35	15
Spark	42	4%	$310k	20	11
Rust	41	4%	$311k	30	12
vLLM	39	4%	$288k	23	15
Inspect	31	3%	—	23	11
Label Studio	20	2%	—	6	1
LangSmith	13	1%	—	12	1
veRL	6	1%	—	4	4

The spread is about the work, not the tool

Among tools with enough employers behind them to compare, postings naming Rust carry a median midpoint of $311k and postings naming TypeScript sit at $245k. Nobody is paid more for knowing one library over another. What that gap separates is model and infrastructure work from data and application work, and those are different jobs that happen to share a board.

Where a tool is really one company

These have enough published bands to compute a median and only one employer behind them, which means the figure would describe that company rather than the tool. In two cases the employer is the company that makes the tool, which is the clearest possible version of the problem.

Tool	Postings	Bands	All from	Their median
LangSmith	13	12	LangChain	$209k
Label Studio	20	6	HumanSignal	$93k

Read those medians as that employer's pay and not as what naming the tool implies. They are excluded from the table above for the same reason.

Method. Counted from the requirements text of 996 postings, which is 87% of the 1147 live roles; the rest did not carry enough description text to parse. A tool counts once per posting that names it. Pay figures are the median midpoint of the published annual USD bands among postings naming that tool, shown only where five or more such bands exist, from three or more distinct employers, with no single employer supplying half or more. That last test is why some rows are blank: a median drawn from one company describes that company. This list is a fixed vocabulary rather than open extraction, so a tool absent from it is unmeasured rather than unmentioned.

What changed on this board this weekWhat changed since last Thursday: new roles, roles that closed, and pay moves.

Browse the open roles →