As of November 7, 2026, Level lists 247 live jobs that mention LLM evaluation. Of these, 177 are rated Builds AI, the top level, which means the work is building AI systems. Only 61 are remote, and 188 are senior or lead roles. Pay data is thin, so read the salary section carefully.
What happened
On November 4, 2026, Hightouch published "A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost". The team tested whether cheap decision models could grade identity resolution as well as frontier LLMs.
Both models reached 88.8% to 97.3% accuracy out of the box, according to the post. A hybrid that asks Claude Opus for help reached frontier accuracy at more than 4x lower cost. The post drew 417 points on Hacker News on November 6.
Someone has to build and maintain such graders, which is evaluation work.
How many LLM evaluation jobs are open?
Level lists 247 live jobs matching "LLM evaluation" as of November 7, 2026. The query matches the phrase anywhere in a posting, so many roles are general AI engineering jobs that list evaluation as one duty.
Capital One alone accounts for 64 of the 247, which is 26%. Without it, 183 roles remain, and 113 of them are Builds AI. Use 183 as the safer count.
By level, 177 roles are Builds AI and 63 are Works on AI. Only 7 roles fall into Uses AI or Little AI. The definitions are on the levels page.
Titles are a stricter test. Of the 247 roles, 23 have "eval" in the title. Examples are Evaluation Engineer at Mistral AI and AI Evaluation Lead at Fetch. Nine of the 23 are at Capital One.
Open roles right now
Who is hiring for it?
After Capital One, Amazon has 20 roles, NVIDIA has 10 and Thomson Reuters has 6. Mistral AI, Scale AI, Ramp, Workday, Glean Work, Turing and xAI each post a role with evaluation or benchmarks in the title.
Three titles say evaluation infrastructure: Inception Labs, Turing and xAI. Those are platform roles, where you build the system that runs the tests.
Which skills do these roles ask for?
The tools tagged most often on the 247 roles are PyTorch in 101 roles and Python in 95. Hugging Face appears under two tags, in 80 and 67 roles, so up to 147 roles combined.
Cloud and agent tooling follow. AWS is tagged in 61 roles, LangChain in 34 and vector databases in 34. OpenAI appears in 31 roles and LangGraph in 23.
For a candidate, that points to a familiar profile. You write Python, you know a deep learning framework, and you can wire a grader into a pipeline. For related paths, see software engineering AI jobs and the guide to LLM engineer jobs.
What do they pay, and can you do it remotely?
Salary is published on a minority of postings. Of the 247 roles, 66 list a USD range. The median of those range midpoints is $208,600. Half the roles pay more and half pay less.
That median covers every kind of role in the group, from junior to lead. Only 6 of the roles with "eval" in the title list a USD range, too few for a separate figure. Compare it with the AI engineer salary guide.
Remote work is the minority here. Level shows 61 of the 247 roles as remote, which is 25%. In the United States, 34 of 142 roles are remote. For a wider search, see remote AI jobs.
By country, 142 roles are in the United States. India has 27, the United Kingdom has 16, Canada has 10 and China has 9.
Can you start in evaluation early in your career?
Mostly no. Of the 247 roles, 152 are tagged senior and 36 lead, so 188 are senior or lead. Another 39 are mid-level.
Only 17 roles are internships or junior roles. Of these, 9 are rated Builds AI and 8 are rated Works on AI. One example is an Orange internship for a pentester who evaluates LLMs. If you are early in your career, filter for internships first.
Method
Source: the Level database, read through the public jobs API, which lists only live postings. Date: data read on November 7, 2026, at about 06:40 UTC. Counts change daily. Scope and filter: all live jobs matching the query "LLM evaluation", which returned 247 roles. The 23 title matches use the word "eval" in the title. The Google US results for "llm evaluation jobs" were read through DataForSEO on the same day. Levels: each posting is placed on one of four levels (Builds AI, Works on AI, Uses AI, Little AI), defined on the levels page. Limits: the query matches the phrase anywhere in a posting, Capital One's volume skews the totals, and salary is known for only 66 roles.
