Senior Technical Program Manager, RL Gyms, Frontier AI RL Gym Assets

AI in this role

ai-agentsai-evaluation

We are seeking an experienced Senior Technical Program Manager to design and manage the program that drives execution of our Reinforcement Learning (RL) Gym development initiative. RL Gyms are standardized evaluation environments that measure AI agent performance across enterprise domains. In this role, you will define program structure, build mechanisms, and drive accountability to deliver RL Gyms at scale — partnering with scientists, engineers, and domain experts across multiple organizations.

This is a high-ownership role where you will shape how the program operates end-to-end — from partner engagement and requirements definition through certification and delivery — rather than building gyms yourself.

Key job responsibilities
Program Execution: Own end-to-end delivery of RL Gym development programs — from requirements gathering and design through implementation, certification, and launch readiness

Cross-functional Coordination: Drive alignment across engineering, science, and domain teams to define gym specifications, collection standards, and certification criteria

Technical Depth: Understand gym harness architecture, evaluation frameworks, and agent benchmarking methodologies (e.g., SWE-bench style evaluations) to make informed tradeoffs and unblock teams

Stakeholder Management: Partner with domain stakeholders across the company to identify high-value workflows suitable for gym development

Certification & Quality: Define and enforce certification standards for RL Gyms, ensuring they meet bar for agent evaluation fidelity and reproducibility

Capacity Planning: Track gym development targets, manage pipeline throughput, and forecast delivery timelines across multiple concurrent gym workstreams

Tooling & Process: Drive improvements to gym tracking tools, development workflows, and operational processes (e.g., DPM tracking, standup cadences)

Risk Management: Identify blockers early, escalate appropriately, and develop mitigation plans to keep programs on track

About the team
The RL Gym Program develops standardized evaluation environments ("gyms") that test AI agents on realistic enterprise tasks. Our gyms span multiple business domains and are used to certify agent readiness for production deployment. We operate at the frontier of AI evaluation, working closely with model teams, infrastructure teams, and business stakeholders to define what "good" looks like for AI agents.

Basic qualifications

- 5+ years of technical product or program management experience
- 7+ years of working directly with engineering teams experience
- 3+ years of software development experience
- 5+ years of technical program management working directly with software engineering teams experience
- Experience managing programs across cross functional teams, building processes and coordinating release schedules
- Bachelor's degree in engineering, computer science or equivalent

Preferred qualifications

- 5+ years of project management disciplines including scope, schedule, budget, quality, along with risk and critical path management experience
- Experience managing projects across cross functional teams, building sustainable processes and coordinating release schedules
- Experience defining KPI's/SLA's used to drive multi-million dollar businesses and reporting to senior leadership
- Experience in creating fully automated solutions

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.

USA, WA, BELLEVUE - 148,700.00 - 201,200.00 USD annually

How we score this

Senior Technical Program Manager, RL Gyms, Frontier AI RL Gym Assets at Amazon scores 27 out of 100 on AI centrality, which makes it AI Level 1 of 4 (Little AI) on this board. The level measures how much of the work is AI, not seniority.

Classification

AI Level 1. The work itself involves no AI, or AI only appears as scenery, such as a company tagline.

AI Level 480 to 100
AI Level 360 to 79
AI Level 240 to 59
AI Level 10 to 39

Bands come from how often the tools, models and workflows of the role are named in the posting itself. Open the description and count.

Prepare for this job

A free preview built only from this posting: what it asks for, what you could be asked in an interview, and how to adjust your resume.

Skills and AI tools this role asks for

AI AgentsAI Evaluation

Questions you could be asked

How do you decide when an AI agent can act on its own versus asking for approval first?
How do you decide that one model's output is better than another's for a given task?

Adapt your resume

List these exact terms on your resume: AI Agents and AI Evaluation. An applicant tracking system matches the wording, not the idea.
Attach one line of real, concrete experience to at least one of them — a tool named with nothing behind it rarely survives a human read.

Want your resume actually rewritten for this job?

The free preview above is everything we have today. A full resume rewrite is not live yet and has no price set. Join the waitlist and we will email you if we open it.