Lead Research Scientist/Engineer (Embedded Evaluations)

Application deadline: We are conducting interviews actively and aim to fill this role as soon as we find someone suitable.

ABOUT THE OPPORTUNITY

The OpenAI/Hugging Face incident may be the biggest warning shot we get before catastrophic loss-of-control. In its aftermath, Anthropic and OpenAI publicly committed to giving embedded evaluators an unprecedented level of access.

Embedded evaluations are the best existing method to find out whether alignment is on track. With OpenAI predicting an automated researcher by March 2028, we don't have time to start with an MVP embedded evaluation and iteratively improve it. The first embedded evaluations will set the standard and we have to get it right. 

Apollo has taken steps toward becoming an embedded evaluator at frontier labs. We want to hire experienced senior researchers with a strong track record of getting things done to execute our embedded evaluation plans.

KEY RESPONSIBILITIES

If embedded evaluations happen, you will be on one of the following teams:

On-site embedded evaluators: You will spend weeks or months embedded at a frontier AI lab, where you will conduct interviews, review the lab’s evidence, gather new evidence, and write a report. You will execute a playbook quickly and adapt to unforeseen circumstances. We hold our embedded evaluators to the highest standards, so you will likely have to first prove yourself within Apollo before being sent into an embedded campaign. That is, you will probably start on the applied research team to gain context on embedded evaluations. For this team, we prefer candidates with the right to work in the US who can be based in San Francisco.

Applied researchers: You will be the force-multiplier behind embedded evaluations. You will prepare on-site evaluators before they are embedded and support them while they are embedded. You will do the conceptual work to determine what evidence would support a safety case, and you will write the playbook for on-site evaluators to optimize how fast they can gather evidence. You will develop tooling for embedded evaluators, both ahead-of-time and urgently on demand. You will be the bridge connecting fundamental research and the on-site evaluators. For example, you might convert the methodology of “Measuring Reward-Seeking via Contrastive Belief Updates” into turnkey code and procedures. This team can work from any of our offices. 

Both are high-stakes teams. On-site embedded evaluators should be exceptionally strong at quick execution under high pressure, and very strong at planning and research. Applied researchers should be exceptionally strong at planning and research, and very strong at quick execution under high pressure. For both teams, we are excited for experienced candidates who have owned ambitious projects end-to-end under tight deadlines. 

KEY REQUIREMENTS

• We don’t require a formal background or industry experience and welcome self-taught candidates.

• Data Analysis & Pattern Recognition: You can extract signal from large, messy datasets. You're comfortable with quantitative analysis and know when qualitative assessment is more appropriate. You can identify anomalies and unexpected model behaviors.

• Writing for high-stakes audiences: You have experience writing about qualitative and quantitative findings for different stakeholders. You can take a pile of unorganized technical information and communicate it effectively to lab safety teams, lab leadership, governments, or the public.

• AI power-user: You’re capable of using AI to accelerate your work, technical or otherwise. You have experience using different models, know which ones to use for which tasks, when not to use AI, and always experiment with new AI workflows.

• Expertise in scheming-related topics: You have a strong conceptual understanding of ideas like metagaming, evaluation awareness, deception, and reward-seeking. You are familiar with classic literature like Is Power-Seeking AI an Existential Risk? and modern research. 

• Hands-on experience with frontier RL: Labs' post-training pipelines are likely far more complicated than open-source versions like Olmo 3. Embedded evaluators need to triage this complexity to reach a verdict on safety claims such as “Training did not incentivize goals or drives that would motivate scheming”. Your deep knowledge of frontier RL will accelerate embedded evaluations.  

• Project management experience: You have owned an ambitious project end-to-end under a hard deadline. You have previously managed strong ICs and can keep research projects on schedule despite unexpected delays or surprising experiment results. On-site embedded evaluators will need to keep months-long evaluations on track, and applied researchers will need to make plans that are robust to changing circumstances and short turnarounds. 

• Professionalism: Your professional conduct is unimpeachable. You know when to tactfully resolve disagreements with partners and when to stand firm in negotiations, and you are excellent at both of those. Your previous boss trusted you to autonomously handle all interactions with partners for long periods. 

• Mission commitment: Apollo estimates that RSI will happen soon, and we are extremely concerned that AGI will catastrophically not be aligned with human values. Hence we have an “intense, but in a good way” culture. We will ask a lot from you, but in return you will personally have a large impact on loss-of-control risk. 

NICE TO HAVES

• Software engineering skills: Strong software engineering skills correlate highly with effective execution, even in an era of AI agents. Our entire stack uses Python.

• Prior embedded expertise: You have previous experience in professional services like auditing or consulting. You’ve learned lessons from working on-site with clients that will generalize to working with frontier AI labs.

We want to emphasize that people who feel they don’t fulfill all of these characteristics but think they would be a good fit for the position, nonetheless, are strongly encouraged to apply. We believe that excellent candidates can come from a variety of backgrounds and are excited to give you opportunities to shine. 

BENEFITS

• This role offers a market-competitive salary, equity, and competitive benefits.

• Depending on your experience, you may be offered a Lead or individual contributor position. 

• Salary: 

• Recognised experts who can lead projects of significant scope: USD 350,000 - USD 590,000 or £230,000 - £395,000  

• Strong individual contributors: USD 245,000 - USD 405,000 or £160,000 - £270,000

• Unlimited vacation

• Unlimited sick leave

• Up to 6 months of paid parental leave

• Comprehensive health, dental and vision insurance

• Retirement savings with competitive employer matching (e.g. 401(k) for US employees)

• Lunch, dinner, and snacks are provided for all employees on workdays

• Paid work trips, including staff retreats, business trips, and relevant conferences

• A yearly $1,000 (USD) professional development budget

• Relocation support and visa fees (if applicable). We are able to sponsor visas to the UK and US.

LOGISTICS

• Time Allocation: Full-time

• Location: This is an in-person role working out of our London or San Francisco office. 

• We prefer on-site embedded evaluators to be permanently located in San Francisco. 

• Applied researchers can work from any of our offices. We offer flexible working hours and WFH arrangements. Remote candidates will only be considered if they are exceptional and can travel at least 1 week/month to either office.

• Visa sponsorship: We sponsor visas in both the UK and US. Sponsorship isn't guaranteed for every role or candidate, but if we make you an offer, we'll work with you to find the right visa route

ABOUT APOLLO RESEARCH

The rapid rise in AI capabilities offers tremendous opportunities, but also presents significant risks. At Apollo Research, we’re primarily concerned with risks from Loss of Control, i.e. risks coming from the model itself rather than e.g. humans misusing the AI. We’re particularly concerned with deceptive alignment / scheming, a phenomenon where a model appears to be aligned but is, in fact, misaligned and capable of evading human oversight.

We work on the detection of scheming (e.g., building evaluations and novel evaluation techniques), the science of scheming (e.g., model organisms and the study of scaling trends), and scheming mitigations (e.g., control). We closely work with multiple frontier AI companies, e.g. to test their models before deployment and collaborate on fundamental research.

At Apollo, we aim for a culture that emphasizes truth-seeking, being goal-oriented, giving and receiving constructive feedback, and being friendly and helpful. If you’re interested in more details about what it’s like working at Apollo, you can find more information here.

ABOUT THE TEAM

Alex Meinke leads the science of scheming team’s research agenda with guidance from Marius Hobbhahn, though team members lead individual projects. You will mostly work with the science of scheming team as well as our team of software engineers, but you will likely sometimes interact with the governance team to translate technical knowledge into concrete recommendations. You can find our full team here.

Equality Statement: Apollo Research is an Equal Opportunity Employer. We value diversity and are committed to providing equal opportunities to all, regardless of age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex, or sexual orientation.

 How to apply: Please complete the application form with your CV. The provision of a cover letter is optional but not necessary. Please also feel free to share links to relevant work samples.

About the interview process: Our multi-stage process includes a screening interview, 3 technical interviews, and a final interview with Marius (CEO). The technical interviews will be closely related to tasks the candidate would do on the job. There are no LeetCode-style general coding interviews. If you want to prepare for the interviews, we suggest (a) thinking about what embedded evaluators should investigate, (b) building a tool that would be useful for embedded evaluators and running it on suitable proxy data, and/or (c) writing a report based on mock findings. 

Your Privacy and Fairness in Our Recruitment Process: We are committed to protecting your data, ensuring fairness, and adhering to workplace fairness principles in our recruitment process. To enhance hiring efficiency, we use AI-powered tools to assist with tasks such as resume screening. These tools are designed and deployed in compliance with internationally recognized AI governance frameworks. Your personal data is handled securely and transparently. We adopt a human-centred approach: all resumes are screened by a human and final hiring decisions are made by our team. If you have questions about how your data is processed or wish to report concerns about fairness, please contact us at info@apolloresearch.ai.