Skip to content

OpenAI

Model Policy Manager, Emerging AI Misuse

OpenAI is hiring a Model Policy Manager, Emerging AI Misuse in San Francisco, United States. It pays $207k-$335k a year and Level rates it ; you can apply on OpenAI.

Level's read on this job

  • Works on AI ●●●○ is how Level rates this role: the daily work is on AI products, without building the model.
  • The posting allows some remote work.
  • Tools and skills the posting cites include AI agents, AI research and threat modeling.
  • Level lists 850 other open roles at OpenAI.
  • The role was posted in the last few days.

AI in this role

Build threat models and safety policies to prevent emerging misuse and abuse of advanced AI models.

ai-agentsai-researchthreat-modelingpolicy-developmentsafety-evaluationrisk-assessment

About the Team

The Model Policy team within OpenAI’s Safety Systems organization defines how increasingly capable AI models should behave in real-world settings. We turn research on emerging risks into model behavior policies, evaluations, and safeguards.

About the Role

As AI systems become more capable, the same advances that expand their legitimate uses can also lower barriers to harm. Emerging misuse risks range from AI assistance with conventional weapons development and the misuse of AI-enabled robotics to fraud, impersonation, and other forms of abuse that people can carry out at scale using AI agents. You’ll build threat models for how people could misuse AI in digital and physical settings, assess what access to AI changes in practice, and use that evidence to define how models should respond.

This role is based in San Francisco. We offer relocation support and work from the office three days a week, with optional work from home on Thursdays and Fridays.

In this role, you will:

  • Build and update threat models as capabilities evolve. Assess what people could plausibly do, what expertise and resources they would need, and how AI changes those barriers.

  • Define when models should help, limit assistance, or refuse in situations where the same capability has legitimate and harmful uses.

  • Turn expert safety judgment into instructions, training examples, and evaluations that teach models when to help, when to limit assistance, and when to refuse.

  • Turn threat models into evaluation scenarios and grading guidance. Analyze model outputs, red-team findings, and deployment data to identify where models enable misuse or block legitimate work.

  • Partner with research, engineering, security, and product teams to apply findings to model training, deployment decisions, and monitoring.

You might thrive in this role if you:

  • Have hands-on experience building, testing, or assessing dual-use technologies, or a track record of threat modeling grounded in how people use and misuse real systems.

  • Demonstrate deep practical experience in one relevant domain and the judgment to know when to bring in experts from others.

  • Are able to assess whether a capability demonstrated in a test materially changes what someone could do outside it.

  • Can translate technical evidence and uncertainty into clear policy boundaries and measurable evaluation criteria.

  • Have experience analyzing gold sets, evaluation results, and training data to spot inconsistent human labels, assess how annotator disagreement affects measured performance, and decide what to investigate next.

  • Are comfortable working hands-on with model data and evaluation results, including inspecting examples, analyzing failure patterns, assessing data quality, and distinguishing policy failures from grader, model, or system failures.

  • Use empirical evidence to develop and refine safety policies and safeguards.

  • Work well across disciplines and revises their judgment as capabilities and evidence change.

  • Enjoy fast-paced, collaborative research environments where priorities shift as models, evidence, and risks change.

Relevant Publications

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristic.

For additional information, please see OpenAI’s Affirmative Action and Equal Employment Opportunity Policy Statement.

Background checks for applicants will be administered in accordance with applicable law, and qualified applicants with arrest or conviction records will be considered for employment consistent with those laws, including the San Francisco Fair Chance Ordinance, the Los Angeles County Fair Chance Ordinance for Employers, and the California Fair Chance Act, for US-based candidates. For unincorporated Los Angeles County workers: we reasonably believe that criminal history may have a direct, adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: protect computer hardware entrusted to you from theft, loss or damage; return all computer hardware in your possession (including the data contained therein) upon termination of employment or end of assignment; and maintain the confidentiality of proprietary, confidential, and non-public information. In addition, job duties require access to secure and protected information technology systems and related data security obligations.

To notify OpenAI that you believe this job posting is non-compliant, please submit a report through this form. No response will be provided to inquiries unrelated to job posting compliance.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.

OpenAI Global Applicant Privacy Policy

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

How we rate this

Model Policy Manager, Emerging AI Misuse at OpenAI rates 75 out of 100 for how much of the daily work is AI. That makes it Works on AI (AI Level 3 of 4). The level is about AI in the job, not seniority.

Classification

Works on AI. The daily work is on AI products, without building the model.

  1. ●●●● Builds AI80 to 100
  2. ●●●○ Works on AI60 to 79
  3. ●●○○ Uses AI40 to 59
  4. ●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.

Prepare for this job

Built from this posting and from similar jobs on Level.

Skills and AI tools this role asks for

AI agentsAI ResearchThreat ModelingPolicy DevelopmentSafety EvaluationRisk Assessment

Questions you could be asked

  1. How do you decide when an AI agent can act on its own versus asking for approval first?
  2. Tell me about a research question you investigated. What did you find?
  3. Tell me about a project where threat modeling was part of your work. What did you do?
  4. Tell me about a project where policy development was part of your work. What did you do?
  5. Tell me about a project where safety evaluation was part of your work. What did you do?

Adapt your resume

  • List these exact terms on your resume: AI agents, AI Research, Threat Modeling, Policy Development, and Safety Evaluation. An applicant tracking system matches the wording, not the idea.
  • Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
  • Show where AI is part of your daily process, not a one-off project. This role expects it to be a running habit.

Get new remote AI jobs (Works on AI ●●●○ or higher) by email

One email a week, each job rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free.

Similar roles

Other roles that work on AI, at other companies.

What kind of AI work fits you?

Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.

Find my next step

More jobs at OpenAI

Related searches

Same AI level

Jobs by city