Level

Duolingo

Senior Engineering Manager, AI Platform

Duolingo is hiring a Senior Engineering Manager, AI Platform in Pittsburgh, United States. It pays $204k-$306k a year and Level rates it ; you can apply on Duolingo.

Level's read on this job

  • Builds AI ●●●● is how Level rates this role: the job is building AI systems.
  • Tools and skills the posting cites include AI evaluation, engineering management and AI tooling.
  • Level classifies it as a senior role.
  • Level lists 60 other open roles at Duolingo.

AI in this role

Senior Engineering Manager to lead the AI Tooling and Ecosystem Team, building infrastructure and workflows for AI model evaluation and datasets.

ai-evaluationengineering-managementai-toolingmlopsdataset-management

Our mission at Duolingo is to develop the best education in the world and make it universally available. It’s a big mission, and that’s where you come in!

At Duolingo, you’ll join a team that cares about finding innovative solutions to complex technical problems, running countless experiments (300+ at a time!) with our massive user base to make data-driven decisions, and educating our users and employees alike. You’ll have limitless learning opportunities, mentorship and collaboration with world-class minds, and a variety of projects with large scopes — while doing work that’s both fun and meaningful. 

Join our life-changing mission to develop education for our half a billion (and growing!) learners around the world.

About the role...

As the manager and team lead of the new AI Tooling and Ecosystem Team, you will lead the development of the suite of tools and services that power Duolingo's use of AI in the product. You'll lead a team of 5-6 engineers to design systems for managing datasets, evaluating the performance of AI models, and building AI workflows. You will help shape how Duolingo builds features using AI, enabling teams to easily define quality metrics, combine human and AI feedback, and compare model performance. We're looking for an engineering manager who has experience building AI products or infrastructure and is excited to play a central role in shaping the future of AI at Duolingo!

🧠  You will...

  • Drive development of tools and services that empower Duolingo's product teams to easily and effectively build AI-powered features and workflows.
  • Manage our shared systems and workflows for AI-related datasets, prompts, and evaluations (both human and LLM-as-a-judge).
  • Build processes that allow teams to define success criteria for their AI use-case, identify the most appropriate models, and construct feedback loops for prompt optimization.
  • Streamline common AI development flows so that teams with varying levels of AI expertise can build out AI systems quickly.
  • Collaborate closely with the AI Core Infrastructure Team to ensure that AI workflows are reliable, measurable, and cost efficient.
  • Partner with AI, Product, Data Science, and Platform leaders to build out new AI tooling that supports common use-cases.
  • Own multi‑quarter roadmaps and cross‑functional alignment that increase AI feature velocity and measurable product impact across Duolingo.

✅  You have...

  • Experience leading, managing, and building a team of software and/or AI engineers, with demonstrated ability to hire, develop, and retain talent.
  • A track record of building AI-related infrastructure or user-facing products powered by AI.
  • Familiarity with AI evaluation methods, especially approaches that combine human and LLM-as-a-judge evaluations.
  • Strong cross‑functional leadership and communication skills; proven ability to align AI, product, data science, and platform partners on strategy and execution.
  • Comfort leading through ambiguity and shaping new ways of working in a fast‑evolving AI landscape.
  • A Bachelor's degree in Computer Science or a related technical field
  • The ability to work from or relocate to Pittsburgh, PA or New York, NY.

⭐  Exceptional candidates will have...

  • Hands‑on experience building evaluation systems and human‑in‑the‑loop data pipelines at scale.
  • A track record of developing internal tools on a platform or infrastructure team, providing tools and services via easy-to-use APIs, MCPs, and web GUIs.
  • Experience navigating build vs. buy decisions and shaping long-term roadmaps based on the needs of internal customers.

The offered salary is dependent upon several factors, including work experience, skills, and internal peer comparisons. The posted range is subject to change in the future. For this role, base salary is supplemented by equity compensation. We encourage you to talk with your recruiter for more information related to compensation for this role!

Salary Range: $204,000—$306,000 USD

Benefits: Take a peek at how we care for our employees' holistic well-being with our benefits here.

Job Alerts: Sign up for job alerts here.

Accommodations: We will do everything we can within reason to make sure that your interview takes place in an environment that fairly and accurately assesses your skills. If you need assistance or accommodation, please contact accommodations@duolingo.com.

Equal Employment Opportunity: Duolingo is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.

Fraud Warning: Unfortunately, there is a rise in scammers pretending to be real Duolingo employees. Duolingo and our employees will never ask for your Social Security number, bank details, or passport info, and we’ll never ask you to deposit a check, purchase equipment, or exchange money during the interview process. Real Duolingo employees always use an email that ends in @duolingo.com or @recruiting.duolingo.com. Stay alert and double-check these details before sharing any information.

By applying for this position your data will be processed as per the Duolingo Applicant Privacy Notice.

How we rate this

Senior Engineering Manager, AI Platform at Duolingo rates 90 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.

Classification

Builds AI. The job is building AI systems.

  1. ●●●● Builds AI80 to 100
  2. ●●●○ Works on AI60 to 79
  3. ●●○○ Uses AI40 to 59
  4. ●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.

Prepare for this job

Built from this posting and from similar jobs on Level.

Skills and AI tools this role asks for

AI EvaluationEngineering ManagementAI ToolingMLOpsDataset Management

Questions you could be asked

  1. How do you decide that one model's output is better than another's for a given task?
  2. Tell me about a project where engineering management was part of your work. What did you do?
  3. Tell me about a project where ai tooling was part of your work. What did you do?
  4. Tell me about a project where mlops was part of your work. What did you do?
  5. Tell me about a project where dataset management was part of your work. What did you do?

Adapt your resume

  • List these exact terms on your resume: AI Evaluation, Engineering Management, AI Tooling, MLOps, and Dataset Management. An applicant tracking system matches the wording, not the idea.
  • Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
  • Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.

Want an expert to read your CV for this job?

Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.

Get new software engineering jobs (Builds AI ●●●●) by email

One email a week with the new software engineering jobs (Builds AI ●●●●), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free. One email a week. Unsubscribe in one click.

Similar roles

Software Engineering roles that build AI, at other companies.

What kind of AI work fits you?

Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.

Find my next step

More jobs at Duolingo

Related searches

Same AI level

Jobs by city