Level

LangChainPosted 1w ago

Software Engineering Manager, AI Observability & Evals Platform

Software Engineering Manager, AI Observability & Evals Platform at LangChain scores 85 out of 100 on AI centrality, which makes it AI Level 4 of 4 (Builds AI) on this board. The level measures how much of the work is AI, not seniority.

Boston, MAleadFullTime$200k-$240k

AI in this role

Lead an engineering team building LangSmith, an observability and evaluation platform for LLM applications and AI agents.

langgraphclaylangsmith
ai-agentsmanagementobservabilityevaluation
About Us

At LangChain, our mission is to make intelligent agents ubiquitous. We build the foundation for agent engineering in the real world, helping developers move from prototypes to production-ready AI agents that teams can rely on. We began as widely adopted open-source tools and have grown to also offer a platform for building, evaluating, deploying, and operating agents at scale.

With $125M raised at Series B from IVP, Sequoia, Benchmark, CapitalG, and Sapphire Ventures, we’re at a stage where we’re continuing to develop new products, growth is accelerating, and all team members have meaningful impact on what we build and how we work together. LangChain is a place where your contributions can shape how this technology shows up in the real world.

Today, our platform includes LangSmith (Observability, Evaluation, Deployment, Fleet, and Sandboxes), our open source frameworks (LangChain, LangGraph, and Deep Agents), and the newly launched LangSmith Engine for autonomous agent improvement. We have 100M+ monthly open source downloads, 6,000+ active LangSmith customers, and 5 of the Fortune 10 use LangSmith in production (+ 35% of the Fortune 500 overall), including teams at Klarna, Clay, Coinbase, Workday, Lyft, Cloudflare, Harvey, Rippling, Vanta, LinkedIn, Monday.com, Nvidia, and Bridgewater.

About the Team

The LangSmith team owns and builds LangChain's core platform for observability, evaluation, and production reliability of AI systems. From tracing and annotation to run rules, evaluations, and beyond, this team owns LangSmith end-to-end. If you want to define what great AI observability looks like at production scale, this is where that work gets done.

 About the role

We’re looking for an Engineering Manager to lead the team building LangSmith, our observability and evaluation platform for LLM applications. You’ll define the technical direction, grow and mentor a world-class engineering team, and partner with product and design to deliver features that help developers build and deploy trustworthy AI systems with confidence.

 What you'll do
  • Build, mentor, and grow a high-performing team of engineers, fostering collaboration, ownership, and accountability

  • Strengthen LangChain’s engineering culture through mentorship, high-quality code, and technical excellence

  • Shape long-term technical direction and ensure scalability and reliability of LangSmith AI Observability Platform

  • Partner with product and design teams to define scope, sequence, and success criteria for key initiatives

  • Maintain a high bar for technical excellence while making sure the team moves with urgency and focus

  • Lead by example in writing clean, maintainable, and well-tested code in Go/Python and Typescript

  • Engage directly with customers to understand their needs and translate them into actionable product improvements

 What you'll bring
  • Experience managing or tech-leading teams delivering complex full-stack products

  • Strong technical background in Go/Python and React/Typescript

  • Solid understanding of databases (Postgres, Redis and ClickHouse)

  • Proven ability to design high throughput data-intensive systems

  • Bias towards shipping quickly and learning from customers

  • Strong communication skills—able to collaborate with internal stakeholders and engage directly with customers

  • Bonus: Experience building AI/LLM-powered applications, developer tools, or observability platforms

 
  • Annual salary range: $200,000- $240,000

Compensation Philosophy:

We offer competitive compensation that includes base salary, variable compensation for relevant roles, meaningful equity, benefits, and perks. Actual compensation and offerings will vary based on role, level, and location. Team members in the EU, UK, and APAC receive locally competitive benefits aligned with regional norms and regulations.

Benefits

Benefits include medical, dental, and vision coverage, flexible vacation, a 401(k) plan, meals on in-office days in the US and more.

Prepare for this job

A free preview built only from this posting: what it asks for, what you could be asked in an interview, and how to adjust your resume.

Skills and AI tools this role asks for

AI AgentsManagementObservabilityEvaluationLangGraphClayLangsmith

Questions you could be asked

  1. How do you decide when an AI agent can act on its own versus asking for approval first?
  2. Tell me about a project where management was part of your work. What did you do?
  3. Tell me about a project where observability was part of your work. What did you do?
  4. Tell me about a project where evaluation was part of your work. What did you do?
  5. Walk me through how you've used LangGraph in your day-to-day work.

Adapt your resume

  • List these exact terms on your resume: AI Agents, Management, Observability, Evaluation, and LangGraph. An applicant tracking system matches the wording, not the idea.
  • Attach one line of real, concrete experience to at least one of them — a tool named with nothing behind it rarely survives a human read.
  • Lead with what you built, trained or shipped — this role is judged on the AI system itself, not the tools around it.

Want your resume actually rewritten for this job?

The free preview above is everything we have today. A full resume rewrite is not live yet and has no price set. Join the waitlist and we will email you if we open it.

Similar roles

Software Engineering roles rated AI Level 4 at other companies.

What kind of AI work fits you?

Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.

Find my next step

More jobs at LangChain

Related searches

Same AI level