Lead AI Ops Engineer

AI in this role

Own end-to-end production monitoring, reliability, and governance for AI agents and LLM applications.

ai-agentsai-safetyai-opsmonitoringobservabilityevaluationprompt-engineering

Our Purpose

Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.

Title and Summary

Lead AI Ops Engineer

AI Ops

Overview:
Mastercard's Operational Intelligence team is building the next generation of data products and AI solutions that help customers access and use operational insights in new ways. As AI agents move from POC to production, their behavior keeps changing in real time: the context they retrieve, the skills they select, how they resolve exceptions. These agents are never really “finished.” They keep learning from production outcomes, so their reliability, safety, and trustworthiness need to be actively operated, not just monitored. That is the job of AI Ops: making sure each agent’s reasoning stays correct, its context stays current, and its behavior stays governed as it learns.

Role:
• Own end to end production monitoring for one or more AI agents, tracking key metrics, KPIs, accuracy, latency, SLA breaches, guardrail triggers, and out of scope rates through dedicated observability dashboards
• Serve as L1/L2 production support, providing first response monitoring, triage, and escalation to tech teams as new agentic services go live
• Identify and diagnose issues by monitoring eval scores and drift alerts, inspecting failing traces and patterns, and classifying root causes such as intent, tool, parameter, or hallucination errors
• Evaluate whether the agent’s reasoning remains correct in production, checking whether context retrieval stays current, whether resolutions match ground truth, and whether the learning loop is drifting
• Monitor and manage post production reliability, including API and authentication failures and issues originating from external AI endpoints, to protect revenue generating, client facing agents
• Partner with AI Engineering to prioritize fixes, run experiments on prompts, context, and routing in Dev, and validate improvements via evals before handing off proven changes for deployment
• Embed governance within the agent’s learning loop so that approval workflows and audit logging travel with every production change, rather than being added after the fact

All About You:
• Experience monitoring or operating production AI/ML or agentic systems, including observability, evaluation, and drift detection practices
• Strong analytical skills to diagnose root causes across intent classification, tool selection, parameter errors, and hallucination patterns
• Understanding of AI/LLM observability concepts such as tracing, telemetry, guardrails, and evaluation engineering including benchmarks, automated evals, human review, and regression testing
• Familiarity with governance, compliance, and responsible AI practices such as PII detection, access control, audit logging, and policy enforcement is a plus
• Ability to work independently alongside existing Tech teams while owning a dedicated agent’s post launch health
• Excellent communication skills to translate production signals into clear, prioritized asks for AI Product and AI Engineering teams
• Comfort working in a fast evolving environment where the process itself, not just the throughput, is what is being continuously operated on and improved

Corporate Security Responsibility

All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:

Abide by Mastercard’s security policies and practices;
Ensure the confidentiality and integrity of the information being accessed;
Report any suspected information security violation or breach, and
Complete all periodic mandatory security trainings in accordance with Mastercard’s guidelines.

How we rate this

Lead AI Ops Engineer at Mastercard rates 75 out of 100 for how much of the daily work is AI. That makes it Works on AI (AI Level 3 of 4). The level is about AI in the job, not seniority.

Classification

Works on AI. The daily work is on AI products, without building the model.

●●●● Builds AI80 to 100
●●●○ Works on AI60 to 79
●●○○ Uses AI40 to 59
●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Open the description and count.

Key facts

Lead AI Ops Engineer is an open job at Mastercard.
Level rates this role AI Level 3 of 4: Works on AI.
The role is based in Gurgaon, India.
The job was posted on October 6, 2026.
Seniority: lead.
Applications are handled by Mastercard. Apply on Mastercard

Prepare for this job

A free preview built only from this posting: what it asks for, what you could be asked in an interview, and how to adjust your resume.

Skills and AI tools this role asks for

AI AgentsAI SafetyAI OpsMonitoringObservabilityEvaluationPrompt Engineering

Questions you could be asked

How do you decide when an AI agent can act on its own versus asking for approval first?
How do you think about the risk of an AI system in this kind of role failing silently?
Tell me about a project where ai ops was part of your work. What did you do?
Tell me about a project where monitoring was part of your work. What did you do?
Tell me about a project where observability was part of your work. What did you do?

Adapt your resume

List these exact terms on your resume: AI Agents, AI Safety, AI Ops, Monitoring, and Observability. An applicant tracking system matches the wording, not the idea.
Attach one line of real, concrete experience to at least one of them — a tool named with nothing behind it rarely survives a human read.
Show where AI is part of your daily process, not a one-off project — this role expects it to be a running habit.

Want an expert to read your CV for this job?

Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.

Get a free CV review

Get new AI jobs (Works on AI ●●●○ or higher) by email

One email a week with the new AI jobs (Works on AI ●●●○ or higher), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Also alert me for new jobs at Mastercard

Free. One email a week. Unsubscribe in one click.