# Machine Learning Engineering Technical Leader at Cisco

Cisco is hiring a Machine Learning Engineering Technical Leader. It pays $216k-$281k a year and Level rates it Builds AI ●●●●; you can [apply on Cisco](https://jobsbylevel.com/go/d54f917b-00c9-4aeb-a099-be4ac746ed2d?utm_medium=ai-agent).

AI Level 4, AI centrality 86 out of 100. Milpitas, California, US.

## Details

- Company: [Cisco](https://jobsbylevel.com/companies/cisco)
- AI level: AI Level 4 (score 86 out of 100)
- Location: Milpitas, California, US
- Salary: $216k-$281k
- Posted: October 23, 2026
- Apply: https://jobsbylevel.com/go/d54f917b-00c9-4aeb-a099-be4ac746ed2d?utm_medium=ai-agent
- Free CV review for this job: https://jobsbylevel.com/cv-review?job=machine-learning-engineering-technical-leader-at-cisco-50b176

## Prepare for this job

The job page lists 5 likely interview questions for this offer, with a CV checklist: https://jobsbylevel.com/jobs/machine-learning-engineering-technical-leader-at-cisco-50b176

## Description

The application window is expected to close on: 12/10/2026 Meet the Team Cisco’s Canvas AI team consists of AI researchers and software developers who collaborate to build innovative products and platforms for Cisco. We are motivated by tackling unique challenges that arise when connecting people and devices at a world-wide scale. Canvas is an agentic workspace where our customers’ IT teams and our AI agents investigate and resolve issues across every domain. It was built for teams that need to move from alert to evidence, from evidence to decision, and from decision to resolution faster. Your Impact As a MLE Tech Lead you will set the multi-quarter strategy for how Cisco measures agentic AI quality, owning evaluation from architecture through production operation. You will build layered, trustworthy evaluation systems (automated checks, calibrated LLM judges, and expert human review), extend them to live production traffic and simulation, and close the loop so failures turn into validated fixes. You will also drive shared measurement standards across partner teams and mentor engineers while representing the domain to senior leadership. Key responsibilities Own the evaluation roadmap and strategy for agentic AI quality, from architecture to production. Design a layered evaluation architecture: deterministic checks on every PR, model-based evaluation pre-release, and human expert calibration. Build and calibrate multi-model LLM-as-judge panels, tracking true-positive and true-negative rates separately. Run the human-in-the-loop flywheel with SMEs, turning annotations into training signal for automated judges. Score production traffic online, alert on regressions and drift, and convert real traces into offline test cases. Lead simulation, synthetic data, and adversarial testing without consuming lab resources. Close the loop from evaluation failure to clustered issue, candidate fix, and shadow/A/B validation. Ensure trustworthy results through experiment snapshots, dataset versioning, coverage reporting, and cost tracking. Drive common standards, data contracts, and judge criteria across partner teams without direct authority. Mentor engineers, set evaluation engineering standards, and represent the domain to senior leadership. Minimum Qualifications BA/BS with 8+ years of relevant experience, MS with 6+ years, or PhD with 3+ years in Computer Science, Machine Learning or a related quantitative field. Experience taking machine learning or LLM-based systems into production and owning them operationally, not only building prototypes. Experience in Python and modern deep learning frameworks such as PyTorch or TensorFlow. Experience designing evaluation, measurement or quality systems for ML or LLM products — datasets, metrics, judges, benchmarks or regression suites. Experience setting technical direction across multiple teams through influence rather than reporting lines. Preferred Qualifications LLM-as-judge design and calibration: practical experience building judge systems and validating them against human experts, including inter-annotator agreement and judge drift. Agentic system evaluation: measuring multi-step reasoning, tool selection, trajectory quality and root-cause correctness rather than single-turn output quality. Production monitoring at scale: asynchronous scoring of live traffic, drift detection, and alerting that engineering teams actually act on. Simulation and synthetic environments: building reproducible test environments or synthetic data generation for domains where ground truth is scarce. Evaluation in CI/CD: integrating automated evaluation into build pipelines and using it to gate releases. Networking, observability or infrastructure domain knowledge: familiarity with the telemetry, topology and failure modes of enterprise network and observability systems. Cost-aware inference: routing, model selection and optimisation techniques that keep evaluation economically viable at production volume. Technical

The description is cut here. Read the full offer: https://jobsbylevel.com/jobs/machine-learning-engineering-technical-leader-at-cisco-50b176

Source: https://jobsbylevel.com/jobs/machine-learning-engineering-technical-leader-at-cisco-50b176

## Cite this page

Level. https://jobsbylevel.com/jobs/machine-learning-engineering-technical-leader-at-cisco-50b176.

Get job alerts: https://jobsbylevel.com/newsletter