Level

Vanguard

Application Engineering Technical Lead - II

Vanguard is hiring an Application Engineering Technical Lead - II. Level rates it ; you can apply on Level.

AI in this role

Leads the technical direction and architecture for AI-powered conversational platforms, agent orchestration, and model governance.

claudecopilotarize
ragai-evaluationai-safetyconversational-aiagent-orchestrationmodel-governanceobservabilityretrieval-augmented-generation

The Application Engineering Technical Lead - II will help shape how AI-powered experiences are built, governed, and scaled across Workplace Technology, with direct influence on the reusable platforms, standards, and engineering practices that enable teams to deliver AI capabilities responsibly.

This role sets technical direction across the team's conversational AI orchestration platform and the surrounding infrastructure, with responsibility for patterns and standards in areas such as:

  • Agent orchestration architecture and agent-to-agent (A2A) integration patterns

  • Conversation history, compliance, and retention pipelines (e.g., WORM-based patterns)

  • Model governance and observability (e.g., Arize, evaluation frameworks)

  • Knowledge management and retrieval systems (RBAC, ingestion pipelines)

  • Model access patterns across multiple providers and model sizes

This role also provides technical evaluation and proof-of-concept leadership as the team explores new AI-powered use cases, helping determine feasibility, implementation patterns, risks, and reuse potential before broader adoption.

Responsibilities:

  • Provides expert-level technical leadership for the team's orchestration platform, knowledge management capabilities, and model governance and observability practices. Leads complex development, design, implementation, and architecture specification activities; recommends development options; and approves technical designs and solutions. Ensures deliverables are viable, secure, and testable, with particular focus on compliance-sensitive pipelines. Resolves complex technical issues, supports root cause analysis, and helps prevent recurring technology problems.

  • Communicates with key stakeholders on project issues, risks, and implications. Evaluates the impact of change requests on the orchestration platform and influences alignment on technical direction.

  • Maintains current working knowledge of AI-amplified PDLC methodology, agent orchestration architecture, model evaluation, observability practices, and emerging model access patterns. Mentors engineers, identifies training needs, and drives adoption of new technical standards across the team.

  • This role reviews and approves architecture documentation and diagrams for the orchestration platform and shared AI infrastructure. It also defines technical standards and processes, including patterns for agent onboarding, knowledge ingestion, and conversation compliance.

  • Identifies opportunities for continuous quality improvement of technical standards, methodologies, and technologies across the team's platforms.

  • Participates in design, code, and test inspections throughout the product life cycle. Serves as a technical consultant at project meetings and presents technical status and issues at milestone reviews.

  • Understands and complies with Information Technology and Information Security policies and procedures, and ensures deliverables meet applicable security, compliance, and governance requirements.

  • Participates in special projects and performs other duties as assigned.

Qualifications:

Preferred attributes include strong technical judgment, comfort operating in ambiguity, ability to influence without direct authority, and a demonstrated commitment to engineering excellence, responsible AI practices, and continuous learning.

  • Minimum of eight years related work experience, with at least five years of development experience.

  • Undergraduate degree or equivalent combination of training and experience. Graduate degree preferred.

  • Hands-on experience with agent orchestration or conversational AI platforms (e.g., agent-to-agent/A2A protocols), cloud-native service architecture, and knowledge management/RAG or semantic search systems.

  • Experience with LLM observability and evaluation tooling (e.g., Arize or comparable) and exposure to multiple model providers and access pattern design.

  • Knowledge of AI-amplified PDLC practices and hands-on experience with languages and tools such as Python, TypeScript, GitHub Copilot, Claude, or comparable AI engineering tools.

Special Factors

Sponsorship

Vanguard is not offering visa sponsorship for this position.

About Vanguard

At Vanguard, we don't just have a mission—we're on a mission.

To work for the long-term financial wellbeing of our clients. To lead through product and services that transform our clients' lives. To learn and develop our skills as individuals and as a team. From Malvern to Melbourne, our mission drives us forward and inspires us to be our best.

How We Work

Vanguard has implemented a hybrid working model for the majority of our crew members, designed to capture the benefits of enhanced flexibility while enabling in-person learning, collaboration, and connection. We believe our mission-driven and highly collaborative culture is a critical enabler to support long-term client outcomes and enrich the employee experience.

How we rate this

Application Engineering Technical Lead - II at Vanguard rates 85 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.

Classification

Builds AI. The job is building AI systems.

  1. ●●●● Builds AI80 to 100
  2. ●●●○ Works on AI60 to 79
  3. ●●○○ Uses AI40 to 59
  4. ●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Open the description and count.

Prepare for this job

A free preview built only from this posting: what it asks for, what you could be asked in an interview, and how to adjust your resume.

Skills and AI tools this role asks for

RAGAI EvaluationAI SafetyConversational AIAgent OrchestrationModel GovernanceObservabilityRetrieval Augmented Generation

Questions you could be asked

  1. How would you design a retrieval step so the model answers from real data instead of guessing?
  2. How do you decide that one model's output is better than another's for a given task?
  3. How do you think about the risk of an AI system in this kind of role failing silently?
  4. Tell me about a project where conversational ai was part of your work. What did you do?
  5. Tell me about a project where agent orchestration was part of your work. What did you do?

Adapt your resume

  • List these exact terms on your resume: RAG, AI Evaluation, AI Safety, Conversational AI, and Agent Orchestration. An applicant tracking system matches the wording, not the idea.
  • Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
  • Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.

Want an expert to read your CV for this job?

Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.

Get new software engineering jobs (Builds AI ●●●●) by email

One email a week with the new software engineering jobs (Builds AI ●●●●), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free. One email a week. Unsubscribe in one click.

Similar roles

Software Engineering roles that build AI, at other companies.

What kind of AI work fits you?

Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.

Find my next step

More jobs at Vanguard

Related searches

Same AI level