QA Engineer - AI Systems III

Proofpoint is hiring a QA Engineer - AI Systems III in Hyderabad, India. Level rates it ; you can apply on Level.

Level's read on this job

On Level's scale, this role is Builds AI ●●●●: the job is building AI systems.
The posting mentions prompt engineering, RAG and AI agents among its tools and skills.
Level classes it as a mid-level role.
Proofpoint has 120 other open roles on Level.
This posting went up in the last few days.

AI in this role

prompt-engineeringragai-agentsai-evaluation

About Us:

Proofpoint is a global leader in human- and agent-centric cybersecurity. We protect how people, data, and AI agents connect across email, cloud, and collaboration tools. Over 80 of the Fortune 100, 10,000 large enterprises, and millions of smaller organizations trust Proofpoint to stop threats, prevent data loss, and build resilience across their people and AI workflows. Our mission is simple: safeguard the digital world and empower people to work securely and confidently. Join us in our pursuit to defend data and protect people.

How We Work:

At Proofpoint you’ll be part of a global team that breaks barriers to redefine cybersecurity guided by our BRAVE core values:

Bold in how we dream and innovate

Responsive to feedback, challenges and opportunities

Accountable for results and best in class outcomes

Visionary in future focused problem-solving

Exceptional in execution and impact

We're looking for an AI Engineer who is passionate about both building AI-powered applications and ensuring they work reliably in production. This role combines AI engineering with quality engineering, focusing on developing LLM-based solutions while creating automated testing and evaluation frameworks to measure quality, accuracy, safety, and performance. You will work closely with engineering, product, and business teams to deliver AI solutions that are not only innovative, but also measurable, stable, and production-ready.

Responsibilities:
- AI Engineering: Build and maintain AI-powered applications using LLMs, RAG pipelines, and agent-based workflows.
- Develop integrations with foundation models and AI services using Python and modern AI frameworks.

- Design prompt workflows, structured outputs, tool calling, and retrieval-based systems.

- Build APIs, backend services, and internal tools using FastAPI or similar frameworks.

- Collaborate with stakeholders to translate business requirements into AI solutions.

- AI Quality & Automation Design and execute automated test suites for AI applications and agentic workflows.

- Create and maintain evaluation datasets, benchmark scenarios, and regression test cases.

- Build pipelines to measure response quality, hallucinations, accuracy, latency, and cost.

- Implement LLM-as-a-judge evaluations and automated scoring frameworks using tools such as DeepEval, RAGAS, or PromptFoo.

- Develop AI-powered test generation for functional, edge-case, and negative testing scenarios.

- Perform adversarial testing including prompt injection, jailbreak attempts, and safety validation.

- Monitor production AI systems through logging, tracing, and quality dashboards.

- Work with engineering teams to establish quality gates before AI features are released.

Required Qualifications:
- 2- 4 years of software engineering, QA automation, AI engineering, or related experience.

- Strong Python and Golang programming skills and understanding of software engineering fundamentals.

- Familiarity with LLM applications, prompt engineering, RAG, or agent-based AI systems.

- Experience in test automation using frameworks such as PyTest or equivalent tools.

- Understanding of API testing, backend services, and REST architectures.

- Knowledge of AI evaluation concepts such as hallucination detection, response quality measurement, benchmark testing, and regression testing.

- Experience with SQL and basic cloud platforms (AWS, Azure, or GCP).
- Strong analytical and problem-solving skills.

Why Proofpoint?

At Proofpoint, we believe that an exceptional career experience includes a comprehensive compensation and benefits package. Here are just a few reasons you’ll love working with us:

Competitive compensation
Comprehensive benefits
Career success on your terms
Flexible work environment
Annual wellness and community outreach days
Always on recognition for your contributions
Global collaboration and networking opportunities

Our Culture:

Our culture is rooted in values that inspire belonging, empower purpose and drive success-every day, for everyone.

We encourage applications from individuals of all backgrounds, experiences, and perspectives. If you need accommodation during the application or interview process, please reach out to accessibility@proofpoint.com.

How to Apply

Interested? Submit your application along with any supporting information- we can’t wait to hear from you!

How we rate this

QA Engineer - AI Systems III at Proofpoint rates 90 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.

Classification

Builds AI. The job is building AI systems.

●●●● Builds AI80 to 100
●●●○ Works on AI60 to 79
●●○○ Uses AI40 to 59
●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Open the description and count.

Key facts

QA Engineer - AI Systems III is an open job at Proofpoint.
Level rates this role AI Level 4 of 4: Builds AI.
The role is based in Hyderabad, India.
The job was posted on October 19, 2026.
Seniority: mid.
Applications are handled by Proofpoint. Apply on Proofpoint

Prepare for this job

Built from this posting and from similar jobs on Level.

Skills and AI tools this role asks for

Prompt engineeringRAGAI agentsAI Evaluation

Questions you could be asked

How do you structure and test a prompt to get consistent output from a language model?
How would you design a retrieval step so the model answers from real data instead of guessing?
How do you decide when an AI agent can act on its own versus asking for approval first?
How do you decide that one model's output is better than another's for a given task?
How would you decide a model or AI system is ready to ship?

Adapt your resume

List these exact terms on your resume: Prompt engineering, RAG, AI agents, and AI Evaluation. An applicant tracking system matches the wording, not the idea.
Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.

Want an expert to read your CV for this job?

Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.

Get a free CV review Check your resume with the ATS resume checker

Get new software engineering jobs (Builds AI ●●●●) by email

One email a week with the new software engineering jobs (Builds AI ●●●●), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free. One email a week. Unsubscribe in one click.