# Senior Product Software Engineer - AI Quality Engineering at Wolters Kluwer

AI Level 4, AI centrality 88 out of 100. USA - Cape Girardeau, MO.

## Details

- Company: [Wolters Kluwer](https://jobsbylevel.com/companies/wolters-kluwer)
- AI level: AI Level 4 (score 88 out of 100)
- Location: USA - Cape Girardeau, MO
- Salary: $83k-$146k
- Posted: October 7, 2026
- Apply: https://jobsbylevel.com/go/c26614b4-1536-46e3-ace6-22c0f7f51f68

## Description

We are seeking an experienced Senior AI Quality Engineer to own and implement quality strategy, evaluation systems, and reliability practices for AI-powered product experiences in Wolters Kluwer Tax and Accounting. In this role, you will architect, build, and maintain evaluation systems that measure retrieval accuracy, citation correctness, response quality, safety, performance, and overall alignment with customer intent. You will work hands-on across test automation, AI evaluation, APIs, data pipelines, observability, and release readiness while partnering with software engineers, product managers, UX, and stakeholders to ensure our generative AI systems are trustworthy, observable, secure, and ready for production use in professional tax and accounting workflows. This role goes beyond validating that features function as designed. You will own the practical quality engineering implementation for non-deterministic AI systems: defining how quality is measured, building automated evaluation and regression capabilities, implementing quality gates in delivery pipelines, driving release-readiness decisions, mentoring team members in AI quality practices, and establishing the standards customers can trust. Key Responsibilities: Own and implement the quality engineering approach for enterprise-scale generative AI applications, including RAG pipelines, LLM orchestration, agentic workflows, APIs, ingestion pipelines, and AI-driven user experiences. Architect and implement evaluation harnesses that measure retrieval accuracy, citation correctness, answer relevance, groundedness, hallucination rate, safety, latency, cost, and end-to-end system behavior. Design automated regression suites that detect quality drift across prompts, embeddings, chunking strategies, model versions, retrieval configuration, orchestration logic, and application workflows. Partner with developers to validate AI system architecture, service integrations, API behavior, data ingestion, observability, resilience, security controls, and production readiness. Lead root-cause analysis for AI quality failures using logs, traces, evaluation outputs, feedback signals, customer scenarios, and domain-specific acceptance criteria. Define, implement, and maintain measurable quality gates for AI releases, and advise product and engineering leaders on risk, readiness, and tradeoffs across accuracy, reliability, latency, cost, and user trust. Develop test automation in Python and related frameworks for APIs, services, chat workflows, data pipelines, and AI evaluation workflows within CI/CD pipelines. Build and maintain observability practices and dashboards that track model behavior, retrieval performance, operational health, quality trends, and customer-impacting failure modes. Contribute to responsible AI practices including bias and fairness checks, content safety validation, prompt injection testing, data privacy validation, and compliance-oriented evidence collection. Collaborate with product managers, UX, subject matter experts, and stakeholders to translate customer workflows and business requirements into testable AI quality criteria. Mentor engineers and quality team members in AI evaluation methods, non-deterministic testing strategies, automation design, and production-quality engineering practices. Stay current with emerging AI quality, LLM evaluation, RAG assessment, and LLMOps practices through research, experimentation, and community engagement. Key Requirements: 5+ years of experience in quality engineering, SDET, software engineering, test automation, reliability engineering, or equivalent engineering roles. 2+ years of hands-on experience testing, evaluating, or validating AI/ML, LLM, RAG, or other non-deterministic systems in production or production-like environments. Strong proficiency with Python test automation frameworks such as Pytest, unittest, or equivalent, including experience building reusable test harnesses and regression suites. Experience

The description is cut here. Read the full offer: https://jobsbylevel.com/jobs/senior-product-software-engineer-ai-quality-engineering-at-wolters-kluwer-948985

Source: https://jobsbylevel.com/jobs/senior-product-software-engineer-ai-quality-engineering-at-wolters-kluwer-948985

## Cite this page

Level. https://jobsbylevel.com/jobs/senior-product-software-engineer-ai-quality-engineering-at-wolters-kluwer-948985.

Get job alerts: https://jobsbylevel.com/newsletter