State StreetGenerative AI Quality Engineer– Tech Lead
State Street is hiring a Generative AI Quality Engineer– Tech Lead in Hyderabad, India. Level rates it ; you can apply on Level.
AI in this role
Test and evaluate Large Language Models and AI-driven workflow agents for financial applications.
Senior Associate - Gen AI Quality Engineer
About the Role
Who we are looking for
We are looking for Software Quality Assurance Testers with 3–8 years of experience, with exposure or interest in LLM (Large Language Model) testing and Agents testing. This role involves analyzing business and technical specifications, defining and executing test cases, and capturing and tracking defects.
You will work on Artificial Intelligence initiatives within a multidisciplinary team focused on emerging trends, cutting‑edge technologies, and their application in financial services for our clients.
This role supports a hybrid work model, allowing you to balance work from home and office based on your needs and role requirements.
What you will be responsible for
As an SQA Tester, you will:
- Analyze business and technical specifications.
- Write functional test cases and perform regression testing.
- Report bugs, verify fixes, and maintain the bug tracking system (e.g., Jira).
- Maintain and execute automated tests using Selenium with Python.
- Create automated test scripts using the Pytest framework.
- Automate web services using Postman and/or Python.
- Execute test plans for Large Language Models (LLMs), including prompt validation, accuracy checks, and basic safety evaluation.
- Support Agents testing for AI-driven workflows to ensure reliability, robustness, and integration quality.
What we value
These skills will help you succeed in this role:
- Ability to understand customer/business user needs and translate them into test scenarios.
- Strong analytical, technical, and problem-solving skills with attention to detail.
- Ability to manage time effectively and prioritize tasks while working within deadlines.
- Experience with BDD & TDD (Gherkin).
- Experience using Jira as an agile management tool.
- Experience in using Selenium with Python for test automation.
- Experience in testing LLMs, including prompt-response validation and evaluation of model accuracy and relevance.
- Exposure to red teaming for identifying vulnerabilities and edge cases in LLMs (preferred but not mandatory at junior level).
- Familiarity with evaluation tools such as OpenAI Evals, Ragas, or custom benchmarking suites (good-to-have).
Education & Preferred Qualifications
- 3–8 years of software quality assurance testing experience.
- Experience with tools like PyCharm, VS Code, Git, Fiddler, Postman, SoapUI, JMeter, Jenkins, Jira.
- Experience in CI/CD-based projects.
- Hands‑on experience or strong exposure to LLM testing and Agents testing is highly preferred.
Additional Requirements
- Basic understanding of artificial intelligence and machine learning concepts is preferred.
- Familiarity with AI governance, bias detection, and safety validation will be an added advantage.
Work Schedule
On-premise
About State Street
Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.
We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future.
As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.
Discover more information on jobs at StateStreet.com/careers
Read our CEO Statement
How we rate this
Generative AI Quality Engineer– Tech Lead at State Street rates 80 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.
Builds AI. The job is building AI systems.
- ●●●● Builds AI80 to 100
- ●●●○ Works on AI60 to 79
- ●●○○ Uses AI40 to 59
- ●○○○ Little AI0 to 39
Levels come from how often the tools, models and workflows of the role are named in the posting itself. Open the description and count.
Prepare for this job
Built from this posting and from similar jobs on Level.
Skills and AI tools this role asks for
Questions you could be asked
- Tell me about a project where software quality assurance was part of your work. What did you do?
- Tell me about a project where llm testing was part of your work. What did you do?
- Tell me about a project where prompt validation was part of your work. What did you do?
- Tell me about a project where test automation was part of your work. What did you do?
- Walk me through how you've used OpenAI in your day-to-day work.
Adapt your resume
- List these exact terms on your resume: Software Quality Assurance, LLM Testing, Prompt Validation, Test Automation, and OpenAI. An applicant tracking system matches the wording, not the idea.
- Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
- Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.
Want an expert to read your CV for this job?
Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.
Get new remote software engineering jobs (Builds AI ●●●●) by email
One email a week with the new remote software engineering jobs (Builds AI ●●●●), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.
Free. One email a week. Unsubscribe in one click.
Similar roles
Software Engineering roles that build AI, at other companies.
What kind of AI work fits you?
Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.
Find my next step