Level

Sword Health

Senior QA Analyst, AI Solutions

Sword Health is hiring a Senior QA Analyst, AI Solutions for a remote role open to applicants in Portugal. It pays €25k-€40k a year and Level rates it ; you can apply on Level.

AI in this role

Test and monitor AI healthcare agents, establishing quality standards and fixing prompt logic and reasoning chains.

prompt-engineeringai-agentsqahealthcareai-testingquality-assurance

At Sword, we’re building AI to heal billions and unlock humanity’s full potential. In doing so, we’re pioneering AI Care, a fundamentally new approach to healthcare built for medical reasoning, safety, and real-time treatment, not generic technology applied after the fact. As both a clinical-centric frontier AI lab and an applied AI platform, Sword is reimagining how care is delivered at scale, removing traditional barriers like appointments, waiting rooms, and stigma so more people can access the care they need—and ultimately get back to lives lived in full.

Since 2020, Sword has expanded across physical therapy, women’s health, cardiometabolic, and mental health, and is now moving beyond the session to a fully AI-native, 24/7 care program that brings physical activity, therapeutic exercise, psychotherapy, nutrition, and behavior change into one connected experience. More than 700,000 members across three continents have completed over 10 million AI sessions, helping 1,000+ enterprise clients avoid more than $1 billion in unnecessary healthcare costs. Backed by 42 clinical studies, 44+ patents, and more than $500 million raised from leading investors including Khosla Ventures, General Catalyst, and Founders Fund, Sword is defining a new standard for healthcare.

 

AI Proficiency at Sword

AI fluency is a core expectation at Sword. Every candidate is assessed against our three-level framework — be ready to share real examples of how AI is already part of how you work.

  • Explorer (Level 1) — Uses AI daily to boost personal productivity

  • Builder (Level 2) — Creates workflows and tools that elevate the whole team

  • Integrator (Level 3) — Embeds AI into products and processes at scale

Every hire must demonstrate at least Level 1. The expected level will vary depending on the seniority of the role.

 

 

What you’ll be doing:

  • Ensure AI agents perform consistently before and after going live;
  • Monitor performance across different healthcare environments and languages;
  • Liaise with the clinical team to create protocols and understand what clinically relevant parameters should be monitored;
  • Establish quality standards that work for AI systems, not traditional software;
  • Fix the underlying prompt logic and reasoning chains;
  • Create new testing methods to prevent similar issues;
  • Figure out how to test systems designed to be intelligent and adaptive;
  • Build frameworks for testing across infinite healthcare scenarios;
  • Define what "good enough" means for AI in healthcare;
  • Manage compliance with MDR (Medical Device Regulation) and AI Act requirements for AI systems together with the compliance team;
  • Document quality processes and verification and validation procedures for regulatory submissions;
  • Help ensure the algorithms perform with the expected accuracy and do not lead to bias;
  • Work with compliance teams to achieve and maintain certification of the relevant algorithms;
  • Help establish industry standards for AI quality assurance;
  • Become one of the few experts in AI healthcare testing methodologies;
  • Build your expertise in a field that's just being created;

 

What you need to have:

  • Comfortable speaking and understanding English (doesn't need to be perfect);
  • 2-6 years of experience in QA, testing, or analytical roles;
  • Strong analytical and problem-solving skills;
  • Comfortable learning new technologies and methodologies;
  • Curious about the evolving AI landscape;
  What we would love to see:
  • Familiarity with secure handling of health data (GDPR, HIPAA);
  • Experience with medical device regulations (MDR, 21 CFR 820, ISO 13485, IEC62304);
  • Comfortable speaking and understanding Portuguese;
  • Experience with AI tools or prompt engineering;
  • Background in healthcare, enterprise software, or regulated industries;
  • Familiarity with quality metrics and monitoring systems;
  • Basic programming or scripting experience.

 

*The range below reflects base, variable, and equity for this role.

This is a starting point, not a ceiling — once someone joins and proves they're outlier talent, we adjust quickly to make sure their compensation matches their impact.

Job titles may span more than one career level, so actual pay depends on skills, qualifications, experience, location, and market demand, among other factors. The range reflects base salary plus any variable, bonus, or sales incentives, and our estimate of the value of private company stock options where applicable. It's subject to change, and future stock value isn't guaranteed. Sword also offers additional benefits beyond total compensation.

Total Compensation Range€25.200—€39.600 EUR    Country-specific benefits may vary and will be detailed separately. Sword Health complies with applicable Federal and State civil rights laws and does not discriminate on the basis of Age, Ancestry, Color, Citizenship, Gender, Gender expression, Gender identity, Gender information, Marital status, Medical condition, National origin, Physical or mental disability, Pregnancy, Race, Religion, Caste, Sexual orientation, and Veteran status.

How we rate this

Senior QA Analyst, AI Solutions at Sword Health rates 65 out of 100 for how much of the daily work is AI. That makes it Works on AI (AI Level 3 of 4). The level is about AI in the job, not seniority.

Classification

Works on AI. The daily work is on AI products, without building the model.

  1. ●●●● Builds AI80 to 100
  2. ●●●○ Works on AI60 to 79
  3. ●●○○ Uses AI40 to 59
  4. ●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Open the description and count.

Prepare for this job

A free preview built only from this posting: what it asks for, what you could be asked in an interview, and how to adjust your resume.

Skills and AI tools this role asks for

Prompt engineeringAI agentsQaHealthcareAI TestingQuality Assurance

Questions you could be asked

  1. How do you structure and test a prompt to get consistent output from a language model?
  2. How do you decide when an AI agent can act on its own versus asking for approval first?
  3. Tell me about a project where qa was part of your work. What did you do?
  4. Tell me about a project where healthcare was part of your work. What did you do?
  5. Tell me about a project where ai testing was part of your work. What did you do?

Adapt your resume

  • List these exact terms on your resume: Prompt engineering, AI agents, Qa, Healthcare, and AI Testing. An applicant tracking system matches the wording, not the idea.
  • Attach one line of real, concrete experience to at least one of them — a tool named with nothing behind it rarely survives a human read.
  • Show where AI is part of your daily process, not a one-off project — this role expects it to be a running habit.

Want an expert to read your CV for this job?

Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.

Get a free CV review

Get new remote software engineering jobs (Works on AI ●●●○ or higher) by email

One email a week with the new remote software engineering jobs (Works on AI ●●●○ or higher), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free. One email a week. Unsubscribe in one click.

Similar roles

Software Engineering roles that work on AI, at other companies.

What kind of AI work fits you?

Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.

Find my next step

More jobs at Sword Health

Related searches

Same AI level