Level

Hims & Hers

Sr. Staff Machine Learning Systems Engineer

Hims & Hers is hiring a Sr. Staff Machine Learning Systems Engineer for a remote role open to applicants in United States. It pays $240k-$265k a year and Level rates it ; you can apply on Hims & Hers.

Level's read on this job

  • Builds AI ●●●● is how Level rates this role: the job is building AI systems.
  • The posting lists this role as remote.
  • Tools and skills the posting cites include MLflow, Databricks and AI evaluation.
  • Level classifies it as a lead role.
  • Level lists 150 other open roles at Hims & Hers.

AI in this role

Senior Staff Machine Learning Systems Engineer owning end-to-end AI evaluation infrastructure, data pipelines, and safety testing for healthcare AI.

mlflowdatabricks
ai-evaluationmachine-learningdata-pipelinesevaluation-frameworksmlopssystem-architecture

Hims & Hers is the leading health and wellness platform, on a mission to help the world feel great through the power of better health. We are redefining healthcare by putting the customer first and delivering access to care that is affordable, accessible, and personal, from diagnosis to treatment to delivery. No two people are the same, so we provide access to personalized care designed for results. By normalizing health & wellness challenges and innovating on their solutions, we’re making better health outcomes easier to achieve. 

Hims & Hers is a public company, traded on the NYSE under the ticker symbol “HIMS.” To learn more about the brand and offerings, you can visit hims.com/about and hims.com/how-it-works . For information on the company’s outstanding benefits, culture, and its talent-first flexible/remote work approach, see below and visit www.hims.com/careers-professionals.

About the Role:

How do we make advanced AI/ML not just powerful but trustworthy enough to run in a regulated healthcare environment? We're looking for a Senior Staff engineer who can own that question end to end: the data pipelines that feed our models and evaluations, and the evaluation infrastructure — judges, scorers, statistical regression gates, red-team testing — that decides whether AI models are effective and safe to ship to patients.

This is a leadership role for someone who works comfortably across disciplines. You'll set technical direction for how we build, version, and trust the data and judgments that our AI products are evaluated against. Your scope will expand from raw data ingestion and feature/dataset pipelines, through evaluation methodology and statistical rigor, to the reporting surfaces that let clinical and product teams act on what we learn.

You'll spend most of your time on problems that don't have an existing playbook: ambiguous, cross-team, and genuinely hard to reason about. The job is to bring clarity to that ambiguity, chart a path the rest of the team and organization can follow, and see it through from idea to production, building the relationships and buy-in along the way to make it stick.

 

You Will:

Own the evaluation as a whole, not just a slice of it

  • Set the technical direction for our evaluation systems; metric, judge and scorer design, the statistical methodology behind regression decisions, and the infrastructure that tracks and categorizes failures over time.

  • Design and scale the data pipelines — ingestion, transformation, dataset versioning, labeling and calibration workflows — that both evaluation and downstream data science work depend on.

  • Proactively address challenges in scaling and complexity AI evaluation.

Lead projects that span teams and quarters

  • Define how we evaluate any new AI service from scratch. Drive multi-team initiatives like replacing manual, inconsistent review processes with statistically sound, automated gates.

  • Own our approach to adversarial and red-team evaluation as a risk-reduction program, designing the test suites and failure taxonomies that catch safety and edge-case issues before they reach patients.

  • Work through complex, cross-team technical disagreements and drive alignment across engineering, product and AI leaders.

Turn hard, ambiguous problems into solutions other teams can build on

  • Originate new approaches and methodology that becomes a reusable standard rather than a one-off fix.

  • Take vague, cross-team pain points ("we do this manually and it's inconsistent") all the way from a rough idea to a fully-specified, shipped system, without needing to hand off any part of the journey.

  • Lead major platform improvements; re-architecting core systems, removing brittle logic, modernizing how things run with impact that's felt org-wide, not just on your own team.

Grow the people and the network around you

  • Build real working relationships across ML engineering, data science, platform engineering, clinical, legal, and product, the kind of trust that gets you looped in early, before decisions are locked in.

  • Become a go-to voice on evaluation methodology and data pipeline design: share what you've learned in internal talks, write things up so other teams can use them, and expect your ideas to shape how others approach similar problems.

  • Mentor other engineers, including experienced ones, and help raise the technical and statistical bar of the teams you work with.

You Have:

  • 10+ years of experience in ML infrastructure, data engineering, or evaluation/testing systems, with a track record of impact that reaches beyond a single team or project.

  • Hands-on depth in evaluation systems: designing and calibrating LLM judges/scorers, building statistically sound regression-testing methodology (e.g., paired significance testing with proper correction for multiple comparisons), measuring agreement against human labels, and designing adversarial/red-team evaluation approaches.

  • Hands-on depth in data pipeline engineering: dataset versioning, feature and benchmark pipelines, labeling and calibration workflows, and high-throughput ingestion and transformation systems.

  • A history of building things that became the standard approach for others — not just solving your own problem, but changing how a broader group of people tackle a category of problem.

  • Experience leading multi-team projects to completion, including navigating and resolving genuine technical disagreement along the way.

  • A track record of mentoring other engineers, including senior ones, and visibly raising the bar for the teams around you.

  • Excellent communication — comfortable adapting the same idea for different audiences, and confident building support for it well before launch.

  • Strong Python, and enough statistical fluency to design and defend a testing framework that real production decisions ride on.

Nice to Have:

  • Experience with Databricks, MLflow, Unity Catalog, or similar data/eval platforms.

  • Experience building reporting tools for people without direct engineering access (e.g., automated Slack digests, spreadsheet reports for non-technical teams).

  • Prior experience in a regulated industry (healthcare, fintech, life sciences).

  • A track record of company-wide talks or write-ups that changed how other teams approached a problem.

Our Benefits (there are more but here are some highlights):

  • Competitive salary & equity compensation for full-time roles

  • Unlimited PTO, company holidays, and quarterly mental health days

  • Comprehensive health benefits including medical, dental & vision, and parental leave

  • Employee Stock Purchase Program (ESPP)

  • 401k benefits with employer matching contribution

  • Offsite team retreats

We are committed to building a workforce that reflects diverse perspectives and prioritizes ethics, wellness, and a strong sense of belonging. If you're excited about this role, we encourage you to apply—even if you're not sure if your background or experience is a perfect match.

Hims considers all qualified applicants for employment, including applicants with arrest or conviction records, in accordance with the San Francisco Fair Chance Ordinance, the Los Angeles County Fair Chance Ordinance, the California Fair Chance Act, and any similar state or local fair chance laws.

It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.

Hims & Hers is committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability, please contact us at accommodations@forhims.com and describe the needed accommodation. Your privacy is important to us, and any information you share will only be used for the legitimate purpose of considering your request for accommodation. Hims & Hers gives consideration to all qualified applicants without regard to any protected status, including disability. Please do not send resumes to this email address.

To learn more about how we collect, use, retain, and disclose Personal Information, please visit our Global Candidate Privacy Statement.

How we rate this

Sr. Staff Machine Learning Systems Engineer at Hims & Hers rates 90 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.

Classification

Builds AI. The job is building AI systems.

  1. ●●●● Builds AI80 to 100
  2. ●●●○ Works on AI60 to 79
  3. ●●○○ Uses AI40 to 59
  4. ●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.

Prepare for this job

Built from this posting and from similar jobs on Level.

Skills and AI tools this role asks for

AI EvaluationMachine learningData PipelinesEvaluation FrameworksMLOpsSystem ArchitectureMLflowDatabricks

Questions you could be asked

  1. How do you decide that one model's output is better than another's for a given task?
  2. Tell me about a project where machine learning was part of your work. What did you do?
  3. Tell me about a project where data pipelines was part of your work. What did you do?
  4. Tell me about a project where evaluation frameworks was part of your work. What did you do?
  5. Tell me about a project where mlops was part of your work. What did you do?

Adapt your resume

  • List these exact terms on your resume: AI Evaluation, Machine learning, Data Pipelines, Evaluation Frameworks, and MLOps. An applicant tracking system matches the wording, not the idea.
  • Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
  • Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.

Want an expert to read your CV for this job?

Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.

Get new remote software engineering jobs (Builds AI ●●●●) by email

One email a week with the new remote software engineering jobs (Builds AI ●●●●), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free. One email a week. Unsubscribe in one click.

Similar roles

Software Engineering roles that build AI, at other companies.

What kind of AI work fits you?

Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.

Find my next step

More jobs at Hims & Hers

Related searches

Same AI level