Senior Resiliency Architect, Hardware Diagnostics and Resiliency Modeling

NVIDIA is hiring a Senior Resiliency Architect, Hardware Diagnostics and Resiliency Modeling in Santa Clara, United States. It pays $184k-$288k a year and Level rates it ; you can apply on NVIDIA.

Level's read on this job

On Level's scale, this role is Works on AI ●●●○: the daily work is on AI products, without building the model.
Level classes it as a senior role.
NVIDIA has 2,700+ other open roles on Level.
Posted in the last few days.

AI in this role

We are now seeking a Senior Resiliency Architect to support the development of hardware diagnostics and resiliency models for Datacenter and Edge Computing products with applications spanning Agentic and Physical AI. In this role, you will be a key member of a team of innovators, challenging the status quo and pushing beyond boundaries. You will have the opportunity to impact the industry's leading GPUs and SoCs powering product lines ranging from the rapidly growing field of artificial intelligence to self-driving cars and robots.

What you'll be doing:

Develop and maintain diagnostics software suite to efficiently stress test NVIDIA GPUs and SOCs to identify hardware defects, including defects that cause silent data corruption. These tests will run in large-scale deployments of Datacenter GPUs and Safety SOCs in package/board/rack configurations spanning GPUs, CPUs, and Networking SOCs.
Analyze coverage gaps in NVIDIA diagnostic suite flagged by silicon failures on customer workloads or test suites. Enhance diagnostics to improve coverage, repeatability of failures detected, and optimize test time.
Model and analyze RAS metrics like Failures in Time (FIT) for permanent and transient errors, and Availability from GPU to Rack to Datacenter. Use models to identify gaps and drive RAS improvements through new hardware and software features.
Analyze field data on hardware interruptions and permanent failures, enhance RAS models to better correlate to field data. Ensure optimal fault attribution, containment, and recovery for hardware errors.
Design, build, and maintain automation scripts and tooling to systematically kick off, monitor, and manage diagnostic jobs across multiple independent compute clusters.
Develop and maintain dashboards that integrate diverse data sources and formats, deliver insights through analytics and visualization, and ensure reliable hosting, secure access, and multi-user performance.
Collaborate with engineering teams to ensure system telemetries accurately categorize cluster failures into hardware faults, test-script bugs, or environment configuration issues.
Ensure availability models and hardware diagnostics evolve to reflect AI workload changes

What we need to see:

Master’s degree in Computer Engineering, Electrical Engineering, Computer Science or closely related degree or equivalent experience.
At least 5+ years of relevant experience.
Knowledge of GPU Architecture and programming, hardware diagnostics
Good understanding of Reliability (FIT) models, AVF estimation, Availability concepts, ECC/parity/CRC strategies.
Proficiency in data mining, automation, usage of AI tools and scripting with Python, SQLor similar.
Proficiency in C/C++, CUDA programming, and using AI tools for coding and debug.
Strong debugging and analytical skills.
Excellent interpersonal skills and ability to collaborate with on-site and remote teams.
Working knowledge of hardware diagnostics, and container-based job execution.

Ways to stand out from the crowd:

Strong statistical judgment - particularly around exposure-adjusted rates, sample size, and distinguishing signal from artefact.
Strong understanding of resiliency and reliability trade-offs in AI data centers, including failure modes, mitigation and recovery strategies, and impact on large-scale training/inference workloads
Experience in developing embedded software or visualization and data analytics tools.

Your base salary will be determined based on your location, experience, and other objective job-related factors. The expected base salary range for this position is 184,000 USD – 287,500 USD.

You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until November 7, 2026.

This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

How we rate this

Senior Resiliency Architect, Hardware Diagnostics and Resiliency Modeling at NVIDIA rates 63 out of 100 for how much of the daily work is AI. That makes it Works on AI (AI Level 3 of 4). The level is about AI in the job, not seniority.

Classification

Works on AI. The daily work is on AI products, without building the model.

●●●● Builds AI80 to 100
●●●○ Works on AI60 to 79
●●○○ Uses AI40 to 59
●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.

Key facts

Senior Resiliency Architect, Hardware Diagnostics and Resiliency Modeling is an open job at NVIDIA.
Level rates this role AI Level 3 of 4: Works on AI.
The listed salary is $184k-$288k a year.
The role is based in US, CA, Santa Clara.
The job was posted on November 3, 2026.
Seniority: senior.
Applications are handled by NVIDIA. Apply on NVIDIA

Get new software engineering jobs (Works on AI ●●●○ or higher) by email

One email a week with the new software engineering jobs (Works on AI ●●●○ or higher), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free. One email a week. Unsubscribe in one click.