Sr. Systems Development Engineer, Trainium Systems
Amazon is hiring a Sr. Systems Development Engineer, Trainium Systems in San Diego, United States. It pays $174k-$235k a year and Level rates it ; you can apply on Amazon.
Level's read on this job
- Builds AI ●●●● is how Level rates this role: the job is building AI systems.
- The posting mentions Trainium, Graviton and AWS Nitro among its tools and skills.
- Level classes it as a senior role.
- Amazon has 21,000+ other open roles on Level.
- Posted in the last few days.
AI in this role
Build end-to-end software systems and services that provision, operate, and monitor AWS AI training and inference hardware.
Who are we?
Annapurna Labs is the part of Amazon that designs the custom silicon behind the cloud, including Trainium for AI training, Graviton CPUs, and the AWS Nitro system. Nearly every server in AWS runs hardware we designed. We build the chips, the systems, and the software that power some of the largest AI training and inference workloads in the world.
What do we do?
We build the software that provisions, operates, updates, and monitors the servers behind AWS's largest AI training workloads, built on Trainium, Graviton, and the AWS Nitro system. We give the people who run this hardware a complete way to stand it up, watch its health, deliver new software to it, and diagnose it, all through open, standard interfaces. We own that software from design through the running system.
Why it's high-impact?
The hardware is just silicon until our software makes it provisionable, observable, and serviceable. Every system runs through what we build. The service you own is on the path for all of them, and the people who run this hardware depend on it every day.
What's the challenge?
There are many ambiguous and difficult challenges in our fast-moving space. Our services require deep system and software expertise. You need to have the ability to work within a fast moving and startup-like environment in a large company. You will identify solutions, trying ideas, given space to fail and iterate to produce products that your customers love.
What you will do?
You will be part of a team that builds services end to end, from a blank page to an operator running it in production. You work backward from what the customer needs to build tools, systems, and network for customer.
Who would succeed in this role?
Deeply technical engineers who stay close to the customer as well as the systems architecture and design. They think about the customer experience and the outcome. A person who works autonomously and dives deep into a problem to understand how things work, when to make a subtle change, and when to disrupt the status quo to achieve the right result.
Key job responsibilities
- Own the tools and systems that are compatible within the operator's network, systems, and infrastructure.
- Build tools and services that enable a customer to operate these servers at scale.
- Build services and tools for monitoring, alerting, and anomaly detection.
- Develop troubleshooting tools and runbooks that enable rapid diagnosis and resolution of site infrastructure issues.
- Consult with customer teams on network design and infrastructure standards, working across organizational boundaries.
- Drive customer expansion readiness, ensuring new customer infrastructure is ready.
- Contribute to architecture decisions for product evolution.
- Develop and build systems that enable operating infrastructure at scale.
A day in the life
You design and build the provisioning and management surface over standard interfaces, and the boot flow operator tools drive. You build the offline software-update service and prove the signed bundle applies and recovers cleanly. You design each system against the interfaces the platform exposes and turn architecture decisions into APIs and tools a customer runs without you in the loop.
About the team
Our team builds the software suite for provisioning, monitoring, software delivery, and the diagnostics of Trainium systems, and we build all of it on open, standard interfaces. We serve the engineers and operators who run large-scale AI training and need to provision, monitor, update, and troubleshoot it on their own terms. We own it end to end.
We're a small team with a large charter. You own real systems on day one. There are no layers between you and the problem. You build it, you ship it, you operate it, you improve it.
Basic qualifications
- Experience leading the design, automation, deployment, and support of large-scale infrastructure
- Experience programming with at least one modern language such as Python, Ruby, Golang, Java, C++, C#, Rust
- Experience with Linux/Unix
- Experience with CI/CD pipelines build processes
Preferred qualifications
- Experience with distributed systems at scale
Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.
Los Angeles County applicants: Job duties for this position include: work safely and cooperatively with other employees, supervisors, and staff; adhere to standards of excellence despite stressful conditions; communicate effectively and respectfully with employees, supervisors, and staff to ensure exceptional customer service; and follow all federal, state, and local laws and Company policies. Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position. These include the duties and responsibilities listed above, as well as the abilities to adhere to company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the Company’s reputation. Pursuant to the Los Angeles County Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.
USA, CA, Cupertino - 173,900.00 - 235,200.00 USD annually
USA, CA, San Diego - 151,200.00 - 204,600.00 USD annually
USA, TX, Austin - 151,200.00 - 204,600.00 USD annually
USA, WA, Seattle - 151,200.00 - 204,600.00 USD annually
How we rate this
Sr. Systems Development Engineer, Trainium Systems at Amazon rates 90 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.
Builds AI. The job is building AI systems.
- ●●●● Builds AI80 to 100
- ●●●○ Works on AI60 to 79
- ●●○○ Uses AI40 to 59
- ●○○○ Little AI0 to 39
Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.
Prepare for this job
Built from this posting and from similar jobs on Level.
Skills and AI tools this role asks for
Questions you could be asked
- Tell me about a project where systems engineering was part of your work. What did you do?
- Tell me about a project where distributed systems was part of your work. What did you do?
- Tell me about a project where software architecture was part of your work. What did you do?
- Tell me about a project where infrastructure was part of your work. What did you do?
- Walk me through how you've used Trainium in your day-to-day work.
Adapt your resume
- List these exact terms on your resume: Systems Engineering, Distributed Systems, Software Architecture, Infrastructure, and Trainium. An applicant tracking system matches the wording, not the idea.
- Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
- Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.
Want an expert to read your CV for this job?
Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.
Get new software engineering jobs (Builds AI ●●●●) by email
One email a week with the new software engineering jobs (Builds AI ●●●●), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.
Free. One email a week. Unsubscribe in one click.
Similar roles
Software Engineering roles that build AI, at other companies.
What kind of AI work fits you?
Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.
Find my next step