AI Infrastructure System Engineer Bangalore

Together AI is hiring an AI Infrastructure System Engineer Bangalore in Bengaluru, India. Level rates it ; you can apply on Together AI.

Level's read on this job

On Level's scale, this role is Builds AI ●●●●: the job is building AI systems.
Tools and skills the posting cites include Cursor, Elevenlabs and Decagon.
Level classifies it as a mid-level role.
Level lists 83 other open roles at Together AI.

AI in this role

cursorelevenlabsdecagon

ai-agentsfine-tuning

About the Role

At Together AI, you’ll build and operate one of the world’s largest GPU fleets used for frontier model training and inference. This isn’t a traditional infrastructure role—we’re looking for engineers who love building systems, automating everything, and solving problems at massive scale.

If you enjoy writing software more than clicking dashboards, obsess over eliminating manual work, and want to build infrastructure that manages tens of thousands of GPUs autonomously, we’d love to talk.

Responsibilities

Design and build fleet automation systems that provision, validate, deploy, upgrade, repair, and retire GPU clusters with minimal human intervention.
Build AI Infrastructure Agents that automate deployment, root-cause failures, incident triage, and autonomous remediation.
Develop Fleet Intelligence platforms that continuously monitor hardware health, firmware, networking, storage, thermals, and workload performance to predict failures before they impact customers.
Build software that maximizes GPU availability, utilization, performance, and reliability across thousands of accelerators.
Create automated validation systems for GPUs, InfiniBand/RoCE fabrics, NVLink/NVSwitch, storage, and distributed AI workloads.
Build internal platforms and developer tools that allow infrastructure to be managed through software—not manual operations.
Continuously improve deployment velocity, reliability, and operational efficiency through automation.
Partner closely with hardware, networking, platform, and AI teams to push the limits of AI infrastructure.

Requirements

3+ years building distributed systems, infrastructure platforms, or large-scale backend software.
Strong software engineering skills in Python, Go, or Rust.
Experience building platforms, automation systems, or developer infrastructure.
Experience with Linux, Kubernetes, Terraform, Ansible, or similar infrastructure technologies.
Strong systems thinking with the ability to understand problems across hardware and software.
A passion for solving complex infrastructure challenges through software.
An automation-first mindset—if a task is repeated, your instinct is to build a system to eliminate it.

Bonus Experience

GPU infrastructure, CUDA, NCCL, NVLink/NVSwitch
InfiniBand or RoCE networking
Bare-metal provisioning and lifecycle management
Large-scale AI training or inference clusters
Hardware health monitoring and predictive failure detection
Distributed storage systems
AI agents and autonomous infrastructure operations

About Together AI

Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.

Equal Opportunity

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Please see our privacy policy at https://www.together.ai/privacy

How we rate this

AI Infrastructure System Engineer Bangalore at Together AI rates 98 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.

Classification

Builds AI. The job is building AI systems.

●●●● Builds AI80 to 100
●●●○ Works on AI60 to 79
●●○○ Uses AI40 to 59
●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.

Key facts

AI Infrastructure System Engineer Bangalore is an open job at Together AI.
Level rates this role AI Level 4 of 4: Builds AI.
The role is based in Bangalore, India.
The job was posted on July 13, 2026.
Seniority: mid.
Applications are handled by Together AI. Apply on Together AI

Prepare for this job

Built from this posting and from similar jobs on Level.

Skills and AI tools this role asks for

AI agentsFine-tuningCursorElevenlabsDecagon

Questions you could be asked

How do you decide when an AI agent can act on its own versus asking for approval first?
Walk me through fine-tuning a model: what data did you use, and how did you check the result?
What are the limits of Cursor that you've run into, and how did you work around them?
What's a project where you used Elevenlabs hands-on?
Walk me through how you've used Decagon in your day-to-day work.

Adapt your resume

List these exact terms on your resume: AI agents, Fine-tuning, Cursor, Elevenlabs, and Decagon. An applicant tracking system matches the wording, not the idea.
Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.

Want an expert to read your CV for this job?

Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.

Get a free CV review Get an express CV review Check your resume with the ATS resume checker

Get new software engineering jobs (Builds AI ●●●●) by email

One email a week with the new software engineering jobs (Builds AI ●●●●), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free. One email a week. Unsubscribe in one click.