# Staff AI Product Engineer at Nscale

Nscale is hiring a Staff AI Product Engineer in Houston, New York and San Francisco, United States. It pays $220k-$330k a year and Level rates it Builds AI ●●●●; you can [apply on Nscale](https://jobsbylevel.com/go/cb607b70-8601-4923-a593-005fe97d2c5c?utm_medium=ai-agent).

AI Level 4, AI centrality 95 out of 100. Houston; New York; San Francisco; Seattle.

## Details

- Company: [Nscale](https://jobsbylevel.com/companies/nscale)
- AI level: AI Level 4 (score 95 out of 100)
- Location: Houston; New York; San Francisco; Seattle
- Salary: $220k-$330k
- Posted: September 15, 2026
- Apply: https://jobsbylevel.com/go/cb607b70-8601-4923-a593-005fe97d2c5c?utm_medium=ai-agent
- Free CV review for this job: https://jobsbylevel.com/cv-review?job=staff-ai-product-engineer-at-nscale-fa9635

## Prepare for this job

The job page lists 5 likely interview questions for this offer, with a CV checklist: https://jobsbylevel.com/jobs/staff-ai-product-engineer-at-nscale-fa9635

## Description

About Nscale Nscale is taking on the hyperscalers by building a vertically integrated GenAI cloud platform. We own the data centers, software, and applications that power today's AI stack using sustainable technology solutions. We thrive on a culture of relentless innovation, ownership, and accountability, where every team member takes pride in their work and drives it with excellence and urgency. As a Nscaler, you'll build trust through openness and transparency, where everyone is inspired to do their best work. Collaboration is key, and we work together swiftly and respectfully, embracing adaptability and resilience in all we do. About the Role Nscale is looking for a Staff AI Engineer (Specialised) to set technical direction for part of the inference platform at the core of our AI cloud: dedicated and serverless inference, bring-your-own-model deployments, and the post-training services that sit alongside them. You’ll be the technical authority for one or more focus areas, working across the teams that build serving, post-training, and platform. Your decisions shape the latency, throughput, and cost of the tokens Nscale serves, and whether we can say with evidence that the models we run are the right ones. You’ll take on open questions, such as how KV cache should move between GPUs, nodes, and storage, or where hand-written kernels beat the open-source defaults, and your answers become the standards others build on. We run state-of-the-art GPU systems, and much of the open-source ecosystem hasn’t caught up with them yet. This role is for people who want to close that gap hands-on while growing the engineers around them. How We Work Dog years. We move quickly and compress a lot of learning into a short time. Don’t let perfect be the enemy of good. Ship, measure, iterate. Be relentless. Own the problem end to end and see it through. One team, one mission. Outcomes over process, and no “not my job”. Focus Areas We’re hiring for depth. You don’t need all of these; we want people who are exceptional in one or more , and we’re deliberately hiring people with different focus areas: KV cache offloading and inference performance: KV cache orchestration across GPU, host memory, and storage; cross-instance cache sharing (LMCache or similar); KV-aware routing; disaggregated prefill/decode; speculative decoding; quantisation (FP8, NVFP4, INT8/4); MoE serving GPU kernels and performance: writing and tuning kernels in CUDA, Triton, CUTLASS or ROCm; attention, MoE and GEMM optimisation; multi-GPU and multi-node communication on NVLink-scale systems Evals and benchmarking: designing evaluation frameworks, turning customer requirements into custom benchmarks, and producing objective model comparisons we can stand behind Post-training and RL: fine-tuning and preference optimisation as a service; RL for LLMs (PPO/GRPO-style, reward modelling, multi-turn and tool-use RL); rollout generation, weight synchronisation, and sharing infrastructure between inference and training Serving platform and APIs: serving engines (vLLM, SGLang, TensorRT-LLM), model onboarding, and OpenAI-compatible APIs that other engineers and customers build on Responsibilities Set technical direction for your focus areas and turn it into work that multiple teams can deliver Lead the resolution of systemic performance and reliability problems across the serving stack, from kernel bottlenecks to fleet-level capacity and multi-tenant isolation Own the trade-offs between cost, latency, throughput, and model quality in your area, and back them with measurement Build reusable frameworks, benchmarks, and tooling that make other AI engineers at Nscale more effective Evaluate emerging serving engines, kernel libraries, RL frameworks, and accelerators, and make clear build/adopt/contribute recommendations Coach and grow engineers across teams, and raise engineering quality broadly Work with research, product, and infrastructure leadership so the platform tracks customer demand

The description is cut here. Read the full offer: https://jobsbylevel.com/jobs/staff-ai-product-engineer-at-nscale-fa9635

Source: https://jobsbylevel.com/jobs/staff-ai-product-engineer-at-nscale-fa9635

## Cite this page

Level. https://jobsbylevel.com/jobs/staff-ai-product-engineer-at-nscale-fa9635.

Get job alerts: https://jobsbylevel.com/newsletter