Skip to content

Bloomberg

Senior Software Engineer - AI Inference

Bloomberg is hiring a Senior Software Engineer - AI Inference in New York, United States. It pays $160k-$240k a year and Level rates it ; you can apply on Bloomberg.

Level's read on this job

  • Level rates this software engineer role Builds AI ●●●●: the job is building AI systems.
  • Pay on this posting starts in line with the typical software engineer posting on Level.
  • The posting mentions vLLM, PyTorch and Kserve among its tools and skills.
  • Level classes it as a senior role.
  • Bloomberg has 336 other open roles on Level.

AI in this role

Design and build scalable core infrastructure and inference platforms for hosting and serving machine learning models.

vLLMPyTorchKserveTensorrtOnnxTritonCUDANcclKubernetes
software-engineeringdistributed-systemsinference-optimizationsystem-designperformance-tuning
Our team: Join the team that is building the core infrastructure for AI at Bloomberg. The Bloomberg AI Inference Platform provides production-grade managed infrastructure for hosting, deploying, and serving all machine learning models, both predictive and cutting-edge generative models. We abstract away infrastructure complexity, empowering engineering teams to focus on creating intelligent applications with guaranteed scalability, performance, and governance. Our platform is built on the open-source KServe project, and the CNCS AI Inference team is a primary contributor to its development.
We'll trust you to:
  • Design and build scalable infrastructure for both online and offline inference workloads.
  • Lead integration of high-performance inference runtimes and serving frameworks, including TensorRT, vLLM, ONNX, and Triton.
  • Drive architecture and technical decisions across Bloomberg’s inference platform, balancing latency, throughput, reliability, and cost.
  • Partner across engineering teams to improve model deployment, observability, and production performance.
  • Mentor junior engineers on system design, debugging, and performance optimization.

You'll need to have:
  • 5+ years of professional software engineering experience.
  • Experience designing, building, and operating production distributed systems.
  • Strong systems intuition and a track record of debugging and optimizing performance-critical services.
  • Ability to own problems end-to-end and quickly ramp up in unfamiliar technical areas.
  • 4+ years of demonstrated experience working with an object-oriented programming language.
  • A degree in Computer Science, Electrical Engineering, or equivalent practical experience.

We'd love to see:
  • Experience deploying and operating machine learning systems at scale.
  • Experience with inference optimization techniques such as batching, caching, request scheduling, or memory-aware serving.
  • Familiarity with PyTorch and GPU software stacks such as CUDA and NCCL.
  • Exposure to high-performance interconnects and distributed computing technologies such as NVLink, InfiniBand, or MPI.
  • Experience with Kubernetes and cloud-native infrastructure.
  • Experience with load balancing, request routing, or traffic management systems.

Representative projects:
  • Autoscaling a heterogeneous compute fleet to match supply and demand aross diverse inference workloads.
  • Building production-grade deployment pipelines to safely roll out new models to millions of users.
  • Developing new inference capabilities such as structured sampling, prompt caching, and advanced serving optimizations.
  • Analyzing observability data from real production workloads to improve latency, throughput, and resource efficiency.
Salary Range = 160,000 - 240,000 USD Annual + Benefits + Bonus
The referenced salary range is based on the Company's good faith belief at the time of posting. Actual compensation may vary based on factors such as geographic location, work experience, market conditions, education/training and skill level.


We offer one of the most comprehensive and generous benefits plans available and offer a range of total rewards that may include merit increases, incentive compensation (exempt roles only), paid holidays, paid time off, medical, dental, vision, short and long term disability benefits, 401(k) +match, life insurance, and various wellness programs, among others. The Company does not provide benefits directly to contingent workers/contractors and interns.
Discover what makes Bloomberg unique - watch our podcast series for an inside look at our culture, values, and the people behind our success.

How we rate this

Senior Software Engineer - AI Inference at Bloomberg rates 95 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.

Classification

Builds AI. The job is building AI systems.

  1. ●●●● Builds AI80 to 100
  2. ●●●○ Works on AI60 to 79
  3. ●●○○ Uses AI40 to 59
  4. ●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.

Prepare for this job

Built from this posting and from similar jobs on Level.

Skills and AI tools this role asks for

Software EngineeringDistributed SystemsInference OptimizationSystem DesignPerformance TuningvLLMPyTorchKserve

Also common in software engineer jobs at this level

Pay

This job starts at
$160k
Typical software engineer job
$167k

This posting starts close to the typical software engineer job. See software engineer jobs

Questions you could be asked

  1. Tell me about a project where software engineering was part of your work. What did you do?
  2. Tell me about a project where distributed systems was part of your work. What did you do?
  3. Tell me about a project where inference optimization was part of your work. What did you do?
  4. Tell me about a project where system design was part of your work. What did you do?
  5. Tell me about a project where performance tuning was part of your work. What did you do?

Adapt your resume

  • List these exact terms on your resume: Software Engineering, Distributed Systems, Inference Optimization, System Design, and Performance Tuning. An applicant tracking system matches the wording, not the idea.
  • Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
  • Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.

Get your CV reviewed before you apply

Send your CV and the role you want next. Your review arrives by email within 3 days.

Get new software engineer jobs (Builds AI ●●●●) by email

One email a week, each job rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free.

Similar roles

Software Engineering roles that build AI, at other companies.

What kind of AI work fits you?

Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.

Find my next step

More jobs at Bloomberg

More software engineer jobs

Related searches

Same AI level

Jobs by city