Machine Learning Engineer - Inference

Together AI is hiring a Machine Learning Engineer - Inference in San Francisco, United States. It pays $200k-$300k a year and Level rates it ; you can apply on Together AI.

Level's read on this job

Builds AI ●●●● is how Level rates this machine learning engineer role: the job is building AI systems.
Pay on this posting starts in line with the typical machine learning engineer posting on Level.
The posting mentions vLLM, PyTorch and Cursor among its tools and skills.
Level classes it as a mid-level role.
Together AI has 84 other open roles on Level.

AI in this role

vllmpytorchcursorelevenlabsdecagon

fine-tuning

About the Role

Together AI is seeking a Machine Learning Engineer to join our Inference Engine team, focusing on optimizing and enhancing the performance of our AI inference systems. This role involves working with state-of-the-art large language models models and ensuring they run efficiently and effectively at scale. If you are passionate about AI inference, PyTorch, and developing high-performance systems, we want to hear from you. This position offers the chance to collaborate closely with AI researchers and engineers to create cutting-edge AI solutions. Join us in shaping the future at Together AI!

Responsibilities

Design and build the production systems that power the Together AI inference engine, enabling reliability and performance at scale.
Develop and optimize runtime inference services for large-scale AI applications.
Collaborate with researchers, engineers, product managers, and designers to bring new features and research capabilities to the world.
Conduct design and code reviews to ensure high standards of quality.
Create services, tools, and developer documentation to support the inference engine.
Implement robust and fault-tolerant systems for data ingestion and processing.

Requirements

3+ years of experience writing high-performance, well-tested, production-quality code.
Proficiency with Python and PyTorch.
Demonstrated experience in building high performance libraries and tooling.
Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, storage, performance, and scale.
Preferred: Knowledge of existing AI inference systems such as TGI, vLLM, TensorRT-LLM, Optimum
Preferred: Knowledge of AI inference techniques such as speculative decoding.
Preferred: Knowledge of CUDA/Triton programming.
Nice to have: Knowledge of Rust, Cython and compilers.

About Together AI

Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.

Compensation

We offer competitive compensation, startup equity, health insurance, and other competitive benefits. The US base salary range for this full-time position is $200,000 - $300,000 + equity + benefits. Our salary ranges are determined by location, level, and role. Individual compensation will be determined by experience, skills, and job-related knowledge.

Equal Opportunity

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunities to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Please see our privacy policy at https://www.together.ai/privacy

How we rate this

Machine Learning Engineer - Inference at Together AI rates 99 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.

Classification

Builds AI. The job is building AI systems.

●●●● Builds AI80 to 100
●●●○ Works on AI60 to 79
●●○○ Uses AI40 to 59
●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.

Key facts

Machine Learning Engineer - Inference is an open job at Together AI.
Level rates this role AI Level 4 of 4: Builds AI.
The listed salary is $200k-$300k a year.
The role is based in San Francisco.
The job was posted on June 6, 2024.
Seniority: mid.
Applications are handled by Together AI. Apply on Together AI

Prepare for this job

Built from this posting and from similar jobs on Level.

Skills and AI tools this role asks for

Fine-tuningvLLMPyTorchCursorElevenlabsDecagon

Also common in machine learning engineer jobs at this level

TensorFlow Deep learning AWS Generative AI C++Kubernetes

Pay

This job starts at: $200k
Typical machine learning engineer job: $199k

This posting starts close to the typical machine learning engineer job. See machine learning engineer jobs

Questions you could be asked

Walk me through fine-tuning a model: what data did you use, and how did you check the result?
Walk me through how you've used vLLM in your day-to-day work.
What are the limits of PyTorch that you've run into, and how did you work around them?
What's a project where you used Cursor hands-on?
Walk me through how you've used Elevenlabs in your day-to-day work.

Adapt your resume

List these exact terms on your resume: Fine-tuning, vLLM, PyTorch, Cursor, and Elevenlabs. An applicant tracking system matches the wording, not the idea.
Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.

Want an expert to read your CV for this job?

Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.

Get a free CV review Get an express CV review Check your resume with the ATS resume checker

Get new machine learning engineer jobs (Builds AI ●●●●) by email

One email a week with the new machine learning engineer jobs (Builds AI ●●●●), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free. One email a week. Unsubscribe in one click.