DecagonResearch Engineer, Audio and Speech
Decagon is hiring a Research Engineer, Audio and Speech in San Francisco, United States. It pays $200k-$400k a year and Level rates it ; you can apply on Decagon.
Level's read on this job
- Level rates this role Builds AI ●●●●: the job is building AI systems.
- Tools and skills the posting cites include PyTorch, Python and AI agents.
- Level classifies it as a mid-level role.
- Level lists 133 other open roles at Decagon.
AI in this role
Design and build real-time voice and audio AI agents, advancing multimodal and full-duplex conversational systems from idea to production.
About Decagon
Decagon is the leading conversational AI platform empowering every brand to deliver concierge customer experiences.
Our technology enables industry-defining enterprises like Avis Budget Group, Block’s Cash App and Square, Chime, Oura Health, and Hunter Douglas to deploy AI agents that power personalized, deeply satisfying interactions across voice, chat, email, SMS, and every other channel.
We’re building a future where customer experiences are being redefined from support tickets and hold music to faster resolutions, richer conversations, and deeper relationships. We’re proud to be backed by world-class investors who share that vision, including a16z, Accel, Bain Capital Ventures, Coatue, and Index Ventures, along with many others.
We’re an in-office company, driven by a shared commitment to excellence and velocity. Our values — Just Get It Done, Invent What Customers Want, Winner’s Mindset, and The Polymath Principle — shape how we work and grow as a team.
About the Team
Read more about the Speech Research Team's work:
The Research team develops the model and decision-making stack that powers Decagon’s conversational agents for enterprise support. We research, adapt, and implement state-of-the-art techniques in model training, prompting, orchestration, and evaluation in order to make our agents more accurate, robust, and efficient in real-world deployments.
Our goal is to push the frontier of applied conversational AI: agents that reliably understand nuanced intent, track long context, and take the right actions under uncertainty. We measure success the way customers feel it: higher resolution rates, better user satisfaction, and consistent behavior at scale.
About the Role
As a Research Engineer focused on Audio and Speech, you’ll be responsible for building the models and agent harnesses that power Decagon’s real-time voice agents and taking them all the way from idea to production. Your work will advance multimodal and full-duplex systems that can listen, reason, speak, and respond naturally in real time.
We’re looking for strong engineers who want to build the next generation of AI voice agents. People here own their work end-to-end, ship real improvements, and are trusted to make high-impact technical decisions.
In this role, you will
Design and build next-generation agent harnesses optimized for streaming speech, turn-taking, interruptions, overlapping speech, and continuous interaction
Research and train multimodal and full-duplex models that jointly understand audio, reason, and generate speech
Improve speech recognition, voice activity detection, endpointing, and speech generation across diverse speakers, environments, domains, and languages
Build evaluations and use production calls to ship measurable improvements in accuracy, latency, naturalness, and task outcomes
Optimize end-to-end inference for responsiveness, throughput, stability, and cost, partnering with Voice Platform and Infrastructure teams to deploy at scale
Your background looks something like this
2+ years of experience in speech, audio ML, multimodal ML, or production machine learning
Experience developing or adapting autoregressive, diffusion, flow-matching, or codec-based speech models
Hands-on experience with streaming agent systems, low-latency inference, production model serving, and evaluation on real-world audio
Fluency in Python and a modern deep-learning framework such as PyTorch, with strong foundations in machine learning and signal processing
A track record of taking research ideas from prototype to reliable, measurable production impact
Even better if you have
Familiarity with speech-to-speech or full-duplex models
Experience with telephony, multilingual speech, noisy-channel robustness, speaker adaptation, or expressive speech generation
Compensation
$200K – $400K + Offers Equity
Benefits
We proudly offer the following benefits for our full-time employees:
Medical, Dental, and Vision benefits for you and your family
Life Insurance and Disability Benefits
Retirement Plan (e.g., 401K, pension)
Parental Leave
Fertility and family building benefits through Carrot
Monthly stipend to support your wellness, lifestyle, and work-life balance
Daily lunches and snacks in the office to keep you at your best
Take what you need vacation policy (subject to local requirements; UK employees receive 25 days of statutory leave)
These benefits are described in more detail in Decagon’s policies, may vary by location, and can change at any time according to applicable compensation and benefits plans.
How we rate this
Research Engineer, Audio and Speech at Decagon rates 95 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.
Builds AI. The job is building AI systems.
- ●●●● Builds AI80 to 100
- ●●●○ Works on AI60 to 79
- ●●○○ Uses AI40 to 59
- ●○○○ Little AI0 to 39
Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.
Prepare for this job
Built from this posting and from similar jobs on Level.
Skills and AI tools this role asks for
Questions you could be asked
- How do you decide when an AI agent can act on its own versus asking for approval first?
- What have you built with speech recognition or text-to-speech, and where did it break?
- Tell me about a project where machine learning was part of your work. What did you do?
- Tell me about a project where audio processing was part of your work. What did you do?
- Tell me about a project where speech recognition was part of your work. What did you do?
Adapt your resume
- List these exact terms on your resume: AI agents, Speech, Machine learning, Audio Processing, and Speech Recognition. An applicant tracking system matches the wording, not the idea.
- Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
- Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.
Want an expert to read your CV for this job?
Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.
Get new research jobs (Builds AI ●●●●) by email
One email a week with the new research jobs (Builds AI ●●●●), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.
Free. One email a week. Unsubscribe in one click.
Similar roles
Research roles that build AI, at other companies.
What kind of AI work fits you?
Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.
Find my next step