Senior Software Engineer, AI Developer Experience
Vanta is hiring a Senior Software Engineer, AI Developer Experience for a remote role open to applicants in Canada. Level rates it ; you can apply on Vanta.
Level's read on this job
- On Level's scale, this software engineer role is Builds AI ●●●●: the job is building AI systems.
- The posting lists this role as remote.
- Tools and skills the posting cites include AI agents, developer experience and evaluation systems.
- Level lists 78 other open roles at Vanta.
- The role was posted in the last few days.
AI in this role
Build background-agent infrastructure and evaluation systems to empower engineers with AI-powered development workflows at Vanta.
At Vanta, our mission is to help businesses earn and prove trust. We believe that security should be monitored and verified continuously, and we empower companies to practice better security and prove it with ease. Vanta has a kind and talented team, and while some have prior security experience, many have been successful at Vanta without it.
Build the systems, infrastructure, and developer workflows that make AI-powered engineering more useful, reliable, and easy for Vanta engineers to adopt.
The AI DX team is building the foundations for a more autonomous software development experience at Vanta, including background agents, agent evaluation systems, and tooling that helps engineers work more effectively with AI. The team combines software engineering, developer experience, AI systems, and product thinking to turn emerging capabilities into reliable workflows used in real engineering work.
As a Senior Software Engineer on the AI DX team, you’ll help shape these systems from the ground up. You’ll work directly with engineers to understand friction in their workflows, build practical solutions, and use evidence from real usage to continuously improve quality, reliability, and adoption.
Visit our Vanta Engineering Blog to learn more about what our team is working on!
What you’ll do as a Senior Software Engineer at Vanta:
Plan, build, and improve background-agent infrastructure that enables reliable, trackable, and repeatable automated engineering work.
Develop systems for evaluating agent performance using task outcomes, quality measures, human intervention, cost, and other signals.
Create evaluation sets and feedback loops that help AI systems improve over time through regular review, tuning, and correction.
Improve how engineers discover, understand, and use shared AI tools and workflows.
Evaluate AI development tools, including code generation, AI-assisted review, and testing, against real engineering needs, and help integrate the ones that prove useful.
Develop visibility into which tools and workflows are useful, where they create friction, and how they should evolve.
Create clear guidance and documentation that helps engineers adopt new AI-enabled development practices.
Improve code review workflows by using feedback from real engineering work to identify and address recurring friction.
Influence the foundations of an autonomous software factory, including systems that help agents complete bounded engineering tasks safely and effectively.
Work directly with engineers and internal users to understand their needs, validate solutions, and make thoughtful product tradeoffs.
Help the team move ambiguous, dependency-heavy projects from early exploration through reliable delivery.
Raise the quality bar through thoughtful code reviews, technical design discussions, testing, observability, and attention to reliability and security.
Partner closely with engineers and cross-functional stakeholders while contributing as a dependable, collaborative teammate.
How to be successful in this role:
Substantial industry software engineering experience.
Experience taking technical leadership roles on teams or complex projects.
Experience improving AI systems, creating evaluation sets, and driving quality hill-climbing through measurement and iteration.
Strong software engineering fundamentals and experience designing, building, testing, and operating reliable systems.
Experience with developer tooling, AI-enabled workflows, agentic systems, infrastructure, or related technical domains.
A track record of working through ambiguity, decomposing complex problems, managing dependencies, and helping projects reach completion.
Experience working in rapidly scaling startups or larger companies, particularly in environments with a strong bias for action.
An action-oriented approach and a willingness to engage directly with users and contribute hands-on.
Product judgment and curiosity: you bring strong technical opinions while staying focused on building platforms that meet users where they are.
Strong written and verbal communication, with the ability to explain technical concepts clearly and create guidance that helps others adopt new tools.
A collaborative approach and a demonstrated ability to make meaningful contributions as part of a team.
Uses AI in day-to-day work to amplify their skills and strengthen their outcomes, applying it across routine workflows, actively building fluency through experimentation, and exercising sound judgment to ensure AI-assisted work meets a high quality bar.
What you can expect as a Vanta’n in Canada:
Industry-competitive salary and equity
100% covered medical, dental, and vision benefits with dependents coverage
Pension contribution
16 weeks paid Parental Leave for all new parents
Health & wellness stipend
Remote workspace, internet, and cellphone stipend
Flexible work hours and location
21 days of Vacation Time and 80 hours of Sick Leave
11 company-paid holidays
Virtual team building activities, lunch and learns, and other company-wide events!
Offices in SF, NYC, London, Dublin, Tel Aviv, and Sydney
#LI-remote
At Vanta, we are committed to hiring diverse talent of different backgrounds and as such, it is important to us to provide an inclusive work environment for all. We do not discriminate on the basis of race, gender identity, age, religion, sexual orientation, veteran or disability status, or any other protected class. As an equal opportunity employer, we encourage and welcome people of all backgrounds to apply.
About Vanta
We started in 2018, in the wake of several high-profile data breaches. Online security was only becoming more important, but we knew firsthand how hard it could be for fast-growing companies to invest the time and manpower it takes to build a solid security foundation. Vanta was inspired by a vision to restore trust in internet businesses by enabling companies to improve and prove their security. From our early days automating security monitoring for compliance standards like SOC 2, HIPAA and ISO 27001 to creating the world's leading Trust Management Platform, our vision remains unchanged.
Now more than ever, making security continuous—not just a point-in-time check— is essential. Thousands of companies rely on Vanta to build, maintain and demonstrate their trust— all in a way that's real-time and transparent.
Referral Instructions
If you are being referred for the role, please contact that person to apply on your behalf.
How we rate this
Senior Software Engineer, AI Developer Experience at Vanta rates 85 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.
Builds AI. The job is building AI systems.
- ●●●● Builds AI80 to 100
- ●●●○ Works on AI60 to 79
- ●●○○ Uses AI40 to 59
- ●○○○ Little AI0 to 39
Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.
Prepare for this job
Built from this posting and from similar jobs on Level.
Skills and AI tools this role asks for
Pay
- Typical software engineer job
- $167k
This posting doesn't list pay. Use the typical software engineer figure when pay comes up. See software engineer jobs
Questions you could be asked
- How do you decide when an AI agent can act on its own versus asking for approval first?
- Tell me about a project where developer experience was part of your work. What did you do?
- Tell me about a project where evaluation systems was part of your work. What did you do?
- Tell me about a project where software engineering was part of your work. What did you do?
- Tell me about a project where code review was part of your work. What did you do?
Adapt your resume
- List these exact terms on your resume: AI agents, Developer Experience, Evaluation Systems, Software Engineering, and Code Review. An applicant tracking system matches the wording, not the idea.
- Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
- Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.
Want an expert to read your CV for this job?
Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.
Get new remote software engineer jobs (Builds AI ●●●●) by email
One email a week with the new remote software engineer jobs (Builds AI ●●●●), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.
Free. One email a week. Unsubscribe in one click.
Similar roles
Software Engineering roles that build AI, at other companies.
What kind of AI work fits you?
Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.
Find my next step