Senior Machine Learning Engineer - Orchestration - Applied AI & LLMs (x/f/m)

Doctolib is hiring a Senior Machine Learning Engineer - Orchestration - Applied AI & LLMs (x/f/m) in Paris, France. Level rates it ; you can apply on Doctolib.

Level's read on this job

Builds AI ●●●● is how Level rates this machine learning engineer role: the job is building AI systems.
Tools and skills the posting cites include Claude, Llama and Python.
Level classifies it as a senior role.
Level lists 155 other open roles at Doctolib.

AI in this role

Define and own the evaluation strategy and automated pipelines for agentic healthcare AI systems and health assistants.

claudellamapythongpt

ai-agentsai-evaluationmachine-learningllmsevaluationagentic-systems

Set a new pulse for healthcare!

We are looking for a Senior Machine Learning Engineer to join the Orchestration team in AI Health Assistant. Your mission will be to ensure our AI Health Companion behaves safely, reliably, and helpfully — for millions of patients and practitioners. You will work in a cross-functional team building robust evaluation pipelines for agentic AI systems, contributing directly to improving the quality and safety of AI-powered healthcare. Working in the tech team at Doctolib means building innovative products and features to improve the daily lives of care teams and patients.

What you'll do

Your responsibilities include but are not limited to:

Define and own the evaluation strategy for our AI agentic system — metrics, protocols, datasets, and tooling
Implement and maintain automated evaluation pipelines to monitor model quality, safety, and alignment across iterations
Run systematic experiments to assess reasoning, factuality, robustness, and user experience
Collaborate closely with model developers and research scientists to provide insights and drive iterative improvement
Contribute to research and internal knowledge sharing on LLM evaluation methodologies and best practices

Life at Doctolib Tech

Our solutions are built on a single fully cloud-native platform that supports web and mobile app interfaces, multiple languages, and is adapted to country and healthcare specialty requirements.
Our stack is composed of Rails, TypeScript, Java, Python, Kotlin, Swift, and React Native.
We leverage AI ethically across our products to empower patients and health professionals. Discover our AI vision here.

Want to learn more about our tech culture and environment? Visit the Doctolib Tech site.

Who you are

Before you read on: if you don't have the exact profile described below, but you feel this job description matches your skill set, we still encourage you to apply. You'll be a great fit if you:

Hold an MSc or PhD in Computer Science, Machine Learning, Data Science, or a related field
Have 7+ years of hands-on experience working with large language models (e.g., GPT, Claude, Llama, or BERT-like architectures)
Have proven experience evaluating agentic or reasoning systems (e.g., autonomous agents, tool-using LLMs, dialogue systems, or task-oriented assistants)
Have a strong track record in experiment design, metric definition, and evaluation automation
Are able to bridge research and production, influencing both modeling and product decisions
Have excellent communication skills, a collaborative mindset, and are fluent in English

It would be fantastic if you:

Have experience in the clinical or medical domain and sensitivity to ethical or regulatory challenges in healthcare AI

What we offer

Free comprehensive health insurance (basic package) for you and your children
25 days of paid vacation per year, plus up to 14 days of RTT
Free mental health and coaching services through our partner Moka.care
Work from abroad for up to 10 days per year thanks to our flexibility days policy
Lunch vouchers (Swile card) worth €8.50 per working day, with €4.50 covered by Doctolib
A subsidy from the work council to refund part of the membership to a sport club or a creative class
50% reimbursement of your public transport subscription
ParentCare Program: Enjoy full salary coverage (100%) during your first month of birth leave, and 75% during the second, covered by Doctolib
Enrollment in Doctolib's long-term employee value sharing plan called DoctoGrowth
For caregivers and workers with disabilities, a package including an adaptation of the remote policy, extra days off for medical reasons, and psychological support
Relocation support in case of international mobility
Access to the best AI tools for coding, development and dedicated training

Our interview process

Recruiter Interview
Technical Deep Dive
Data System Design
Behavioral Interview
At least one reference check

We want your experience to be clear, respectful, and transparent. Learn more about our hiring process on our candidate experience page.

Job details

Permanent position
Full-time
Hybrid in our Levallois-Perret office (up to 2 remote days per week)
Start date: as soon as possible

We welcome everyone

At Doctolib, we are committed to improving access to healthcare for everyone. This translates into our recruitment process. We evaluate candidates based solely on qualifications and motivation, without any form of discrimination. The more diverse ideas are heard, the more our product will truly improve healthcare for all. You are welcome to apply to Doctolib, regardless of your gender, religion, age, sexual orientation, ethnicity, or disability. To ensure equal opportunities, we invite you to exclude personal information (e.g., pictures, age) from your applications. If you require any accommodation, please let us know for support during the hiring process. Join us in building the healthcare we all dream of!

Your data privacy

All information provided is processed by Doctolib for application management. For data processing details, click here: France. Please contact hr.dataprivacy(at)doctolib.com for inquiries or to exercise your rights.

How we rate this

Senior Machine Learning Engineer - Orchestration - Applied AI & LLMs (x/f/m) at Doctolib rates 85 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.

Classification

Builds AI. The job is building AI systems.

●●●● Builds AI80 to 100
●●●○ Works on AI60 to 79
●●○○ Uses AI40 to 59
●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.

Key facts

Senior Machine Learning Engineer - Orchestration - Applied AI & LLMs (x/f/m) is an open job at Doctolib.
Level rates this role AI Level 4 of 4: Builds AI.
The role is based in Paris, Paris, France.
The job was posted on July 3, 2026.
Seniority: senior.
Applications are handled by Doctolib. Apply on Doctolib

Prepare for this job

Built from this posting and from similar jobs on Level.

Skills and AI tools this role asks for

AI agentsAI EvaluationMachine learningLLMsEvaluationAgentic SystemsClaudeLlama

Also common in machine learning engineer jobs at this level

PyTorch TensorFlow Deep learning AWS Generative AI C++

Pay

Typical machine learning engineer job: $199k

This posting doesn't list pay. Use the typical machine learning engineer figure when pay comes up. See machine learning engineer jobs

Questions you could be asked

How do you decide when an AI agent can act on its own versus asking for approval first?
How do you decide that one model's output is better than another's for a given task?
Tell me about a project where machine learning was part of your work. What did you do?
Tell me about a project where llms was part of your work. What did you do?
Tell me about a project where evaluation was part of your work. What did you do?

Adapt your resume

List these exact terms on your resume: AI agents, AI Evaluation, Machine learning, LLMs, and Evaluation. An applicant tracking system matches the wording, not the idea.
Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.

Want an expert to read your CV for this job?

Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.

Get a free CV review Get an express CV review Check your resume with the ATS resume checker

Get new machine learning engineer jobs (Builds AI ●●●●) by email

One email a week with the new machine learning engineer jobs (Builds AI ●●●●), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free. One email a week. Unsubscribe in one click.