Member of Technical Staff (Search Quality Analyst)

Perplexity is hiring a Member of Technical Staff (Search Quality Analyst) in Belgrade, Serbia. Level rates it ; you can apply on Perplexity.

Level's read on this job

Builds AI ●●●● is how Level rates this role: the job is building AI systems.
Some of the work can be done remotely, according to the posting.
The posting mentions Databricks, Python and SQL among its tools and skills.
Level classes it as a senior role.
Perplexity has 130 other open roles on Level.

AI in this role

Member of Technical Staff focused on search quality, building LLM evaluation pipelines, and improving search systems.

databrickspythonsqlapache-spark

search-qualitydata-analysisllm-as-a-judgemetrics-designa-b-testing

Perplexity is looking for an experienced analyst to help us build and improve our core search technologies. You'll work at the intersection of data analysis and engineering - designing metrics, building data pipelines, and improving the quality of our search and answer systems.

This role is hybrid in Belgrade, London or Berlin.

Responsibilities

Find and diagnose quality issues in our search pipeline
Design metrics from scratch to track and measure search quality
Build datasets for model training, including LLM-as-a-judge labeling pipelines
Improve search snippet quality and page selection algorithms for indexing
Design and analyze A/B experiments to validate improvements

Qualifications

4+ years of experience as a data analyst or in a related role
Strong coding skills — expected to write production-grade code at a mid-level backend engineer level
Proficiency with SQL and Python
Demonstrated hands-on experience with at least one of the following:
Designing metrics from scratch (not just analyzing existing A/B experiments)
Building labeling pipelines using LLM-as-a-judge
Training ML models that shipped to production with measurable metric improvements
Designing evals with known ground truth (e.g. SimpleQA, BrowseComp) or driving meaningful improvements on such evals

Nice to have

Experience working on search-related products
Knowledge of statistics and A/B experiment design
Experience with Apache Spark or Databricks

How we rate this

Member of Technical Staff (Search Quality Analyst) at Perplexity rates 85 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.

Classification

Builds AI. The job is building AI systems.

●●●● Builds AI80 to 100
●●●○ Works on AI60 to 79
●●○○ Uses AI40 to 59
●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.

Key facts

Member of Technical Staff (Search Quality Analyst) is an open job at Perplexity.
Level rates this role AI Level 4 of 4: Builds AI.
The role is based in Belgrade and is marked remote.
The job was posted on September 2, 2026.
Seniority: senior.
Applications are handled by Perplexity. Apply on Perplexity

Prepare for this job

Built from this posting and from similar jobs on Level.

Skills and AI tools this role asks for

Search QualityData AnalysisLLM As A JudgeMetrics DesignA B TestingDatabricksPythonSQL

Questions you could be asked

Tell me about a project where search quality was part of your work. What did you do?
Tell me about a project where data analysis was part of your work. What did you do?
Tell me about a project where llm as a judge was part of your work. What did you do?
Tell me about a project where metrics design was part of your work. What did you do?
Tell me about a project where a b testing was part of your work. What did you do?

Adapt your resume

List these exact terms on your resume: Search Quality, Data Analysis, LLM As A Judge, Metrics Design, and A B Testing. An applicant tracking system matches the wording, not the idea.
Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
Lead with what you built, trained or shipped. This role is judged on the AI system itself, not the tools around it.

Want an expert to read your CV for this job?

Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.

Get a free CV review Get an express CV review Check your resume with the ATS resume checker

Get new remote AI jobs (Builds AI ●●●●) by email

One email a week with the new remote AI jobs (Builds AI ●●●●), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free. One email a week. Unsubscribe in one click.