Member of Technical Staff (Web Access Engineer, Crawler)

Perplexity is hiring a Member of Technical Staff (Web Access Engineer, Crawler) for a remote role open to applicants in United Kingdom. Level rates it ; you can apply on Perplexity.

Level's read on this job

Level rates this role Builds AI ●●●●: the job is building AI systems.
The posting lists this role as remote.
Level classifies it as a lead role.
Level lists 134 other open roles at Perplexity.
The role was posted in the last few days.

AI in this role

Perplexity is hiring a Web Access Engineer for the Crawler team to build reliable fetching and rendering systems for AI-powered search. You'll work on freely available public web content, within site permissions and approved access policies, and develop the infrastructure that supplies content to downstream indexing and retrieval systems.

Your focus will be diagnosing network errors, page-loading issues, and incomplete rendering. You'll trace problems from network protocols through client-side execution, distinguish technical failures from intentional access restrictions, and turn findings into production fixes that improve content quality and efficiency at scale.

Responsibilities

Own fetching and rendering improvements end to end, from investigation through implementation and production rollout, partnering with crawler engineers and SREs.
Investigate fetch and rendering failures across DNS, TLS, HTTP, caching, redirects, and client-side rendering, and classify root causes.
Improve page loading, completion detection, and content capture to preserve relevant content and handle dynamic pages reliably.
Ensure the crawler respects site-imposed access restrictions and rate limits, identifies itself clearly, and retries only transient technical errors.
Build tools to reproduce failures, compare crawler output against expected page content, and turn findings into regression tests.
Define metrics for fetching reliability, content completeness, and error classification, and use them to prioritize improvements.

Qualifications

5+ years of software engineering experience in web infrastructure, browsers, CDNs, or crawling, with strong programming skills in at least one of Go, Python, Rust, or C++.
Deep understanding of HTTP/1.1, HTTP/2, and HTTP/3, including status semantics, caching, conditional requests, content encoding, and redirects; strong knowledge of TLS, DNS, cookies, and sessions.
Understanding of CDN caching and content delivery, HTTP error handling, retry backoff, and server-defined rate limits.
Knowledge of page lifecycle, network idle, service workers, client-side rendering, and shadow DOM; experience debugging JavaScript and fluency with Chrome DevTools, CDP, HAR files, curl, and packet captures.
Comfort working independently when the path forward is not fully defined.

Nice to have

Experience developing CDN, caching, or web performance systems.
Experience operating crawlers, headless browsers, or distributed content-acquisition systems at scale.
Contributions to a browser engine or JavaScript runtime.

How we rate this

Member of Technical Staff (Web Access Engineer, Crawler) at Perplexity rates 85 out of 100 for how much of the daily work is AI. That makes it Builds AI (AI Level 4 of 4). The level is about AI in the job, not seniority.

Classification

Builds AI. The job is building AI systems.

●●●● Builds AI80 to 100
●●●○ Works on AI60 to 79
●●○○ Uses AI40 to 59
●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.

Key facts

Member of Technical Staff (Web Access Engineer, Crawler) is an open job at Perplexity.
Level rates this role AI Level 4 of 4: Builds AI.
The role is based in London and is marked remote.
The job was posted on November 13, 2026.
Seniority: lead.
Applications are handled by Perplexity. Apply on Perplexity