Member Of Technical Staff (Web Access Engineer, Crawler)
Perplexity
Perplexity is hiring a Web Access Engineer for the Crawler team to build reliable fetching and rendering systems for AI-powered search. You'll work on freely available public web content, within site permissions and approved access policies, and develop the infrastructure that supplies content to downstream indexing and retrieval systems.
Your focus will be diagnosing network errors, page-loading issues, and incomplete rendering. You'll trace problems from network protocols through client-side execution, distinguish technical failures from intentional access restrictions, and turn findings into production fixes that improve content quality and efficiency at scale.
Responsibilities
Own fetching and rendering improvements end to end, from investigation through implementation and production rollout, partnering with crawler engineers and SREs.
Investigate fetch and rendering failures across DNS, TLS, caching, redirects, and client-side rendering, and classify root causes.
Improve page loading, completion detection, and content capture to preserve relevant content and handle dynamic pages reliably.
Ensure the crawler respects site-imposed access restrictions and rate limits, identifies itself clearly, and retries only transient technical errors.
Build tools to reproduce failures, compare crawler output against expected page content, and turn findings into regression tests.
Define metrics for fetching reliability, content completeness, and error classification, and use them to prioritize improvements.
Qualifications
5+ years of software engineering experience in web infrastructure, browsers, CDNs, or crawling, with strong programming skills in at least one of Go, Python, Rust, or C++.
Deep understanding of and including status semantics, caching, conditional requests, content encoding, and redirects;
strong knowledge of TLS, DNS, cookies, and sessions.
Understanding of CDN caching and content delivery, error handling, retry backoff, and server-defined rate limits.
Knowledge of page lifecycle, network idle, service workers, client-side rendering, and shadow DOM;
experience debugging JavaScript and fluency with Chrome DevTools, CDP, HAR files, curl, and packet captures.
Comfort working independently when the path forward is not fully defined.
Nice to have
Experience developing CDN, caching, or web performance systems.
Experience operating crawlers, headless browsers, or distributed content-acquisition systems at scale.
Contributions to a browser engine or JavaScript runtime.
#J-18808-Ljbffr
Your focus will be diagnosing network errors, page-loading issues, and incomplete rendering. You'll trace problems from network protocols through client-side execution, distinguish technical failures from intentional access restrictions, and turn findings into production fixes that improve content quality and efficiency at scale.
Responsibilities
Own fetching and rendering improvements end to end, from investigation through implementation and production rollout, partnering with crawler engineers and SREs.
Investigate fetch and rendering failures across DNS, TLS, caching, redirects, and client-side rendering, and classify root causes.
Improve page loading, completion detection, and content capture to preserve relevant content and handle dynamic pages reliably.
Ensure the crawler respects site-imposed access restrictions and rate limits, identifies itself clearly, and retries only transient technical errors.
Build tools to reproduce failures, compare crawler output against expected page content, and turn findings into regression tests.
Define metrics for fetching reliability, content completeness, and error classification, and use them to prioritize improvements.
Qualifications
5+ years of software engineering experience in web infrastructure, browsers, CDNs, or crawling, with strong programming skills in at least one of Go, Python, Rust, or C++.
Deep understanding of and including status semantics, caching, conditional requests, content encoding, and redirects;
strong knowledge of TLS, DNS, cookies, and sessions.
Understanding of CDN caching and content delivery, error handling, retry backoff, and server-defined rate limits.
Knowledge of page lifecycle, network idle, service workers, client-side rendering, and shadow DOM;
experience debugging JavaScript and fluency with Chrome DevTools, CDP, HAR files, curl, and packet captures.
Comfort working independently when the path forward is not fully defined.
Nice to have
Experience developing CDN, caching, or web performance systems.
Experience operating crawlers, headless browsers, or distributed content-acquisition systems at scale.
Contributions to a browser engine or JavaScript runtime.
#J-18808-Ljbffr
Vacancy posted 49 minutes ago
Similar jobs that could be interesting for youBased on the Member Of Technical Staff (Web Access Engineer, Crawler) in Winnipeg, MB vacancy
- Keeping a crawler running is not the same as keeping it productive. Queues can grow while... ...years of experience in SRE, production engineering, or infrastructure-heavy backend roles,... ...Nice to have Experience operating web crawlers, search infrastructure, or browser fleets...Suggested
- ...drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft... ..., Berlin and Seoul. Join us! Role Overview: As a Senior Member of Technical Staff specializing in web data for pre-training, you will play a...SuggestedFull timeWork at officeLocal areaRemote workHome office
- ...our small but mighty team of engineers all report directly to the CTO... ...Newfoundland, with remote team members in Nova Scotia, New Brunswick,... ...You balance pragmatism with technical excellence, and you know which... ...:5+ years building production web products across frontend and backend...SuggestedInternshipWork at officeRemote workFlexible hours
- ...effectively with cross-functional teams to understand requirements and provide technical guidance. What We're Looking For (qualifications) MS/PhD in Computer Science, Electrical Engineering or a related field with a focus on computer vision. Proven research experience...SuggestedRemote workNight shift
- Perplexity is seeking a Web Access Engineer for the Crawler team to build reliable fetching and rendering systems powering AI-powered search. You will diagnose network and rendering issues, implementing production fixes that improve content quality at scale. You will investigate...Suggested
$140k - $240k per year
Location: Toronto (Office) On-site | Full-time Compensation: $140K - $240KThe Role: Member of Technical Staff (Applied AI) The Senior Applied AI Engineer will play a pivotal role in defining how AI agents navigate the web. This individual will focus on building open-source...Full timeInternshipWork at office$220k - $405k per year
...This Role Perplexity is seeking strong engineers with a passion for delivering frontier... ...implementing, and scaling these interfaces. As a member of our team, you'll work on an eclectic... ...experience alike. You'll also define technical strategy for how we scale to meet...Contract work$150k - $200k per year
...:We are seeking a skilled and experienced Senior Infrastructure Engineer to join our dynamic team in Toronto. As a Senior Infrastructure... ...microservices.Strong communication and teamwork skills, embracing technical challenges and innovation.Base Salary Range:CAD $150,000 - $200,...Worldwide- ...Fortune 100 companies. Our work covers audio, video, and other new data types that advance perceptual intelligence. We are hiring Members of Technical Staff to create new data infrastructure as well as improve upon existing infrastructure.The Role As a Member of Technical Staff...Long term contractInternshipWork at office
$160k - $210k per year
...Cockroach Continuum. We're looking for a staff software engineer to help build out the Migration... ...database teams.Take part in a collaborative, technically rigorous culture, and help shape... ...projects.After 3 months, you’ll be a full member of the team, contributing to the MOLT...InternshipWork at officeLocal areaRemote workWorldwideFlexible hours- ...drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft... ...with these hours for effective collaboration. As a Member Of Technical Staff, You Will Design and write high-performant and scalable...Full timeWork at officeLocal areaRemote workHome office
- ...informed and see every side of the story. Our mobile app, web app, and browser extension are home to a community of... ...Winner of Mobile Apps Showdown at CES The Opportunity:As a Staff Engineer, you'll own the technical direction of the web properties that convert curious...Remote work
$155k - $230k per year
...We are building a commercial engine that allows pharmaceutical marketers... ...and rapidly iterating technical approaches at the frontiers of... ...layer: Model clean schemas and access patterns for Postgres; use... ...Strong proficiency in full-stack web development technologies such...Work at officeRemote workNight shift- Member of Technical Staff – Robotics About Us Veeda AI is building the next generation of multimodal foundation world models for Physical AI. We're a small, fast-moving team of engineers and researchers from leading AI labs, tackling some of the most challenging problems...
- ...Physical AI. We're a small, fast-moving team of engineers and researchers from leading AI labs,... ...Science, Computer Graphics, Robotics, or a related technical field. Experience building or substantially extending a physics engine or simulation framework (Newton, MuJoCo,...
- ...drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft... ...on where you can be located for this role. As a Member Of Technical Staff, You Will Design, build and scale AI systems for serving...Full timeWork at officeLocal areaRemote workHome office
$155k - $230k per year
...help customers direct, inspect, and use their output. Take on engineering problems with depth. Make long-running agents reliable, stream... ...APIs. You'll help decide how agents run, recover from failures, access tools and context, and communicate with the application. You'll...InternshipWork at officeRemote work- ...Cohere is a team of researchers, engineers, designers, and more, who are... ...scale. This role demands deep technical expertise in production... ...organizational needs. We provide access to world class compute resources... ...can do your best work. As a Member of Technical Staff, you will:...Full timeWork at officeLocal areaRemote workHome office
- ...Physical AI. We're a small, fast-moving team of engineers and researchers from leading AI labs,... ..., Computer Engineering, or a related technical field. Deep hands-on experience with PyTorch... ...a kernel will stall from its memory access pattern before profiling. Experience profiling...
- ...customers. Cohere is a team of researchers, engineers, designers, and more, who are all... ...systems work at scale. This role demands deep technical expertise in production environments and... ...depending on organizational needs. We provide access to world class compute resources, data,...Full timeWork at officeLocal areaRemote workHome office
- Member of Technical Staff, Model Efficiency1 day ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features... ...? Our team is a fast-growing group of researchers and engineers focused on building reliable ML systems and pushing the...Full timeWork at officeRemote workFlexible hours
- ...world models for Physical AI. We're a small, fast-moving team of engineers and researchers from leading AI labs, tackling some of the most... ...-on experience in Computer Science, Engineering, or a related technical field. You have strong Python and software engineering...Night shift
- Member of Technical Staff, Post training team Join to apply for the Member of Technical Staff, Post training team role at CohereMember... ...for our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft....Full timeWork at officeRemote workFlexible hours
- Member of Technical Staff – World Models About Us Veeda AI is building the next generation of multimodal foundation world models for Physical AI. We're a small, fast-moving team of engineers and researchers from leading AI labs, tackling some of the most challenging problems...
- ...world models for Physical AI. We're a small, fast-moving team of engineers and researchers from leading AI labs, tackling some of the most... ...'s or Ph.D. in Computer Science, Engineering, or a related technical field, or equivalent hands-on experience. Demonstrated ability...
- Enhance AI capabilities at Onix as a Member of Technical Staff with a focus on small language models. Engage in innovative projects combining efficient... ...learning and retrieval methods • Ability to balance engineering and research expertise • Clear articulation of technical narratives...
- ...types that advance perceptual intelligence. We are hiring Members of Technical Staff to create new data infrastructure as well as improve upon existing... ...in applied machine learning, AI research, or research engineering. Exceptional candidates with equivalent research...Long term contractTemporary workWork at office
- Join the forefront of AI research as a Technical Staff Member for Generative Modeling at Veeda AI. Leverage your skills in creating advanced world models for robotics applications. As part of Veeda AI's pioneering team, you will engage in complex problem-solving at the intersection...
- Member of Technical Staff, Small Language ModelsFull-time Onix is the Personal Intelligence platform. Each onix is a small language model trained... ...our experts and users care about. Privacy is an engineering surface, not a compliance line. Per‑expert isolation, on‑device...Live InRelocation
- Contribute to groundbreaking Physical AI innovations at Veeda AI as a Technical Staff Member. Engage in multi-node video world model training while tackling key performance challenges. Veeda AI is seeking a Technical Staff Member to enhance its world-class team working with...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Member Of Technical Staff (Web Access Engineer, Crawler). Be the first to apply!
Related searches
- service support technician Winnipeg, MB
- sr. IT support analyst Winnipeg, MB
- technical analyst Winnipeg, MB
- technical system analyst Winnipeg, MB
- desktop support analyst Winnipeg, MB
- senior technical analyst Winnipeg, MB
- network technical analyst Winnipeg, MB
- service desk analyst Winnipeg, MB
- senior technical specialist Winnipeg, MB
- decision support analyst Winnipeg, MB

