Web Scraping Lead - Global Team

Job Summary

We are looking for an experienced Web Scraping Manager to lead our web scraping engineering team and own the design, scalability, reliability, and performance of our scraping infrastructure. You ll lead engineers, drive technical architecture and best practices, solve complex scraping challenges, and collaborate with Engineering, Data, Analytics, and Product teams to deliver reliable, high-quality data.

Responsibilities

• Lead, mentor, and grow a team of web scraping engineers.
• Own the architecture, roadmap, scalability, and reliability of scraping systems.
• Guide the development and maintenance of scrapers using Playwright, Puppeteer, Selenium, etc.
• Solve complex challenges involving dynamic websites, browser automation, fingerprinting, sessions, cookies, headers, and proxies.
• Establish monitoring, logging, alerting, and data-quality practices.
• Drive performance, reliability, and infrastructure improvements.
• Partner with cross-functional teams to prioritize scraping requirements and deliver high-quality data.
• Participate in hiring, technical reviews, and engineering planning.

This is a fully-remote opportunity based in India. Standard work hours are from 11am to 8pm IST, but there is flexibility here.

You Are Likely To Succeed If:

• 7+ years of software engineering experience, with strong web scraping experience.
• Previous engineering management/team leadership experience preferred.
• Strong hands-on experience with Playwright, Puppeteer, or Selenium.
• Deep understanding of HTTP, REST APIs, HTML/DOM, browser rendering, and TLS/SSL.
• Strong experience with fingerprinting, anti-bot challenges, cookies, headers, sessions, and proxy rotation.
• Experience building scalable, reliable scraping infrastructure.
• Strong knowledge of monitoring, logging, metrics, and production troubleshooting.
• Excellent communication and stakeholder-management skills.
• Strong ownership, leadership, and mentoring abilities.

This role may be performed fully remotely within the United States.

Please note that for this position, we are not able to consider candidates who currently or in the future will require visa sponsorship.

Disclaimer: This job posting and Location has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.