Web Scraping Lead - Global Team (India)

6 hours, 43 minutes ago
Full-time
Lead
Software Development
YipitData

YipitData

YipitData specializes in market research for the disruptive economy by analyzing billions of data points daily to deliver accurate and timely insights on over 1,000 companies, aiding investors in making informed decisions and helping businesses enhance...

Professional Services
251-1K
Founded 2013
$492M raised

Description

  • Lead, mentor, and grow a team of web scraping engineers.
  • Own the architecture, roadmap, scalability, and reliability of scraping systems.
  • Guide development and maintenance of scrapers using Playwright, Puppeteer, Selenium, and related tools.
  • Solve challenges involving dynamic websites, browser automation, fingerprinting, sessions, cookies, headers, and proxies.
  • Establish monitoring, logging, alerting, and data-quality practices.
  • Drive performance, reliability, and infrastructure improvements.
  • Partner with Engineering, Data, Analytics, and Product teams to prioritize requirements and deliver reliable data.
  • Participate in hiring, technical reviews, and engineering planning.

Requirements

  • 7+ years of software engineering experience with strong web scraping experience.
  • Previous engineering management or team leadership experience preferred.
  • Hands-on experience with Playwright, Puppeteer, or Selenium.
  • Deep knowledge of HTTP, REST APIs, HTML/DOM, browser rendering, and TLS/SSL.
  • Experience with fingerprinting, anti-bot challenges, cookies, headers, sessions, and proxy rotation.
  • Experience building scalable and reliable scraping infrastructure.
  • Strong knowledge of monitoring, logging, metrics, and production troubleshooting.
  • Excellent communication, stakeholder-management, ownership, leadership, and mentoring skills.
  • Ability to work remotely in India during standard hours of 11 a.m.–8 p.m. IST, with flexibility.
  • Candidates must not require current or future visa sponsorship.

Benefits

  • Competitive salary of $73,000–$90,000 annually with comprehensive benefits.
  • Flexible work hours and fully remote or hybrid work options.
  • Unlimited vacation, parental leave, wellness budget, learning reimbursement, and team events.
  • Health, dental, and vision plans, including options with no monthly employee premium.
  • 401(k) match, commuter benefits, and a monthly work-from-home budget for remote workers.
  • Career ladders, regular career discussions, internal mobility, and manager or individual-contributor growth paths.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Software Engineering Manager, Compliance - Core

Fanatics 5K-10K Retailing

Fanatics Betting & Gaming is seeking an Engineering Manager to lead Java/AWS backend teams building secure, reliable account, eligibility, responsible gaming, geolocation, and regulatory compliance systems for its U.S. sportsbook.

AWS Java
7 hours, 58 minutes ago

Software Engineering Manager .NET

Lone Wolf Technologies 251-1K Internet Software & Services

Lone Wolf is seeking a Software Engineering Manager to lead a small team maintaining and evolving its Azure-based e-signature platform used in real estate transactions, with responsibility for reliability, security, performance, and ongoing product improvements.

Agile Angular Azure CSS Cybersecurity HTML JavaScript .NET REST API Solid.js SQL Server TypeScript
1 day, 7 hours ago

Software Engineering Manager .NET

Lone Wolf Technologies 251-1K Internet Software & Services

Lone Wolf is seeking a Software Engineering Manager to lead a small team maintaining and evolving an Azure-based e-signature platform that supports secure, reliable, legally valid real estate transactions.

Agile Angular Azure CSS HTML JavaScript .NET REST API Solid.js SQL Server TypeScript
1 day, 7 hours ago

Architect, Product Development

TTEC Digital 1K-5K Professional Services

The Architect, Product Development at TTEC Digital provides technical leadership for AI-powered customer experience and contact center products, guiding solution architecture, engineering teams, implementations, and customer deployments.

AWS Azure CI/CD Kubernetes Microservices Vertex AI
1 day, 8 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers