Web Scraping Lead - Global Team (India)

3 weeks ago
Full-time
Lead
Software Development
YipitData

YipitData

YipitData specializes in market research for the disruptive economy by analyzing billions of data points daily to deliver accurate and timely insights on over 1,000 companies, aiding investors in making informed decisions and helping businesses enhance...

Professional Services
251-1K
Founded 2013
$492M raised

Description

  • Lead, mentor, and grow a team of web scraping engineers.
  • Own the architecture, roadmap, scalability, and reliability of scraping systems.
  • Guide development and maintenance of scrapers using Playwright, Puppeteer, Selenium, and related tools.
  • Solve challenges involving dynamic websites, browser automation, fingerprinting, sessions, cookies, headers, and proxies.
  • Establish monitoring, logging, alerting, and data-quality practices.
  • Drive performance, reliability, and infrastructure improvements.
  • Partner with Engineering, Data, Analytics, and Product teams to prioritize requirements and deliver reliable data.
  • Participate in hiring, technical reviews, and engineering planning.

Requirements

  • 7+ years of software engineering experience with strong web scraping experience.
  • Previous engineering management or team leadership experience preferred.
  • Hands-on experience with Playwright, Puppeteer, or Selenium.
  • Deep knowledge of HTTP, REST APIs, HTML/DOM, browser rendering, and TLS/SSL.
  • Experience with fingerprinting, anti-bot challenges, cookies, headers, sessions, and proxy rotation.
  • Experience building scalable and reliable scraping infrastructure.
  • Strong knowledge of monitoring, logging, metrics, and production troubleshooting.
  • Excellent communication, stakeholder-management, ownership, leadership, and mentoring skills.
  • Ability to work remotely in India during standard hours of 11 a.m.–8 p.m. IST, with flexibility.
  • Candidates must not require current or future visa sponsorship.

Benefits

  • Competitive salary of $73,000–$90,000 annually with comprehensive benefits.
  • Flexible work hours and fully remote or hybrid work options.
  • Unlimited vacation, parental leave, wellness budget, learning reimbursement, and team events.
  • Health, dental, and vision plans, including options with no monthly employee premium.
  • 401(k) match, commuter benefits, and a monthly work-from-home budget for remote workers.
  • Career ladders, regular career discussions, internal mobility, and manager or individual-contributor growth paths.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Engineering Manager - Observability | Spain | Remote

Grafana 1K-5K IT Services

Grafana Labs is seeking a remote Engineering Manager to lead the distributed Ingest team building configuration and deployment tools for Grafana Cloud observability products, including Fleet Management, Kubernetes Monitoring, and Instrumentation Hub.

Helm Kubernetes OpenTelemetry Prometheus
27 minutes ago

Engineering Manager - AutoReview

CoLab 51-250 Construction & Engineering

CoLab is hiring an Engineering Manager to lead the team developing AutoReview, AI-powered tools that automate mechanical design reviews for complex engineering products.

27 minutes ago

Engineering Manager - Observability | Ireland | Remote

Grafana 1K-5K IT Services

Grafana Labs is hiring a remote Engineering Manager to lead the Ingest team, which builds tooling for deploying and managing observability collectors across Grafana Cloud.

Helm Kubernetes OpenTelemetry Prometheus
42 minutes ago

Team Lead Kotlin Infrastructure

JetBrains 1K-5K Internet Software & Services

JetBrains is seeking a Team Lead to guide the Kotlin Infrastructure team in evolving the developer platform, build systems, and engineering workflows that support Kotlin development at scale.

CI/CD Gradle Kotlin Secrets Management TeamCity
57 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers