RemoteLeaf Logo
Log in Sign up

Anti-Bot Engineer (Remote, Full-Time) [HR177]

1 month, 3 weeks ago
India
Full-time
Senior
Backend Engineer
Software Development
Docker Go HTTP JavaScript Kubernetes Playwright Puppeteer Python Rust Selenium
Apply Now
Smart Working

Smart Working

Smart Working is a company that specializes in software development outsourcing and staff augmentation. They offer nearshore software development services, outsourcing solutions, and staff augmentation with a focus on providing highly skilled Indian de...

Internet Software & Services
View All Jobs 18

Description

  • Design, build, and maintain large-scale Python-based scraping systems for highly protected websites.
  • Architect resilient extraction systems for dynamic, JavaScript-heavy pages using browser automation and hybrid approaches.
  • Continuously adapt scraping systems to page structure changes, request flow changes, and anti-bot mechanisms.
  • Build production-grade data extraction pipelines with a focus on accuracy, observability, and fault tolerance.
  • Implement proxy rotation, fingerprinting, session management, and request routing strategies to support stability at scale.
  • Monitor system health, detect anomalies proactively, and debug complex production failures across distributed systems.
  • Optimize scraping infrastructure for performance, cost efficiency, and reliability.
  • Collaborate with data engineering and product teams to ensure scraped data is structured, validated, and trusted.
  • Operate and improve systems running continuously in cloud environments.
  • Document system architecture, scraping logic, and operational procedures for long-term maintainability.

Requirements

  • 7+ years of professional software engineering experience, with a focus on backend systems, data engineering, or distributed systems.
  • Proven experience building and operating large-scale production web scraping systems.
  • Deep hands-on experience scraping Google or similarly complex, heavily protected anti-bot environments.
  • Strong expertise in Python or comparable production languages such as Go, Rust, or JavaScript.
  • Strong understanding of HTTP internals, including headers, cookies, TLS, redirects, sessions, and browser networking behaviour.
  • Experience with browser automation frameworks such as Playwright, Selenium, Puppeteer, or equivalent.
  • Strong knowledge of HTML parsing, DOM traversal, and high-performance data extraction techniques.
  • Proven experience handling anti-bot systems, including rate limiting, CAPTCHAs, IP rotation, and fingerprinting.
  • Experience designing asynchronous and concurrent systems for high-throughput workloads.
  • Strong debugging skills across distributed, failure-prone production systems.
  • Experience running cloud-based systems at scale with strong operational ownership.
  • Comfortable working in highly adversarial and fast-changing technical environments.
  • Experience with Docker and Kubernetes in production environments (nice to have).
  • Exposure to distributed task queues or large-scale job orchestration systems (nice to have).
  • Experience with monitoring, anomaly detection, or data quality validation systems (nice to have).
  • Background in search intelligence, advertising tech, or competitive intelligence platforms (nice to have).
  • Familiarity with observability tooling and production-grade system monitoring (nice to have).
  • Exposure to AI-assisted development workflows or agentic coding tools (nice to have).

Benefits

  • Fixed shifts: 12:00 PM - 9:30 PM IST in summer and 1:00 PM - 10:30 PM IST in winter.
  • No weekend work.
  • Laptop provided from day one.
  • Full medical insurance provided from day one.
  • Mentorship, community support, and forums for sharing ideas.
  • Long-term career opportunity where contributions are valued.
  • Remote-first work with a focus on work-life balance.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Epic Games

Senior Backend Engineer, Developer Platform (Horde)

Epic Games 5K-10K Internet Software & Services

Epic Games is hiring a senior programmer for its Unreal Engine developer platform team to work on Horde, the internal platform that supports CI/CD, remote execution, build health, storage, and developer workflows.

United States Canada Full-time Senior Backend Engineer
$173k-$279k
AWS Azure C# CI/CD gRPC MongoDB .NET Redis
15 hours, 23 minutes ago
Hasbro

Principal Identity Engineer

Hasbro 5K-10K Consumer Goods

Wizards of the Coast is hiring a Principal Identity Engineer to own the technical direction of its enterprise identity architecture, supporting Zero Trust, privileged access, and identity governance across cloud and on-premises environments.

United States Full-time Lead Security Engineer
$173k-$259k
Active Directory CI/CD Git OpenID Connect PowerShell Pulumi Python REST API SAML Terraform
15 hours, 38 minutes ago
Coupang

Senior Staff Back-end Engineer (eCommerce Engineering)

Coupang 1K-5K Internet Software & Services

Coupang is hiring a Senior Staff Backend Engineer to lead the architecture and technical strategy for its core e-commerce platform services across global markets.

South Korea Full-time Lead Backend Engineer
Apache Airflow Apache Spark AWS Cassandra Grafana Java Kafka Kotlin Kubernetes Microservices OpenTelemetry Prometheus Spring
15 hours, 38 minutes ago
Dragos

Senior Device Engineer

Dragos 251-1K Professional Services

Dragos is hiring a Senior Device Engineer to build automation software that identifies, fingerprints, and interacts with network-connected devices across critical infrastructure environments.

United States Full-time Senior Embedded Systems Engineer Security Engineer
$165k-$165k
Cybersecurity Docker Embedded Systems Go HTTP JavaScript Node.js TCP/IP TLS TypeScript
15 hours, 38 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers