Anti-Bot Engineer (Remote, Full-Time) [HR177]

30 minutes ago
Full-time
Senior
Software Development
Smart Working

Smart Working

Smart Working is a company that specializes in software development outsourcing and staff augmentation. They offer nearshore software development services, outsourcing solutions, and staff augmentation with a focus on providing highly skilled Indian de...

Internet Software & Services

Description

  • Design, build, and maintain large-scale Python-based scraping systems for highly protected websites.
  • Architect resilient extraction systems for dynamic, JavaScript-heavy pages using browser automation and hybrid approaches.
  • Continuously adapt scraping systems to page structure changes, request flow changes, and anti-bot mechanisms.
  • Build production-grade data extraction pipelines with a focus on accuracy, observability, and fault tolerance.
  • Implement proxy rotation, fingerprinting, session management, and request routing strategies to support stability at scale.
  • Monitor system health, detect anomalies proactively, and debug complex production failures across distributed systems.
  • Optimize scraping infrastructure for performance, cost efficiency, and reliability.
  • Collaborate with data engineering and product teams to ensure scraped data is structured, validated, and trusted.
  • Operate and improve systems running continuously in cloud environments.
  • Document system architecture, scraping logic, and operational procedures for long-term maintainability.

Requirements

  • 7+ years of professional software engineering experience, with a focus on backend systems, data engineering, or distributed systems.
  • Proven experience building and operating large-scale production web scraping systems.
  • Deep hands-on experience scraping Google or similarly complex, heavily protected anti-bot environments.
  • Strong expertise in Python or comparable production languages such as Go, Rust, or JavaScript.
  • Strong understanding of HTTP internals, including headers, cookies, TLS, redirects, sessions, and browser networking behaviour.
  • Experience with browser automation frameworks such as Playwright, Selenium, Puppeteer, or equivalent.
  • Strong knowledge of HTML parsing, DOM traversal, and high-performance data extraction techniques.
  • Proven experience handling anti-bot systems, including rate limiting, CAPTCHAs, IP rotation, and fingerprinting.
  • Experience designing asynchronous and concurrent systems for high-throughput workloads.
  • Strong debugging skills across distributed, failure-prone production systems.
  • Experience running cloud-based systems at scale with strong operational ownership.
  • Comfortable working in highly adversarial and fast-changing technical environments.
  • Experience with Docker and Kubernetes in production environments (nice to have).
  • Exposure to distributed task queues or large-scale job orchestration systems (nice to have).
  • Experience with monitoring, anomaly detection, or data quality validation systems (nice to have).
  • Background in search intelligence, advertising tech, or competitive intelligence platforms (nice to have).
  • Familiarity with observability tooling and production-grade system monitoring (nice to have).
  • Exposure to AI-assisted development workflows or agentic coding tools (nice to have).

Benefits

  • Fixed shifts: 12:00 PM - 9:30 PM IST in summer and 1:00 PM - 10:30 PM IST in winter.
  • No weekend work.
  • Laptop provided from day one.
  • Full medical insurance provided from day one.
  • Mentorship, community support, and forums for sharing ideas.
  • Long-term career opportunity where contributions are valued.
  • Remote-first work with a focus on work-life balance.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

.NET Developer

Distro 11-50 Internet Software & Services

Solvo Global is hiring a remote .NET Software Engineer in Nairobi to build and maintain modern data processing solutions using .NET, React, Azure, and SQL Server.

Agile Azure C# CI/CD CSS Git GitHub Actions GraphQL HTML JavaScript Jest .NET React REST API Scrum SQL Server TypeScript
0 minutes ago

Director of IT, Information Security & Data Privacy

Energage 51-250 Professional Services

Energage is seeking a Director of IT, Information Security & Data Privacy to lead its internal technology, cybersecurity, and privacy programs in a fast-growing SaaS environment.

AWS Azure Cybersecurity
0 minutes ago

IA Engineer (Python + LLM)

Inter 51-250 Banks

Inter is hiring a Python backend engineer to develop AI agents and related platform integrations for its global financial super app.

Agile AWS Azure CI/CD GCP Kafka Microservices Python RabbitMQ REST API
0 minutes ago

Security Engineer, DevSecOps - Mexico

JumpCloud 251-1K Internet Software & Services

JumpCloud is hiring a remote Security Engineer for its DevSecOps team in Mexico to design and build cloud security automation, detection, and vulnerability management solutions that protect the company’s data and infrastructure.

AWS DevSecOps GCP GitHub Actions Go Python SIEM Terraform
0 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers