RemoteLeaf Logo
Log in Sign up

Python Scraping Engineer (Remote, Full-Time) [HR160]

3 weeks, 2 days ago
India
Full-time
Senior
Backend Engineer
Software Development
Docker HTTP Kubernetes Playwright Puppeteer Python Selenium TLS
Apply Now
Smart Working

Smart Working

Smart Working is a company that specializes in software development outsourcing and staff augmentation. They offer nearshore software development services, outsourcing solutions, and staff augmentation with a focus on providing highly skilled Indian de...

Internet Software & Services
View All Jobs 95

Description

  • Design, develop, and maintain large-scale Python-based web scraping systems.
  • Build scrapers capable of extracting data from Google and other highly protected websites.
  • Develop scraping pipelines for dynamic, JavaScript-heavy websites using browser automation.
  • Continuously adapt scraping systems to changes in page structure, request flows, and anti-bot defences.
  • Engineer robust data extraction and processing pipelines to ensure high data quality and reliability.
  • Implement advanced scraping strategies including proxy rotation, fingerprinting, and request routing.
  • Monitor scraping systems in production and quickly identify and debug failures.
  • Optimise scraping infrastructure for performance, reliability, cost efficiency, and low latency.
  • Collaborate with data engineers, data scientists, and product teams to ensure collected data is usable and trusted.
  • Maintain clear technical documentation and operational runbooks, and contribute to AI-assisted development workflows.

Requirements

  • Strong professional experience with Python development and building production-grade web scraping systems.
  • Hands-on experience scraping Google or similarly protected platforms and handling headless browser fingerprinting/anti-bot evasion.
  • Deep understanding of HTTP, TLS, cookies, headers, redirects, and browser networking behavior.
  • Experience using browser automation frameworks such as Playwright, Selenium, or Puppeteer.
  • Strong knowledge of HTML parsing, DOM traversal, and structured data extraction.
  • Experience handling rate limiting, CAPTCHA systems, IP rotation, and bot detection mechanisms.
  • Experience building asynchronous or concurrent scraping architectures and operating systems at scale in cloud environments.
  • Strong debugging and troubleshooting skills for complex distributed systems.
  • Experience with Docker or Kubernetes (nice to have).
  • Experience with distributed task systems/job queues, data quality monitoring/anomaly detection, or search/advertising/competitive intelligence datasets (nice to have).

Benefits

  • Fixed shifts (Summer: 12:00 PM–9:30 PM IST; Winter: 1:00 PM–10:30 PM IST).
  • No weekend work to support real work–life balance.
  • Day 1 benefits: company-provided laptop and full medical insurance.
  • Remote-first role with a supportive mentorship community and forums.
  • Long-term career focus where contributions are valued and supported.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

The TARA Group

Director, Demand Side Platform (DSP) Engineering

The TARA Group 11-50 Media

The Director of DSP Engineering will lead development of a demand side platform at a US-based company, driving real-time bidding, backend performance, and campaign management tools that support revenue growth and client success.

United States Full-time Executive Backend Engineer Director of Engineering
$145k-$145k
AWS Docker GCP Go Kubernetes Machine Learning
1 hour, 2 minutes ago
Remote

Senior Backend Engineer

Remote 251-1K Professional Services

Remote is hiring an engineer to help build tools, APIs, integrations, and agentic workflows for its globally distributed HR and Payroll products in a fully remote, async environment.

Remote
Full-time Senior Backend Engineer
$53k-$120k
Angular AWS CI/CD Clojure Docker Elixir Erlang GitLab Haskell Kubernetes Next.js Phoenix PostgreSQL React Scala Vue.js
2 hours, 9 minutes ago
Ethics Code

Java Engineer - Mid to Senior

Ethics Code Diversified Consumer Services

ETHICS CODE is seeking a Mid to Senior Java Engineer to design, develop, and maintain business-critical Java applications within a collaborative engineering team.

Dominican Republic Full-time Mid Level Backend Engineer
Agile Git Hibernate Java MySQL Oracle PostgreSQL REST API Spring
2 hours, 37 minutes ago
Brillio

Senior Software Development Lead - R01564232

Brillio 1K-5K IT Services

Brillio Romania is hiring a Senior Software Development Lead to develop backend services for high-performance, microservices-based systems in Bucharest or remotely.

Romania Full-time Senior Backend Engineer
CI/CD Docker Git Java Kafka Kotlin Microservices PostgreSQL REST API
2 hours, 43 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers