RemoteLeaf Logo
Log in Sign up

Python Scraping Engineer (Remote, Full-Time) [HR160]

2 hours, 58 minutes ago
India
Full-time
Senior
Backend Engineer
Software Development
Docker HTTP Kubernetes Playwright Puppeteer Python Selenium TLS
Apply Now
Smart Working

Smart Working

Smart Working is a company that specializes in software development outsourcing and staff augmentation. They offer nearshore software development services, outsourcing solutions, and staff augmentation with a focus on providing highly skilled Indian de...

Internet Software & Services
View All Jobs 16

Description

  • Design, develop, and maintain large-scale Python-based web scraping systems.
  • Build scrapers capable of extracting data from Google and other highly protected websites.
  • Develop scraping pipelines for dynamic, JavaScript-heavy websites using browser automation.
  • Continuously adapt scraping systems to changes in page structure, request flows, and anti-bot defences.
  • Engineer robust data extraction and processing pipelines to ensure high data quality and reliability.
  • Implement advanced scraping strategies including proxy rotation, fingerprinting, and request routing.
  • Monitor scraping systems in production and quickly identify and debug failures.
  • Optimise scraping infrastructure for performance, reliability, cost efficiency, and low latency.
  • Collaborate with data engineers, data scientists, and product teams to ensure collected data is usable and trusted.
  • Maintain clear technical documentation and operational runbooks, and contribute to AI-assisted development workflows.

Requirements

  • Strong professional experience with Python development and building production-grade web scraping systems.
  • Hands-on experience scraping Google or similarly protected platforms and handling headless browser fingerprinting/anti-bot evasion.
  • Deep understanding of HTTP, TLS, cookies, headers, redirects, and browser networking behavior.
  • Experience using browser automation frameworks such as Playwright, Selenium, or Puppeteer.
  • Strong knowledge of HTML parsing, DOM traversal, and structured data extraction.
  • Experience handling rate limiting, CAPTCHA systems, IP rotation, and bot detection mechanisms.
  • Experience building asynchronous or concurrent scraping architectures and operating systems at scale in cloud environments.
  • Strong debugging and troubleshooting skills for complex distributed systems.
  • Experience with Docker or Kubernetes (nice to have).
  • Experience with distributed task systems/job queues, data quality monitoring/anomaly detection, or search/advertising/competitive intelligence datasets (nice to have).

Benefits

  • Fixed shifts (Summer: 12:00 PM–9:30 PM IST; Winter: 1:00 PM–10:30 PM IST).
  • No weekend work to support real work–life balance.
  • Day 1 benefits: company-provided laptop and full medical insurance.
  • Remote-first role with a supportive mentorship community and forums.
  • Long-term career focus where contributions are valued and supported.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

CI&T

[Job-28474] Senior Java Backend Developer (with Kotlin Backend), Brazil

CI&T 5K-10K Internet Software & Services

Desenvolvedor Backend Sênior (Java/Kotlin) na CI&T para atuar em projeto de um grande banco digital do Brasil, entregando funcionalidades end-to-end e garantindo serviços prontos para produção.

Brazil Full-time Senior Backend Engineer
43 minutes ago
CI&T

[Job - 28662] Desenvolvedor Java Pleno, Brazil

CI&T 5K-10K Internet Software & Services

Pleno Java Developer at CI&T working with a distributed team to develop and evolve a client's scalable, high-performance solution and implement the client's primary partnership channel.

Brazil Full-time Mid Level Backend Engineer
Agile Angular CI/CD Docker Generative AI Kubernetes Microservices Node.js REST API Spring Boot
58 minutes ago
TWO95 International

Sr Java Developer - Remote

TWO95 International 51-250 Internet Software & Services

Senior Java Developer contractor for a remote client on a 12+ month engagement to deliver and maintain production web applications and microservice-based systems.

United States Contract Senior Backend Engineer
Agile Angular AWS GraphQL Java Microservices Spring Boot TypeScript
1 hour, 43 minutes ago
lago

Amazon SP API Developer

lago Professional Services

Amazon SP API Developer at Lago responsible for building and maintaining accounting-focused software that integrates Amazon SP API data to automate financial reporting and reconcile Amazon FBA financial reports into a unified accounting system.

Serbia Full-time Senior Backend Engineer
1 hour, 58 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers