Senior Python Data Scraping Engineer (Freelance)

1 hour, 49 minutes ago
Part-time
Senior
Software Development
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Own end-to-end data extraction workflows across complex websites and deliver structured datasets with complete coverage and accuracy.
  • Use internal tools such as Apify and OpenRouter, along with custom workflows, to accelerate data collection, validation, and execution.
  • Extract data reliably from dynamic and interactive web sources, including JavaScript-rendered content and changing site behavior.
  • Enforce data quality standards through validation checks, cross-source consistency controls, and systematic verification before delivery.
  • Scale scraping operations for large datasets using batching or parallelization while monitoring failures and maintaining stability.
  • Adapt scraping approaches to handle APIs, proxies, anti-bot mechanisms, and site structure changes.
  • Clean, normalize, structure, and validate data from diverse sources before delivery.

Requirements

  • 5+ years of relevant experience in data engineering, web scraping, automation, or software development.
  • Bachelor’s or Master’s degree in Engineering, Applied Mathematics, Computer Science, or a related technical field is a plus.
  • Strong technical foundation with practical experience in scripting, automation, and AI-assisted workflows.
  • Strong Python web scraping experience using BeautifulSoup, Selenium, or similar tools.
  • Experience scraping dynamic content such as JavaScript, AJAX, and infinite scroll, as well as APIs via proxies.
  • Proven ability to extract data from complex structures, including hierarchies, archived pages, and inconsistent HTML.
  • Solid background in data cleaning, normalization, and validation, with delivery formats such as CSV, JSON, and Google Sheets.
  • Experience handling anti-bot mechanisms and dynamic site structures at scale.
  • Experience with cloud infrastructure such as AWS and containerization with Docker in real workflows.
  • Hands-on experience with LLM frameworks such as LangChain, OpenRouter, or similar tools for automation tasks.
  • Upper-intermediate English proficiency (B2) or above.
  • GitHub link is a plus.

Benefits

  • Part-time freelance remote work.
  • Estimated workload of around 10–20 hours per week during active project phases.
  • Compensation up to $25 per hour equivalent, depending on level and pace of contribution.
  • Opportunity to work on AI projects through Mindrift’s platform.
  • Access to provided tools such as Apify and OpenRouter as part of the workflow.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Senior Software Engineer - Backend Platform

Customer.io 11-50 Professional Services

Customer.io is hiring an experienced backend engineer for its Platform team to build and own the distributed infrastructure that powers a high-scale marketing automation platform processing billions of events and messages worldwide.

AWS DynamoDB Ember GCP Go MySQL PostgreSQL React System Design
1 hour, 34 minutes ago

Senior Ruby on Rails Developer (LLM)

OnTheGoSystems 51-250 Internet Software & Services

OTGS is hiring a senior Ruby on Rails developer to build and improve scalable production products, with a focus on end-to-end feature ownership and practical use of AI/LLM technologies.

AWS CI/CD LLM React Ruby Ruby on Rails WordPress
2 hours, 4 minutes ago

Sterling OMS Developer

Weekday 11-50 Construction & Engineering

Weekday’s client is hiring a Sterling OMS Core Developer in India to build and customize IBM Sterling Order Management applications, integrations, and microservices-based enterprise solutions.

CI/CD Java Kafka Microservices REST API
2 hours, 5 minutes ago

Server Backend Developer (Go + Python) for Imunify360 (work anywhere, remote)

CloudLinux 51-250 IT Services

CloudLinux Inc. is seeking a Backend Developer for its Imunify360 Server Team to build and maintain scalable backend systems for a large-scale security platform serving shared and VPS/dedicated servers.

Ansible CI/CD ClickHouse Docker Git Go Grafana HAProxy Kafka Linux LLM MongoDB Puppet Python RabbitMQ Redis SaltStack
2 hours, 19 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers