RemoteLeaf Logo
Log in Sign up

Director of Software Engineering (Node.js & Web Scraping Expert)

3 hours, 30 minutes ago
United States
Full-time
Executive
Director of Engineering
Software Development
Agile Apache Airflow Apache Spark AWS Azure CI/CD Docker Elasticsearch GCP Kafka Kanban Kubernetes Machine Learning Microservices MongoDB NLP Node.js Playwright PostgreSQL Puppeteer RabbitMQ Redis Scrum Selenium SQL
Apply Now
PortPro

PortPro

PortPro is a leading provider of drayage software, offering the most advanced and modern operating system for drayage carriers, brokers, and companies. Their drayOS platform is designed to improve operations, save time, and increase revenue in containe...

Internet Software & Services
11-50
View All Jobs 4

Description

  • Architect, develop, and maintain scalable Node.js-based web scraping systems.
  • Design and implement pipelines for processing large volumes of structured and unstructured data.
  • Manage anti-bot solutions, including CAPTCHA handling, session management, fingerprinting, and IP rotation.
  • Optimize scraping performance, reliability, efficiency, proxy usage, and data storage strategies.
  • Lead adoption of cloud-native, containerized, and serverless architectures.
  • Implement monitoring and alerting for scraping failures, IP bans, and performance bottlenecks.
  • Collaborate with Product, DevOps, and Data Science teams to integrate extracted data into applications and analytics.
  • Oversee CI/CD pipelines, automated deployments, microservices, API integrations, and real-time data streaming.
  • Lead, mentor, and grow the engineering team while defining the technology roadmap.
  • Establish engineering standards for security, code quality, agile delivery, and legal and ethical compliance.

Requirements

  • 10+ years of software engineering experience, including 5+ years in web scraping and large-scale data extraction.
  • Advanced hands-on experience with Node.js, Puppeteer, Playwright, Cheerio, Selenium, and headless browser automation.
  • Experience with CAPTCHA handling, IP rotation, session management, browser fingerprinting, and anti-bot evasion.
  • Deep knowledge of residential, datacenter, rotating, and VPN proxy management.
  • Experience with SQL and NoSQL databases such as MongoDB, PostgreSQL, Redis, and Elasticsearch.
  • Familiarity with Kafka, RabbitMQ, Spark, Airflow, or similar data-processing frameworks.
  • Strong experience with CI/CD, Docker, Kubernetes, and AWS, GCP, or Azure.
  • Proven ability to scale engineering teams and lead complex projects, with strong communication and stakeholder-management skills.
  • Experience with Scrum or Kanban and a demonstrated commitment to mentorship and continuous learning.
  • Preferred: experience with machine learning for data extraction, NLP, enterprise crawling frameworks such as Scrapy, Colly, or Apify, and data-driven businesses or web-scraping startups.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Raydar

Director of Engineering (Remote)

Raydar Professional Services

Engineering leader for an education technology company, overseeing multiple product engineering teams while remaining hands-on with code and architecture to deliver reliable products and AI capabilities.

United States Full-time Executive Director of Engineering
$210k-$240k
AWS MongoDB Python TypeScript
3 days, 4 hours ago
3Cloud

Director - Data Platforms & Architecture

3Cloud 251-1K Internet Software & Services

The Director in Delivery at 3Cloud leads teams and enterprise-scale Azure and Databricks programs, connecting business goals to cloud modernization, data platform strategy, and reliable multi-team execution.

United States Full-time Executive Data Architect Director of Engineering
$163k-$244k
Agile Apache Spark Azure CI/CD Databricks Machine Learning Microservices Power BI Scrum Snowflake System Design
5 days, 3 hours ago
Zone & Co

Director, Engineering (The Americas)

Zone & Co 251-1K Diversified Financial Services

Zone & Co is seeking a Director of Engineering to lead distributed engineering teams across the Americas while partnering globally to scale AI, platform, Quote-to-Cash, and ZoneProcure solutions for Oracle NetSuite customers.

United States Full-time Executive Director of Engineering
$190k-$250k
AWS CI/CD JavaScript NetSuite Python REST API TypeScript
6 days, 2 hours ago
PQShield

Director of Software Engineering

PQShield 11-50 Semiconductors & Semiconductor Equipment

PQShield is seeking a Director of Software Engineering to lead teams developing secure, scalable post-quantum cryptography software products, libraries, protocols, and architectures.

United Kingdom Austria Belgium Finland France Germany Netherlands Spain Full-time Executive Director of Engineering
Agile C C++ CI/CD Cybersecurity Embedded Systems Encryption Kanban Rust Scrum
6 days, 2 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers