RemoteLeaf Logo
Log in Sign up

Director of Software Engineering (Node.js & Web Scraping Expert)

6 days, 9 hours ago
United States
Executive
Director of Engineering
Software Development
Agile Apache Airflow Apache Spark AWS Azure CI/CD Docker Elasticsearch GCP Kafka Kanban Kubernetes Machine Learning Microservices MongoDB NLP Node.js Playwright PostgreSQL Puppeteer RabbitMQ Redis Scrum Selenium Serverless
Apply Now
PortPro

PortPro

PortPro is a leading provider of drayage software, offering the most advanced and modern operating system for drayage carriers, brokers, and companies. Their drayOS platform is designed to improve operations, save time, and increase revenue in containe...

Internet Software & Services
11-50
View All Jobs 5

Description

  • Architect, develop, and maintain scalable distributed web-scraping systems using Node.js.
  • Design data-extraction pipelines for high volumes of structured and unstructured data.
  • Develop and optimize solutions for CAPTCHA handling, session management, fingerprinting, IP rotation, and proxy services.
  • Oversee data storage and processing strategies to ensure availability, consistency, reliability, and efficiency.
  • Partner with Product, DevOps, and Data Science teams to integrate extracted data into analytics and business applications.
  • Implement microservices, API integrations, and real-time data-streaming practices.
  • Lead the transition to cloud-native, containerized, and serverless scraping architectures.
  • Manage monitoring and alerting for scraping failures, IP bans, and performance bottlenecks.
  • Optimize cloud resources, CI/CD pipelines, automated deployments, and system scalability.
  • Lead, mentor, and grow the engineering team while defining and executing the technology roadmap.

Requirements

  • 10+ years of software engineering experience, including 5+ years in web scraping and large-scale data extraction.
  • Hands-on expertise in Node.js, Puppeteer, Playwright, Cheerio, Selenium, and headless-browser automation.
  • Experience handling CAPTCHAs, IP rotation, session management, and anti-bot evasion techniques.
  • Deep knowledge of residential, datacenter, rotating, and VPN proxy management.
  • Experience with SQL and NoSQL databases such as MongoDB, PostgreSQL, Redis, and Elasticsearch.
  • Familiarity with data-processing frameworks such as Kafka, RabbitMQ, Spark, and Airflow.
  • Strong experience with CI/CD, Docker, Kubernetes, and cloud deployment on AWS, GCP, or Azure.
  • Proven ability to scale engineering teams and lead complex technical projects.
  • Strong problem-solving, debugging, communication, stakeholder-management, and mentorship skills.
  • Experience with machine learning/NLP for data extraction, browser fingerprinting, web-crawling frameworks such as Scrapy, Colly, or Apify, or data-driven/web-scraping startups (preferred).

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Ellevation Education

Director, Software Engineering

Ellevation Education 51-200 Diversified Consumer Services

Lead Ellevation Education’s cross-squad platform and security initiatives, building the technical systems and engineering leadership needed to support software used by nearly 2,000 school districts serving English Learners.

Anywhere United States Full-time Executive Director of Engineering
$200k-$250k
CI/CD Cybersecurity Penetration Testing
9 hours, 26 minutes ago
Agiloft

Senior Director, Software Engineering - Contract Intelligence

Agiloft 51-200 Capital Markets

Agiloft is seeking a Senior Director of Software Engineering to lead the Contract Intelligence organization building Astra Agents, the Contract Intelligence Graph, and shared AI-enabled platform capabilities for its data-first CLM software.

Canada Full-time Executive Director of Engineering
9 hours, 41 minutes ago
Agiloft

Senior Director, Software Engineering - Contract Intelligence

Agiloft 51-200 Capital Markets

Agiloft is seeking a Senior Director, Software Engineering - Contract Intelligence to lead the engineering strategy, teams, and delivery of its AI-powered Astra Agents, Contract Intelligence Graph, and shared CLM platform capabilities.

United States Full-time Executive Director of Engineering
2 days, 9 hours ago
PetDesk

Director of Engineering

PetDesk 51-200 Health Care Providers & Services

PetDesk is seeking a Director of Engineering to provide hands-on technical leadership for its growing veterinary platform, shaping scalable architecture and delivering reliable integrations that support clinics, pet owners, manufacturers, and partners.

United States Full-time Executive Director of Engineering
$186k-$202k
.NET
4 days, 9 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers