PortPro

PortPro

PortPro is a leading provider of drayage software, offering the most advanced and modern operating system for drayage carriers, brokers, and companies. Their drayOS platform is designed to improve operations, save time, and increase revenue in containe...

Internet Software & Services
11-50

Description

  • Architect, develop, and maintain scalable distributed web-scraping systems using Node.js.
  • Design data-extraction pipelines for high volumes of structured and unstructured data.
  • Develop and optimize solutions for CAPTCHA handling, session management, fingerprinting, IP rotation, and proxy services.
  • Oversee data storage and processing strategies to ensure availability, consistency, reliability, and efficiency.
  • Partner with Product, DevOps, and Data Science teams to integrate extracted data into analytics and business applications.
  • Implement microservices, API integrations, and real-time data-streaming practices.
  • Lead the transition to cloud-native, containerized, and serverless scraping architectures.
  • Manage monitoring and alerting for scraping failures, IP bans, and performance bottlenecks.
  • Optimize cloud resources, CI/CD pipelines, automated deployments, and system scalability.
  • Lead, mentor, and grow the engineering team while defining and executing the technology roadmap.

Requirements

  • 10+ years of software engineering experience, including 5+ years in web scraping and large-scale data extraction.
  • Hands-on expertise in Node.js, Puppeteer, Playwright, Cheerio, Selenium, and headless-browser automation.
  • Experience handling CAPTCHAs, IP rotation, session management, and anti-bot evasion techniques.
  • Deep knowledge of residential, datacenter, rotating, and VPN proxy management.
  • Experience with SQL and NoSQL databases such as MongoDB, PostgreSQL, Redis, and Elasticsearch.
  • Familiarity with data-processing frameworks such as Kafka, RabbitMQ, Spark, and Airflow.
  • Strong experience with CI/CD, Docker, Kubernetes, and cloud deployment on AWS, GCP, or Azure.
  • Proven ability to scale engineering teams and lead complex technical projects.
  • Strong problem-solving, debugging, communication, stakeholder-management, and mentorship skills.
  • Experience with machine learning/NLP for data extraction, browser fingerprinting, web-crawling frameworks such as Scrapy, Colly, or Apify, or data-driven/web-scraping startups (preferred).

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Director of Engineering, Data

Xsolla 201-500 Internet Software & Services

Xsolla is seeking a Director of Engineering, Data to lead global data and ML teams building personalization, recommendation, ad-tech, and data-platform systems for its gaming commerce products.

Apache Airflow Apache Spark dbt Feature Engineering GCP Machine Learning MLflow Snowflake
4 days, 18 hours ago

Director of Engineering, Artificial Intelligence (AI)

Egen.ai IT Services

Egen is hiring a Director of Engineering to lead a 20-person span within its growing AI Practice, strengthen practice operations, and deliver production AI platform and governance solutions for enterprise clients.

GCP Machine Learning MLOps Salesforce
4 days, 18 hours ago

Software Development Director

Xsolla 201-500 Internet Software & Services

Xsolla is seeking a Software Development Director to lead its global engineering organization and deliver scalable, secure products and platforms for the video game industry.

Agile AWS Azure CI/CD GCP Microservices
4 days, 18 hours ago

Software Development Director

Xsolla 201-500 Internet Software & Services

Xsolla is seeking a Software Development Director to lead its global engineering organization and deliver scalable, secure products and platforms for the video game industry.

Agile AWS Azure CI/CD GCP Microservices
4 days, 18 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers