RemoteLeaf Logo
Log in Sign up

Lead Data Engineer (Contract, Full-Time) [HR208]

1 week, 1 day ago
India
Contract
Lead
Data Engineer
Software Development
Apache Airflow Apache Spark AWS Azure Elasticsearch GCP Kafka Machine Learning MongoDB MySQL OpenSearch Pandas PostgreSQL Python SQL
Apply Now
Smart Working

Smart Working

Smart Working is a company that specializes in software development outsourcing and staff augmentation. They offer nearshore software development services, outsourcing solutions, and staff augmentation with a focus on providing highly skilled Indian de...

Internet Software & Services
View All Jobs 23

Description

  • Architect and build scalable data pipelines and infrastructure for AI and product systems.
  • Design and maintain data ingestion, transformation, and storage architectures.
  • Develop and manage batch and real-time data pipelines.
  • Build and optimize vector search, retrieval, and machine-learning data workflows.
  • Ensure data reliability, security, governance, and quality across the platform.
  • Collaborate with AI and backend engineers on training, inference, and product features.
  • Implement monitoring, observability, and data-quality frameworks.
  • Optimize large-scale datasets and query-system performance.
  • Define technical architecture, engineering standards, culture, and hiring practices as the founding data hire.
  • Partner with founders and product leadership to inform product decisions with data capabilities.

Requirements

  • 7+ years of professional experience, primarily in dedicated data engineering roles.
  • Strong experience designing data pipelines and distributed data systems.
  • Experience with relational databases, preferably PostgreSQL; MySQL or similar is acceptable.
  • Experience with NoSQL and vector databases.
  • Strong Python programming experience.
  • Experience building scalable backend systems and designing data models and storage architectures.
  • Ability to make and justify architectural decisions.
  • Strong understanding of data-processing performance and optimization.
  • Experience with Apache Spark, Apache Airflow, Kafka, Elasticsearch, or OpenSearch is highly desirable.
  • Experience with PostgreSQL, MongoDB, Qdrant, Milvus, pgvector, Pandas, or Polars is highly desirable.
  • Experience with AI or machine-learning platforms, stream processing, event-driven architectures, cloud infrastructure, or early-stage startups is preferred.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

8th Light

Principal Data Engineer (Remote)

8th Light 51-250 Internet Software & Services

8th Light is seeking a Principal Engineer to lead client-facing data platform architecture and delivery, turning complex software and data challenges into scalable solutions.

United States Full-time Lead Data Engineer
$152k-$221k
Apache Spark AWS CI/CD Databricks Kafka MLOps Python SQL
16 hours ago
Climate First Bank

Data Engineer

Climate First Bank 51-250 Banks

Climate First Bank is hiring a Data Engineer to build and optimize Azure-based enterprise data solutions, pipelines, and analytics capabilities for a mission-driven financial institution.

United States Full-time Lead Data Engineer
Apache Spark Azure CI/CD Databricks Encryption Git Kafka Power BI Python Scala SQL SQL Server Terraform
16 hours, 15 minutes ago
Pie Insurance

Senior Data Engineer

Pie Insurance 251-1K Insurance

Pie Insurance is hiring a hands-on Data Engineer to build and operate reliable data infrastructure supporting commercial insurance quoting, underwriting, servicing, pricing, and financial reporting.

United States Full-time Senior Data Engineer
$140k-$175k
Apache Airflow CI/CD Python Snowflake SQL
16 hours, 30 minutes ago
C

Data Software Engineer

Catena Clearing Internet Software & Services

Catena is hiring a Data Software Engineer to build and operate the ingestion, normalization, storage, and derived-data pipelines that unify telematics data from 248+ providers into a reliable API product.

United States Full-time Mid Level Data Engineer
AWS GitHub Actions PostgreSQL Python REST API Snowflake SQL TypeScript
16 hours, 45 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers