RemoteLeaf Logo
Log in Sign up

Lead Data Engineer

11 hours, 20 minutes ago
India
Full-time
Lead
Data Engineer
Software Development
Apache Airflow Apache Spark AWS Kafka Python SQL
Apply Now
Eltropy

Eltropy

Eltropy is a digital conversations platform for CFIs, providing AI-driven communication tools to enhance operations, engagement, and productivity while ensuring security and compliance.

Communications Equipment
51-250
Founded 2013
$21M raised
View All Jobs 3

Description

  • Design, develop, and manage scalable ETL/ELT pipelines for structured and unstructured data on AWS.
  • Architect and implement data lake and data warehouse solutions with strong modeling, governance, scalability, and performance practices.
  • Own the technical design and end-to-end delivery of data engineering initiatives.
  • Build data models and pipelines that support analytics, reporting, and business intelligence use cases.
  • Automate, orchestrate, monitor, and observe data pipelines using tools such as Apache Airflow and AWS Step Functions.
  • Enable self-service analytics by designing data models and semantic layers for Generative BI tools.
  • Optimize pipelines, queries, and warehouse performance for scalability, reliability, and cost efficiency.
  • Establish and maintain standards for data quality, integrity, security, governance, and compliance.
  • Collaborate with Product, Engineering, Analytics, and Business stakeholders to translate requirements into data solutions.
  • Mentor junior and mid-level engineers and help build a high-performing data engineering team.

Requirements

  • 5+ years of professional experience in Data Engineering with ownership of complex projects.
  • Strong proficiency in Python and SQL.
  • Hands-on experience with ETL/ELT frameworks and orchestration tools such as AWS Glue, Apache Airflow, and AWS Step Functions.
  • Strong experience with the AWS data stack, including S3, Glue, Redshift, Athena, Lambda, and Kinesis or Kafka.
  • Proven experience designing and implementing data lakes, data warehouses, and scalable data pipelines.
  • Strong understanding of data modeling, dimensional modeling, data warehousing, and database design.
  • Experience with query optimization, performance tuning, and cost optimization across data platforms.
  • Experience with Spark/PySpark and/or EMR for large-scale data processing.
  • Experience enabling self-service analytics through well-designed data models and semantic layers.
  • Hands-on experience with BI/analytics platforms such as ThoughtSpot, Amazon QuickSight, or similar tools.
  • Strong understanding of data quality, governance, security, and data lifecycle management.
  • Excellent problem-solving, communication, stakeholder management, and technical leadership skills.
  • Experience leading a technical team or mentoring data engineers (preferred).
  • Experience owning projects from architecture and design through implementation and production (preferred).
  • Experience working in a SaaS or product-based environment (preferred).
  • Familiarity with data lakes, lakehouses, streaming pipelines, and event-driven architectures (preferred).
  • Experience with Kafka/Kinesis and real-time or near-real-time data pipelines (preferred).
  • AWS certifications in Data Engineering, Solutions Architecture, or related areas (preferred).
  • Contributions to open-source projects or involvement in the broader data engineering community (preferred).

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Inca Digital

Senior Data Engineer, Blockchain data and/or NLP pipelines

Inca Digital 11-50 Internet Software & Services

Inca Digital is hiring a Data Engineer to build and operate data pipelines and intelligence products that transform blockchain, market, and unstructured text data into client-facing analytics, alerts, and APIs.

United States Full-time Senior Data Engineer
Apache Airflow AWS AWS CDK Blockchain ClickHouse Dagster dbt Docker FastAPI Git GitHub Actions Go Kafka Kong Kubernetes LLM MongoDB Neo4j NLP OpenSearch PostgreSQL Prefect Python Snowflake SQL Terraform
10 hours, 35 minutes ago
Nansen

Senior Data Engineer

Nansen 51-250 Internet Software & Services

Nansen is hiring a data engineer to build and operate the data infrastructure behind its onchain analytics product, with a clear path from individual contributor to Engineering Manager.

Europe Full-time Senior Data Engineer
Blockchain ClickHouse dbt PostgreSQL Python SQL
10 hours, 50 minutes ago
Kargo

Lead Data Engineer - Identity

Kargo 251-1K Media

Kargo is hiring a senior data engineering leader to own its identity graph, onboarding pipelines, and audience data layer for adtech products across web, CTV, mobile, and in-app inventory.

Europe Full-time Lead Data Engineer Technical Lead
Apache Airflow Apache Spark Argo CD AWS ClickHouse GitHub Actions Grafana Kafka Kubernetes Prometheus Python Snowflake SQL
11 hours, 20 minutes ago
Kargo

Staff Data Platform Engineer

Kargo 251-1K Media

Kargo is hiring a founding Data Platform Engineer to build the company-wide platform that makes data products discoverable, reliable, observable, and self-serve across its advertising stack.

Europe Lead Data Engineer
Apache Spark AWS CI/CD Kubernetes Python Snowflake Terraform
11 hours, 20 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers