RemoteLeaf Logo
Log in Sign up

Data Engineer

1 week, 2 days ago
India, Philippines
Full-time
Senior
Data Engineer
Software Development
Apache Airflow Apache Spark AWS Bash Databricks dbt Docker EC2 GitHub Actions Kubernetes MongoDB Python SQL Terraform
Apply Now
Sleek

Sleek

Sleek: The SMEs' go-to platform for company registration, compliance, and accounting services in Singapore, HK, AU, and the UK. Designed by entrepreneurs, for entrepreneurs, Sleek offers hassle-free solutions to start and manage businesses digitally.

Professional Services
251-1K
Founded 2017
$34M raised
View All Jobs 10

Description

  • Design, build, and maintain robust ETL and ELT data pipelines on Databricks and AWS.
  • Build and maintain ingestion workflows from databases, APIs, and event streams using batch and streaming patterns.
  • Implement full-load, incremental, and change data capture (CDC) ingestion strategies.
  • Develop and manage orchestration workflows in Apache Airflow.
  • Configure and maintain source-to-destination connectors using Airbyte.
  • Build and optimize data engineering workloads on the Databricks Lakehouse platform.
  • Write and optimize complex SQL queries and dbt models for data transformation.
  • Develop data models across staging, intermediate, and mart layers following data warehousing best practices.
  • Ensure data quality, observability, reliability, and documentation across the data platform.
  • Use Docker, EKS, and AWS services to containerize and support data services and applications.

Requirements

  • 3+ years of professional experience in data engineering.
  • Strong understanding of data platform architecture, including Lakehouse, Data Warehouse, and Data Lake patterns.
  • Hands-on experience designing ETL/ELT pipelines with batch processing and stream processing.
  • Familiarity with ingestion patterns such as full load, incremental, CDC, and event-driven processing.
  • 5+ years of hands-on experience as a Database Administrator in production environments.
  • Strong understanding of database internals, including storage engines, transactions, isolation levels, locking, MVCC, and query planners.
  • Experience supporting mission-critical OLTP workloads with high availability requirements.
  • Solid scripting skills in Bash and/or Python for automation.
  • Experience building data pipelines on Databricks, including Delta Live Tables, Jobs, and Notebooks.
  • Proficiency with PySpark or Spark SQL for large-scale data processing.
  • Familiarity with Delta Lake concepts such as ACID transactions, time travel, and schema evolution.
  • Proficiency with Apache Airflow for authoring, scheduling, and monitoring DAGs.
  • Experience with Airbyte for managing source-to-destination data connectors.
  • Hands-on experience administering MongoDB, including replica sets, sharding, indexing, and aggregation tuning.
  • Strong SQL skills, including query optimization, window functions, CTEs, and complex joins.
  • Experience with dbt for transformation, testing, documentation, and model layering.
  • Practical experience with AWS services such as S3, Lambda, IAM, and CloudWatch.
  • Nice to have: experience with Docker and Kubernetes (EKS) for deploying and scaling data services.
  • Nice to have: experience running Airflow and Airbyte on Kubernetes.
  • Nice to have: experience with data quality frameworks such as Great Expectations or Soda.
  • Nice to have: infrastructure as code experience with Terraform.
  • Nice to have: exposure to data governance tools or data cataloging, such as Databricks Catalog.
  • Nice to have: familiarity with CI/CD pipelines for data engineering, such as GitHub Actions.
  • Nice to have: experience with Python for pipeline scripting and automation.

Benefits

  • Competitive market salary.
  • Generous paid time off and holiday schedules.
  • Flexible work-from-home arrangements.
  • Ability to work fully remote from anywhere in the world for one month each year.
  • Flexi benefits for home office equipment or health and fitness expenses.
  • Employee share ownership plan for eligible staff.
  • Internal and external training programs.
  • A culture that emphasizes humility, kindness, diversity, and inclusion.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Bluelight Consulting

Azure Data Engineer - Remote, Latin America

Bluelight Consulting 11-50 Internet Software & Services

Bluelight is hiring a remote Azure Data Engineer to design and maintain ETL and data integration workflows for client data engineering projects across Latin America.

Mexico Latin America Full-time Mid Level Data Engineer
Agile Apache Spark Azure Git Power BI Python REST API SQL SQL Server Tableau
1 hour, 6 minutes ago
Bluelight Consulting

Azure Data Engineer - Remote, Latin America

Bluelight Consulting 11-50 Internet Software & Services

Bluelight is hiring a remote Azure Data Engineer in Latin America to design and maintain ETL and data integration processes for client analytics using Azure Synapse and Python.

Ecuador Latin America Full-time Mid Level Data Engineer
Agile Apache Spark Azure Git Machine Learning Power BI Python REST API SQL Tableau
1 hour, 16 minutes ago
tvScientific

Software Engineer II, Big Data

tvScientific 11-50 Media

tvScientific is hiring a Data Engineer to build and evolve the company’s AWS-based data infrastructure and pipelines that support its CTV advertising platform and data-heavy operations.

United States Full-time Mid Level Data Engineer
$124k-$255k
Apache Spark AWS Machine Learning Scala SQL
1 hour, 39 minutes ago
Bluelight Consulting

Data Engineer (Azure) - Remote, Latin America

Bluelight Consulting 11-50 Internet Software & Services

Bluelight is hiring a remote ETL Data Engineer in Latin America to design and maintain Azure-based data integration processes that support reliable analytics and decision-making for client projects.

Colombia Latin America Full-time Mid Level Data Engineer
Agile Apache Spark Azure Git Power BI Python REST API SQL Tableau
1 hour, 40 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers