RemoteLeaf Logo
Log in Sign up

Senior Data Engineer (GCP • Python • Iceberg • Delta Lake • Kafka • Snowflake • Databricks)

4 weeks, 2 days ago
United States
Full-time
Senior
Data Engineer
Software Development
Apache Spark Databricks GCP Kafka Python Snowflake
Apply Now
Railroad19

Railroad19

Railroad19 is a software development company that specializes in providing customized software solutions. They have a team of talented developers and engineers who work ahead of the curve to help businesses stay ahead of the competition. Their services...

Internet Software & Services
51-250
Founded 1997
View All Jobs 2

Description

  • Design and implement the UniForm write layer with Delta and Iceberg dual metadata.
  • Build GCS to BigQuery ingestion pipelines for structured operational datasets.
  • Develop production data solutions in Python and Spark.
  • Implement Kafka-based CDC patterns for real-time and near-real-time ingestion.
  • Develop data lineage, dependency tracking, and modular adapter code.
  • Configure Snowflake Horizon external tables for zero-copy reads.
  • Build Delta Lake tables with Iceberg UniForm enabled for reuse across consumers without conversion.
  • Implement and certify Delta Sharing endpoints for Databricks consumers.
  • Build governed access-layer components including RBAC, connector registry entries, and tenant-scoped authorization.
  • Align semantic layer models with LookML and KPI catalog definitions.
  • Collaborate with cross-functional teams to deliver end-to-end features.
  • Troubleshoot issues across the full stack and contribute to code quality.

Requirements

  • 6+ years of enterprise-level experience in Python and Spark.
  • Advanced experience with GCP BigQuery.
  • Strong working knowledge of Apache Iceberg, Delta Lake, and Iceberg UniForm.
  • Experience with Delta Sharing and Kafka/CDC pipelines.
  • Specific work experience with Snowflake Horizon Catalog and Databricks Unity Catalog.
  • Solid experience with data lake architecture and ingestion pipeline design.
  • Excellent communication skills and ability to work cohesively with multiple teams.
  • Prior delivery experience in enterprise SaaS, media, or advertising technology (preferred).
  • Active daily use of AI-assisted development tools, with Claude Code preferred (preferred).

Benefits

  • 100% remote work.
  • Competitive compensation.
  • Medical, dental, and vision insurance.
  • Pet insurance.
  • Paid time off.
  • 401(k).
  • Full-time employment only; no 1099, C2C, or agencies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Concurrency

Senior Data Engineer (Contract)

Concurrency 51-250 Internet Software & Services

Concurrency is hiring a Data Engineer for its Cloud, Data & AI Solutions Team to deliver end-to-end Microsoft Azure and Fabric data solutions that support client digital transformation.

United States Contract Senior Data Engineer
Apache Spark Azure CI/CD Git Machine Learning Power BI Python SQL
23 hours, 33 minutes ago
Netomi

Data Engineer II

Netomi 51-250 IT Services

Netomi is hiring a Data Engineer in Gurugram to build reliable data infrastructure and analytics solutions that improve AI-powered enterprise customer experiences and product operations.

India Full-time Mid Level Data Engineer
Apache Airflow Apache Spark CI/CD Databricks Docker HIPAA Kafka Kubernetes Luigi MySQL PostgreSQL Prefect Python RabbitMQ Snowflake SQL
23 hours, 33 minutes ago
CI&T

[Job-31682] Senior Data Engineer [Databricks]

CI&T 5K-10K Internet Software & Services

A CI&T busca uma pessoa engenheira de dados para estruturar e acelerar a plataforma de dados de um grande varejista farmacêutico brasileiro, modernizando a orquestração, a qualidade dos dados e a distribuição de produtos baseados em Databricks.

Brazil Full-time Senior Data Engineer
Apache Airflow Apache Spark AWS Databricks SFTP Terraform
23 hours, 48 minutes ago
payabl.

Data Engineer

payabl. 51-250 Diversified Financial Services

As a Data Engineer at payabl., you will build and maintain reliable data pipelines and business-ready datasets that support analytics and decision-making across the company’s global payments platform.

Cyprus Poland Portugal Full-time Mid Level Data Engineer
Apache Airflow Apache Spark AWS ClickHouse Dagster Databricks dbt Docker Git Kafka Kubernetes MariaDB MongoDB MySQL PostgreSQL Power BI Python Snowflake SQL Tableau Terraform
23 hours, 48 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers