RemoteLeaf Logo
Log in Sign up

Senior Data Engineer (GCP • Python • Iceberg • Delta Lake • Kafka • Snowflake • Databricks)

1 hour, 34 minutes ago
United States
Full-time
Senior
Data Engineer
Software Development
Apache Spark Databricks GCP Kafka Python Snowflake
Apply Now
Railroad19

Railroad19

Railroad19 is a software development company that specializes in providing customized software solutions. They have a team of talented developers and engineers who work ahead of the curve to help businesses stay ahead of the competition. Their services...

Internet Software & Services
51-250
Founded 1997
View All Jobs 2

Description

  • Design and implement the UniForm write layer with Delta and Iceberg dual metadata.
  • Build GCS to BigQuery ingestion pipelines for structured operational datasets.
  • Develop production data solutions in Python and Spark.
  • Implement Kafka-based CDC patterns for real-time and near-real-time ingestion.
  • Develop data lineage, dependency tracking, and modular adapter code.
  • Configure Snowflake Horizon external tables for zero-copy reads.
  • Build Delta Lake tables with Iceberg UniForm enabled for reuse across consumers without conversion.
  • Implement and certify Delta Sharing endpoints for Databricks consumers.
  • Build governed access-layer components including RBAC, connector registry entries, and tenant-scoped authorization.
  • Align semantic layer models with LookML and KPI catalog definitions.
  • Collaborate with cross-functional teams to deliver end-to-end features.
  • Troubleshoot issues across the full stack and contribute to code quality.

Requirements

  • 6+ years of enterprise-level experience in Python and Spark.
  • Advanced experience with GCP BigQuery.
  • Strong working knowledge of Apache Iceberg, Delta Lake, and Iceberg UniForm.
  • Experience with Delta Sharing and Kafka/CDC pipelines.
  • Specific work experience with Snowflake Horizon Catalog and Databricks Unity Catalog.
  • Solid experience with data lake architecture and ingestion pipeline design.
  • Excellent communication skills and ability to work cohesively with multiple teams.
  • Prior delivery experience in enterprise SaaS, media, or advertising technology (preferred).
  • Active daily use of AI-assisted development tools, with Claude Code preferred (preferred).

Benefits

  • 100% remote work.
  • Competitive compensation.
  • Medical, dental, and vision insurance.
  • Pet insurance.
  • Paid time off.
  • 401(k).
  • Full-time employment only; no 1099, C2C, or agencies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

GoGuardian

Data Engineer II

GoGuardian 251-1K Internet Software & Services

GoGuardian is hiring a Data Engineer II to develop and improve the data platform, analytics, and AI/ML ecosystem supporting product insights and machine learning across the company.

United States Full-time Junior Data Engineer
$130k-$150k
Apache Airflow Apache Spark AWS CI/CD Dagster Databricks dbt Machine Learning MLOps Pandas Prefect Python SQL Terraform
49 minutes ago
Smart Working

Lead Data Engineer (Contract, Full-Time) [HR208] (UK)

Smart Working Internet Software & Services

Smart Working is hiring a Lead Data Engineer to architect and scale the data infrastructure behind an AI assistant for property management operations in a remote-first, long-term role.

United Kingdom Contract Lead Data Engineer
Apache Airflow Apache Spark AWS Azure Elasticsearch GCP Kafka Machine Learning MongoDB MySQL OpenSearch Pandas PostgreSQL Python
1 hour, 19 minutes ago
CI&T

[Job-31133] Senior/Mid Level, Data Engineer, Brazil

CI&T 5K-10K Internet Software & Services

CI&T is hiring a Mid/Senior Data Engineer to build scalable, intelligent data solutions for large volumes of data in a cloud and AI-driven environment.

Brazil Full-time Mid Level Data Engineer
Apache Spark AWS Datadog Kafka Python Serverless
1 hour, 34 minutes ago
CI&T

[Job - 31120] Mid Level Data Developer, Brazil

CI&T 5K-10K Internet Software & Services

CI&T is hiring a Data Developer to support a consumer electronics and audio technology client by maintaining reliable data models, transformations, and pipelines for analytics and reporting.

Brazil Full-time Mid Level Data Engineer
CI/CD dbt Git Snowflake SQL Tableau
1 hour, 34 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers