RemoteLeaf Logo
Log in Sign up

Senior Data Engineer

4 hours, 29 minutes ago
Europe
Full-time
Senior
Data Engineer
Software Development
Apache Airflow AWS Azure CI/CD ClickHouse GCP GitHub Actions Go Java Python Scala Snowflake SQL
Apply Now
N-iX

N-iX

N-iX is a global software development company that helps organizations achieve lasting business value through advanced technology solutions, serving various sectors with expertise in cloud, data analytics, and IoT.

Internet Software & Services
1K-5K
Founded 2002
View All Jobs 21

Description

  • Architect and optimize scalable, high-throughput ingestion pipelines on GCP.
  • Build serverless and containerized pipelines using BigQuery, Dataflow, Pub/Sub, Cloud Storage, and GKE.
  • Design CDC and incremental update workflows for multi-platform delta synchronization.
  • Develop dbt transformation models and Airflow orchestrations using Data Vault and dimensional modeling patterns.
  • Implement hash-key and hash-diff methods for row-level asset and metadata change tracking.
  • Maintain Snowflake integration layers, including deduplication, canonical filtering, and policy restrictions.
  • Deploy and optimize ClickHouse for high-concurrency, low-latency metadata retrieval and indexing.
  • Create integration views and API query layers for real-time delta consumption.
  • Enforce metadata standardization, canonical asset mapping, and automated data validation across media asset types.
  • Collaborate with teams integrating metadata pipelines with feature extraction, search, and licensing platforms.

Requirements

  • 4+ years of production experience with GCP data technologies, including BigQuery, Dataflow, Pub/Sub, GKE, and Cloud Storage.
  • Deep proficiency with Snowflake warehousing, semi-structured queries, and integration layers.
  • Deep proficiency with ClickHouse, including real-time aggregation, columnar storage, and indexing.
  • Experience architecting CDC pipelines and incremental delta loading at multi-terabyte scale.
  • Advanced Python and SQL skills.
  • Hands-on expertise with dbt Core or Cloud and Apache Airflow in an enterprise CI/CD environment such as GitHub Actions.
  • Experience with Data Vault 2.0, hash-key generation, and raw, staging, and integration layer design preferred.
  • Experience with AI/ML metadata, computer vision feature vectors, C2PA provenance, or semantic search preferred.
  • Familiarity with media metadata schemas for images, 3D models, and video derivatives preferred.
  • Familiarity with Go, Java, Scala, AWS, Azure, or Oracle Cloud preferred.

Benefits

  • Flexible remote, office-based, or hybrid working arrangements.
  • Competitive salary and compensation package.
  • Personalized career growth opportunities.
  • Professional development through mentorship, tech talks, training, and centers of excellence.
  • Active technical communities with regular knowledge sharing.
  • Education reimbursement.
  • Corporate events, team-building activities, anniversary gifts, and location-specific benefits.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

CI&T

[Job - 32120] Principal Data Developer, Brazil

CI&T 5K-10K Internet Software & Services

CI&T is seeking a Senior Data Platform Engineer/Architect to build a new Databricks lakehouse platform for a legal-services client, transforming historical legal records into reliable data for analytics, machine learning, and business intelligence.

Brazil Full-time Lead Data Engineer
AWS AWS CDK Azure Databricks Machine Learning Terraform
4 hours ago
Zip

Senior Data Engineer

Zip 1K-5K Banks

Zip is seeking a Senior Data Engineer to build and operate scalable cloud data platforms, pipelines, and trusted data products supporting analytics, risk, finance, product, and AI initiatives.

United States Full-time Senior Data Engineer
$134k-$175k
Apache Airflow AWS Azure CI/CD Dagster Databricks dbt Git Kafka Machine Learning Python Snowflake SQL
4 hours, 59 minutes ago
Ultra Tendency

Data Engineer (m/f/*)

Ultra Tendency 51-250 Internet Software & Services

Ultra Tendency sucht deutschlandweit remote einen Data Engineer – Databricks, der für Kunden skalierbare Datenplattformen und Lakehouse-Lösungen entwickelt und sein Wissen in Workshops und der Ultra Tendency Academy vermittelt.

Germany Full-time Mid Level Data Engineer
Ansible Apache Spark CI/CD Databricks dbt Git Kafka Microservices Python SQL Terraform
4 hours, 59 minutes ago
Coforge

Lead Data Engineer

Coforge 10K-50K IT Services

Coforge is hiring a remote Lead Data Engineer in Costa Rica, Peru, Colombia, or Bolivia to lead teams and build, migrate, and scale enterprise data platforms using Azure Databricks, Snowflake, DBT, and related technologies.

Bolivia Colombia Costa Rica Peru Lead Data Engineer
AWS Azure CI/CD dbt Git Kafka Python Snowflake SQL Terraform
1 day, 4 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers