RemoteLeaf Logo
Log in Sign up

Lead Data Engineer - Identity

1 month, 1 week ago
Europe
Full-time
Lead
Data Engineer
Software Development
Apache Airflow Apache Spark Argo CD AWS ClickHouse GitHub Actions Grafana Kafka Kubernetes Prometheus Python Snowflake SQL
Apply Now
Kargo

Kargo

Kargo specializes in creating innovative advertising solutions that enhance mobile, CTV, social, and e-commerce campaigns, enabling brands to effectively connect with consumers through high-quality mobile ad experiences.

Media
251-1K
Founded 2003
View All Jobs 3

Description

  • Build and sequence a new identity graph, including identifier sync, translation, clustering, and opt-out handling.
  • Standardize partner and client onboarding across web, CTV, and mobile identifiers, including cleanroom onboarding.
  • Prepare the identity audience data layer for self-serve audience creation, activation, state, and reporting.
  • Own observability and alert response by centralizing signals, monitors, and alerts, and creating runbooks and triage context.
  • Lead and grow the domain’s data engineers through standards, code reviews, knowledge-sharing, and ADRs.
  • Set technical direction for the domain and raise engineering standards for testability and cost efficiency.
  • Collaborate with Product, Data Partnerships, and Data Science to turn ambiguity into a sequenced roadmap.
  • Develop and maintain data pipelines and transformations that support scalable ingestion and reporting.

Requirements

  • Experience designing and owning large-scale, interdependent data systems, including at least one built from scratch.
  • Experience leading engineers while staying hands-on, including setting direction, reviewing work, and developing people.
  • Strong Python, Airflow, and Spark skills, with idiomatic, testable, cost- and performance-tuned transformations.
  • Strong SQL skills, especially in Snowflake.
  • Experience with AWS and Kubernetes, including reading infrastructure logs to diagnose failures and slowness.
  • Experience working with third-party APIs inside ingestion pipelines.
  • Fluency with AI tooling in day-to-day work and an understanding of how to make codebases legible to AI tools.
  • Preferred: identity resolution or graph work in AdTech, including matching and device/household graphs.
  • Preferred: experience with privacy and consent obligations such as opt-outs, deletion, GDPR, and CCPA.
  • Preferred: experience with data cleanrooms for partner or client onboarding.
  • Preferred: CI/CD with GitHub Actions or ArgoCD, and monitoring with VictoriaMetrics, Prometheus, or Grafana.
  • Nice to have: production-scale Iceberg or a comparable table format.
  • Nice to have: streaming or near-real-time processing tools such as Kafka or Redpanda.
  • Nice to have: low-latency stores such as Aerospike.
  • Nice to have: experience with OLAP databases like Clickhouse.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

CodeRoad

Senior Data Engineer

CodeRoad 51-250 Internet Software & Services

CodeRoad is hiring a Senior Data Engineer in Latin America to build and optimize secure, agent-ready data foundations that support real-time analytics, RAG applications, and AI agent workflows.

Latin America Full-time Senior Data Engineer
Apache Airflow AWS Dagster dbt Kafka PostgreSQL Prefect SQL Terraform
42 minutes ago
Quantum Health

Vendor Integration Engineer

Quantum Health 1K-5K Professional Services

Quantum Health is seeking a Vendor Integration Engineer to design, deliver, and support partner data exchanges and APIs that enable reliable healthcare operations for its members.

United States Full-time Mid Level Data Engineer
Apache Airflow AWS GCP HIPAA JSON REST API SFTP SOAP SQL XML
1 hour, 42 minutes ago
PerfectServe

Staff Data Engineer - US Remote

PerfectServe 251-1K Internet Software & Services

PerfectServe is hiring a senior data platform engineer to turn its unified healthcare data platform strategy into a production-ready system that supports trusted analytics and business decisions across its product portfolio.

United States Full-time Lead Data Engineer
$170k-$200k
Apache Airflow AWS Dagster Databricks dbt GCP Prefect Snowflake SQL
1 hour, 42 minutes ago
Alpaca

Staff Software Engineer - Margin & Risk

Alpaca 51-250 Capital Markets

Alpaca is seeking a Staff Software Engineer to lead the architecture and development of high-consequence margin and risk systems across equities, options, crypto, and future asset classes.

Europe Full-time Lead Technical Lead
C++ Go gRPC Java Kafka Kubernetes Microservices PostgreSQL RabbitMQ Rust
1 day ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers