MediaRadar

MediaRadar: Revolutionizing ad sales with comprehensive advertising intelligence for media buyers and sellers.

Media

Consumer Discretionary

51-250 (150)

Founded 2006

$7M raised

7 open positions

Links

View All Jobs

Data Engineering Lead

1 month, 2 weeks ago

United States

Full-time

Lead

Data Engineer

DevOps and Infrastructure

Agile Apache Airflow Apache Spark Azure CI/CD Docker Git Machine Learning PostgreSQL Python SQL

Apply Now

MediaRadar

MediaRadar: Revolutionizing ad sales with comprehensive advertising intelligence for media buyers and sellers.

Media

51-250

Founded 2006

$7M raised

View All Jobs 7

Description

Architect and implement complex end-to-end data pipelines using Azure Databricks and PySpark.
Design, build, and maintain a scalable Medallion Architecture (Bronze/Silver/Gold) for data processing and delivery.
Spend 70-80% of time coding and performing technical stewardship as a hands-on 'player-coach' for the team.
Optimize Apache Spark jobs, tune Databricks units, define cluster policies, and implement caching strategies to minimize compute costs and improve performance.
Proactively audit and refactor pipelines every 3-6 months to maintain effectiveness and reduce cloud costs.
Develop a proactive monitoring and alerts framework to achieve 99.9% reliability and mitigate system issues before they impact end users.
Build and maintain an end-to-end Data Validation Framework (e.g., Great Expectations) to enforce data accuracy and consistency.
Ensure data is available in the Gold layer within the required 24-hour turnaround time and minimize job failure rates.
Architect high-performance PostgreSQL schemas (indexing, partitioning) and optimize complex analytical queries.
Lead a lean team toward cross-trained agility: manage sprint cycles, conduct code reviews, enforce CI/CD best practices, translate business requirements into technical user stories, and collaborate with ML and offshore teams for model integration and knowledge transfer.

Requirements

10+ years of experience in Data or Software Engineering with deep codebase involvement.
3+ years as a Technical Lead managing agile teams.
Mandatory experience with Python.
Mandatory experience with PostgreSQL and pgvector.
Mandatory experience with Azure Databricks, PySpark, and Delta Lake.
Experience with Docker, Git, Azure DevOps, and CI/CD pipelines.
Proven ability to lead lean, high-impact teams while maintaining high individual output and advocating cross-training.
Experience scaling data processing through automation and performance/cost optimization.
Desired: experience with Apache Airflow (workflow orchestration) and familiarity with Azure Kubernetes Service (AKS).
Eligible to work remotely within the USA.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

SAP BW Lead

Lingaro 5K-10K IT Services

SAP BW Lead for Poland’s CC Data Engineering & Management team at SAP, responsible for leading SAP BW-related data engineering work in a full-time remote role.

Poland Full-time Lead Data Engineer

2 hours, 8 minutes ago

Apply

2 hours, 8 minutes ago

Senior Data Engineer

Egen.ai IT Services

Egen is seeking a Senior Data Engineer to build scalable, client-facing data platforms and API integrations on Google Cloud, with a focus on healthcare data solutions.

Anywhere Full-time Senior Data Engineer

$125k-$150k

Apache Airflow AWS dbt GCP JSON Python REST API Salesforce SQL

2 hours, 48 minutes ago

Apply

2 hours, 48 minutes ago

Data Engineering Team Lead (Agentic Search)

Nebius 51-250 Internet Software & Services

Nebius is seeking a Data Engineering Team Lead to own the data platform supporting its agent-native search product, spanning ingestion, warehouse architecture, analytics, and trustworthy datasets for product and business decisions.