RemoteLeaf Logo
Log in Sign up

Sr Data Engineer

14 hours, 32 minutes ago
United States
Full-time
Senior
Data Engineer
Software Development
Apache Airflow Apache Spark Azure CI/CD Dagster Databricks dbt Git Python SQL
Apply Now
Protective Life

Protective Life

Protective Life Insurance Company offers financial security through insurance and investment products, serving people and protecting families for over a century.

Insurance
1K-5K
Founded 1907
View All Jobs 28

Description

  • Design, develop, and maintain Databricks pipelines using Python, SQL, Apache Spark, Delta Lake, dbt, and dlt.
  • Build reliable Bronze/Raw ingestion from APIs, databases, files, cloud storage, and SaaS platforms, including incremental loading, state management, and replay.
  • Develop Silver/Prep transformations for cleansing, validation, standardization, deduplication, conformance, and enrichment.
  • Create Gold/Prod data products, including dimensional models, facts, dimensions, aggregates, serving tables, and ML feature and training datasets.
  • Define and maintain data contracts, quality rules, freshness expectations, and backward-compatibility assessments.
  • Orchestrate assets in Dagster Cloud, including schedules, sensors, dependencies, backfills, deployments, and observability.
  • Apply Unity Catalog governance, access controls, lineage, and Azure Key Vault secret management.
  • Optimize Spark jobs and Delta Lake processing for performance and cost, including MERGE, schema evolution, time travel, and OPTIMIZE.
  • Troubleshoot production failures, data-quality issues, source changes, late or duplicate data, backfills, and recovery procedures; participate in on-call support.
  • Build CI/CD, monitoring, alerting, documentation, runbooks, and architecture standards while collaborating with technical and business stakeholders.

Requirements

  • Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent practical experience.
  • 3+ years of experience building and supporting production data pipelines in cloud data platforms.
  • Strong hands-on Python and SQL skills.
  • Experience with Databricks or a comparable Spark-based lakehouse, including Delta Lake, MERGE, and incremental loading.
  • Understanding of medallion or multi-layer lakehouse architecture and ELT patterns.
  • Experience ingesting data from APIs, relational databases, files, or SaaS applications.
  • Working knowledge of dimensional modeling, slowly changing dimensions, data quality, and data contracts.
  • Experience with orchestration tools such as Dagster, Databricks Workflows, Airflow, or Azure Data Factory.
  • Experience with Git, pull requests, automated testing, CI/CD, and Azure DevOps or comparable tools.
  • Preferred: Databricks certification, Unity Catalog, dbt on Databricks, Python Delta Lake modeling, dlt/Airbyte/Meltano/Fivetran, and Dagster assets, checks, sensors, schedules, or branch deployments.

Benefits

  • Comprehensive health, dental, and vision insurance.
  • Mental health benefits and an employee assistance program.
  • Paid time off, paid parental leave, short-term disability, and a cultural observance day.
  • Healthcare account contributions, a pension plan, and a 401(k) with company matching.
  • ProHealth Rewards wellness platform with opportunities to earn cash rewards.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

CI&T

[Job - 31856] Mid Data Developer ( Azure + Databricks ), Brasil

CI&T 5K-10K Internet Software & Services

A CI&T busca um(a) Engenheiro(a) de Dados Mid para construir soluções modernas e escaláveis de dados, contribuindo para a evolução de sua plataforma de dados em ambientes produtivos e orientados a produtos.

Brazil Full-time Mid Level Data Engineer
Apache Spark CI/CD Databricks Git Machine Learning SQL Server
14 hours, 2 minutes ago
Lingaro

Data Engineering Tech Lead - freelance

Lingaro 5K-10K IT Services

Lead a data engineering team in delivering scalable, secure, and efficient data solutions while advising customers and coordinating project execution across stakeholders.

Poland Freelance Lead Data Engineer Technical Lead
Azure CI/CD Python Scala SQL
14 hours, 17 minutes ago
CompassX Group

Senior Data Engineer (IICS & Redshift) (Independent Contractor)

CompassX Group 11-50 Internet Software & Services

CompassX is seeking an experienced independent data engineer for a short-term project supporting a client’s data integration, warehousing, and reporting initiatives in a pharmaceutical environment.

Anywhere Contract Senior Data Engineer
$125k-$156k
AWS Power BI Python SQL
14 hours, 17 minutes ago
CI&T

[Job - 31720] Data developer (Python, FICO DMPS e FICO DM (ou Blaze Advisor), Brasil

CI&T 5K-10K Internet Software & Services

Pessoa desenvolvedora sênior na CI&T para construir, evoluir e sustentar motores de decisão FICO, automatizando regras de negócio e garantindo performance, rastreabilidade, governança e integração com sistemas corporativos.

Brazil Full-time Senior Data Engineer
Apache Spark Azure Databricks Git JSON Microservices MongoDB Python REST API SQL XML
14 hours, 47 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers