RemoteLeaf Logo
Log in Sign up

Sr Data Engineer

3 days, 17 hours ago
United States
Full-time
Senior
Data Engineer
Software Development
Apache Airflow Apache Spark Azure CI/CD Dagster Databricks dbt Git Python SQL
Apply Now
Protective Life

Protective Life

Protective Life Insurance Company offers financial security through insurance and investment products, serving people and protecting families for over a century.

Insurance
1001-5000
Founded 1907
View All Jobs 3

Description

  • Design, develop, and maintain production data pipelines using Databricks, Python, SQL, Apache Spark, and Delta Lake.
  • Build reliable Raw/Bronze ingestion from APIs, relational databases, files, cloud storage, and SaaS platforms, including incremental loads, state management, and failure recovery.
  • Develop Prep/Silver transformations that cleanse, validate, standardize, deduplicate, conform, and enrich source data.
  • Create Prod/Gold data products, including dimensional models, fact and bridge tables, slowly changing dimensions, aggregates, and serving tables.
  • Maintain versioned, reproducible feature and training datasets for ML engineering.
  • Define and maintain data contracts, including schemas, ownership, consumers, quality rules, freshness expectations, and backward-compatibility assessments.
  • Implement automated data-quality testing, observability, alerting, and SLA/SLO monitoring for freshness, volume, quality, latency, and cost.
  • Orchestrate and operate data assets in Dagster Cloud, including dependencies, schedules, sensors, backfills, deployments, and production troubleshooting.
  • Apply governance and security through Unity Catalog and Azure Key Vault, and build CI/CD for data assets in Azure DevOps.
  • Collaborate with architects, analysts, product owners, and business stakeholders; participate in code reviews and document data products, architecture, and runbooks.

Requirements

  • Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field; equivalent practical experience considered.
  • 3+ years of experience building and supporting production data pipelines in a cloud data platform environment.
  • Strong hands-on Python and SQL experience.
  • Experience with Databricks or a comparable Spark-based lakehouse, including Delta Lake, MERGE, and incremental-load patterns.
  • Understanding of medallion or multi-layer lakehouse architecture.
  • Experience ingesting data from APIs, relational databases, files, or SaaS applications, including incremental processing and state management.
  • Knowledge of dimensional modeling, ELT patterns, and data-quality practices.
  • Experience with orchestration tools such as Dagster, Databricks Workflows, Airflow, or Azure Data Factory.
  • Experience with Git-based source control, pull requests, automated testing, and CI/CD; Azure DevOps or comparable.
  • Ability to troubleshoot production data failures, performance bottlenecks, source changes, and monitoring or alerting issues.
  • Databricks certification and experience with Unity Catalog, dbt, dltHub or similar ingestion tools, and Dagster assets and deployments are preferred.

Benefits

  • Comprehensive health, dental, and vision insurance.
  • Mental health benefits and an employee assistance program.
  • Paid time off, paid parental leave, short-term disability, and a cultural observance day.
  • Healthcare account contributions, pension plan, and 401(k) with company matching.
  • ProHealth Rewards wellbeing platform with cash rewards.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Via Logic

Cloud Platform and Data Engineer [U.S. Citizenship Required, Public Trust Eligibility]

Via Logic 1-10 Internet Software & Services

Via Logic is hiring a remote Cloud Platform and Data Engineer to support DHS programs by operating secure AWS infrastructure and Salesforce-connected applications, integrations, and data workflows.

United States Full-time Mid Level Data Engineer Platform Engineer
$100k-$120k
Agile Ansible AWS Bash Cybersecurity DevSecOps Docker Encryption Git Helm Java Jenkins Kubernetes Microservices Monday.com New Relic PostgreSQL Power BI Python Redis Salesforce Secrets Management SQL Tableau Terraform
16 hours, 32 minutes ago
C

Data Engineer - Web Scraping (3 months contract)

CXG

Join CXG as a remote Data Engineer - Web Scraping on a 3-month contract to maintain web-scraping operations and build data pipelines that support insights for global luxury brands.

India Vietnam Pakistan Taiwan Morocco Contract Junior Data Engineer
Apache Airflow Azure dbt Git Python SQL Terraform
1 day, 5 hours ago
Inductive

Data Engineer

Inductive 51-200 Health Care Providers & Services

Join InductiveHealth as a Data Engineer to modernize and operate secure, scalable public health data pipelines that support disease surveillance and decision-making for public health agencies.

United States Full-time Mid Level Data Engineer
$130k-$145k
Agile AWS Azure CI/CD Docker Elasticsearch Encryption GitHub GitHub Actions Grafana Kafka Kubernetes OpenSearch Python Secrets Management SQL Terraform
1 day, 16 hours ago
Ampd Energy

Senior Engineer - Platform & Data Infrastructure

Ampd Energy 51-200 Electrical Equipment

Ampd Energy is seeking a hands-on senior software engineer to scale and stabilize Enernet, its cloud platform for monitoring and managing connected battery energy storage systems on construction sites.

India Full-time Senior Data Engineer
AWS ClickHouse Go Grafana GraphQL InfluxDB Kafka NestJS Node.js REST API SQL Terraform TimescaleDB TypeScript Vue.js
1 day, 16 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers