RemoteLeaf Logo
Log in Sign up

Senior Data Engineer (Databricks)

1 month ago
United States
Full-time
Senior
Data Engineer
Software Development
Apache Spark AWS Azure CI/CD Databricks dbt GCP Git Go Java Kafka MLflow MLOps Python Scala SQL
Apply Now
Livefront

Livefront

Since 2001, Livefront has helped companies grow by designing, building, and scaling digital products people love. Let's create something remarkable, together.

Internet Software & Services
11-50
View All Jobs 8

Description

  • Design and build production pipelines using Lakeflow Declarative Pipelines, Autoloader, Structured Streaming, and Declarative Automation Bundles.
  • Architect Databricks Lakehouse solutions using medallion architecture, Delta Lake, and Unity Catalog.
  • Develop and maintain DLT, PySpark, and dbt transformation layers with data quality controls and SLAs.
  • Build data and AI foundations using Unity Catalog, Feature Store, MLflow, and Model Serving.
  • Collaborate with product and backend engineers on data models, APIs, and application data contracts.
  • Consult with clients to assess data challenges, define strategies, and implement sustainable solutions.
  • Design governed data-to-application architectures using Lakebase-backed services and Databricks Apps.
  • Champion governance, access control, lineage, compliance, observability, and data quality.
  • Contribute to Livefront’s Databricks practice through accelerators, enablement, certifications, and partner materials.

Requirements

  • 7–10 years of data engineering experience, including at least 5 years in production Databricks environments.
  • Production expertise with Lakeflow Declarative Pipelines, Autoloader, Structured Streaming, Lakeflow Jobs, and Unity Catalog.
  • Experience designing production-scale Lakehouse architectures, including medallion patterns, Delta Lake, partitioning, Z-ordering, and query optimization.
  • Proficiency in SQL and Python with strong data modeling, schema design, and query optimization skills.
  • Experience with pipeline testing, observability, data quality frameworks, Git, CI/CD, and Declarative Automation Bundles.
  • Working knowledge of AWS and Azure services for Databricks deployments; GCP familiarity preferred.
  • Strong communication and problem-solving skills, including the ability to explain complex concepts to varied stakeholders.
  • Experience with Kafka, Kinesis, alternative cloud data platforms, healthcare, or fintech is preferred.
  • MLOps/LLMOps experience with MLflow, Model Serving, Vector Search, Databricks Apps, or Lakebase is preferred.
  • Databricks or Apache Spark certification, experience with Java/Go/Scala, open-source contributions, or public technical education is preferred.

Benefits

  • Compensation of $150,000–$180,000.
  • Opportunities to work on high-impact products and accounts with broad reach.
  • Respectful, trust-based, egoless collaborative work environment.
  • Professional growth through mentoring, education, community involvement, and Databricks practice-building.
  • Fast-moving hiring process with a take-home exercise allotted up to one week.
  • Equal-opportunity evaluation based on merit, competence, and qualifications.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

RevStar

Databricks Engineer

RevStar 51-250 Internet Software & Services

RevStar is seeking a remote, US-based Databricks Engineer to build and optimize cloud-agnostic Lakehouse data pipelines and MLOps frameworks for enterprise AI projects across AWS, Azure, and GCP.

United States Mexico Argentina Brazil Chile Colombia Venezuela Contract Mid Level Data Engineer
Agile Apache Airflow Apache Spark AWS Azure CI/CD Databricks Feature Engineering HIPAA Kafka MLflow MLOps Prefect Python SQL Terraform
4 hours, 58 minutes ago
phData

Senior Data Engineer

phData 251-1K IT Services

Data Engineer at phData, a remote-first global data and AI consultancy, designing and delivering secure, scalable cloud data solutions for enterprise clients.

India Full-time Mid Level Data Engineer
Apache Airflow Apache Spark AWS Azure Cassandra Databricks dbt Elasticsearch GCP Hadoop HDFS Kafka Luigi Python REST API Scala Snowflake Solr SQL System Design
1 day, 4 hours ago
Abacus Insights

Forward Deployed Data Engineer

Abacus Insights 51-250 Insurance

Abacus Insights is hiring a Forward Deployed Data Engineer to work alongside health plan clients, building governed data foundations, pipelines, and integrations that turn fragmented payer data into analytics-ready products.

United States Full-time Senior Data Engineer
Apache Spark Databricks Snowflake SQL
1 day, 4 hours ago
Azumo

Data Engineer

Azumo 51-250 Internet Software & Services

Azumo is hiring a fully remote Latin America-based Data Engineer to build and operate production ingestion, transformation, storage, warehouse, and retrieval infrastructure that supports client AI systems.

Mexico Argentina Brazil Chile Colombia Peru Full-time Senior Data Engineer
Apache Airflow Apache Spark AWS Azure CI/CD Dagster Databricks dbt Docker Flink GCP Git GitHub Actions HIPAA Jupyter Kafka Prefect Python Snowflake SQL Terraform
1 day, 4 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers