RemoteLeaf Logo
Log in Sign up

Databricks Engineer

2 months, 1 week ago
United States
Full-time
Mid Level
Data Engineer
Data Science and Analytics
Agile Apache Airflow Apache Spark AWS CI/CD Databricks Feature Engineering Generative AI HIPAA Kafka Machine Learning MLflow MLOps Prefect Python SQL Terraform
Apply Now
RevStar

RevStar

RevStar Consulting is a client-centric cloud consulting firm that specializes in developing modern, user-focused, cloud-native web and mobile applications. They offer custom integrations, implementations, and solutions using the latest cloud technologi...

Internet Software & Services
51-250
Founded 2009
View All Jobs 9

Description

  • Develop and optimize data pipelines using Apache Spark and Delta Lake within Databricks.
  • Implement ETL/ELT workflows for data ingestion, transformation, and storage.
  • Design scalable Lakehouse architecture solutions across structured and unstructured data sources.
  • Integrate Databricks with cloud storage platforms such as Azure Data Lake, AWS S3, and Google Cloud Storage.
  • Optimize Spark jobs for scalability, cost efficiency, and low latency.
  • Implement monitoring, alerting, automated validation, and data quality processes.
  • Support ML model training and deployment within Databricks using MLflow for tracking and versioning.
  • Collaborate with data scientists, ML engineers, architects, and business stakeholders to deliver aligned solutions.
  • Implement feature engineering pipelines and help integrate models into production environments.
  • Ensure data security, access control, governance, compliance, and documentation best practices.

Requirements

  • 3+ years of hands-on experience in data engineering with big data processing and cloud-native architectures.
  • 2+ years of hands-on experience with Databricks, including Apache Spark, Delta Lake, and MLflow.
  • Databricks Certified Data Engineer Associate or higher certification is mandatory.
  • Proficiency in Python, SQL, and Spark-based frameworks.
  • Experience developing and optimizing large-scale ETL/ELT pipelines.
  • Strong understanding of Lakehouse architecture and cloud-agnostic data solutions.
  • Familiarity with CI/CD pipelines and Infrastructure-as-Code tools for Databricks, such as Terraform and Databricks CLI.
  • Knowledge of data governance, security, and compliance best practices.
  • Experience working in Agile environments and following DevOps/MLOps best practices.
  • Preferred: additional Databricks certifications, such as Databricks Certified Machine Learning Associate.
  • Preferred: experience with real-time streaming tools such as Kafka, Kinesis, or Event Hub.
  • Preferred: familiarity with orchestration tools such as Apache Airflow or Prefect.
  • Preferred: background in AI/ML integration within Databricks, including feature engineering and model deployment.
  • Preferred: experience in client-facing roles or consulting environments.

Benefits

  • Paid time off.
  • Remote-first working environment.
  • Comprehensive health coverage including medical, dental, and vision.
  • 401(k) retirement plan.
  • Annual learning and development stipend for conferences, certifications, or courses.
  • Peer mentorship and coaching.
  • Professional growth opportunities with exposure to AWS GenAI, data, and cloud technologies.
  • Company outings and volunteer opportunities.
  • Collaborative, innovative culture.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

D

Cyber Data Engineer

D2 Technical Services 11-50 IT services and consulting

D2 Technical Services is hiring a systems engineer to support national security cybersecurity operations by managing infrastructure, data pipelines, and SIEM environments.

United States Full-time Senior Data Engineer Security Engineer
$140k-$145k
AWS Bash Cybersecurity DHCP DNS Elasticsearch GCP Linux PowerShell Python SIEM Splunk
23 hours, 8 minutes ago
Voyager

Senior DataOps Engineer

Voyager Life Sciences Tools & Services

Voyager Technologies is hiring a remote US-based DataOps Engineer to own cloud-to-data platform migration and operations for mission-critical analytics in a regulated national security environment.

United States Full-time Senior Data Engineer
$150k-$180k
Ansible Apache Airflow Apache Spark AWS Azure Cassandra CloudFormation dbt Docker Git GitHub GitHub Actions GitLab GitLab CI Kafka Kubernetes MongoDB Pandas PostgreSQL PowerShell Python Redis Scala SQLAlchemy SQL Server Terraform
23 hours, 38 minutes ago
Dragos

Data Engineering Manager

Dragos 251-1K Professional Services

Dragos is seeking a Data Engineering Manager to lead the team building the core data layer that supports its cybersecurity products and threat-focused engineering work.

United States Full-time Lead Data Engineer Engineering Manager
$220k-$220k
Agile Apache Airflow ClickHouse Cybersecurity dbt Docker Kubernetes PostgreSQL Rust SOC
23 hours, 38 minutes ago
Movement Labs

Senior Data Engineers

Movement Labs 11-50 Media

Movement Labs is hiring Senior and Mid-level Data Engineers to support its progressive political data platform, with a focus on building and operating systems for voter outreach, experimentation, and election readiness.

United States Contract Senior Data Engineer
$90k-$119k
Apache Airflow Dagster dbt GCP Python SQL
23 hours, 53 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers