RemoteLeaf Logo
Log in Sign up

Databricks Engineer

4 months, 3 weeks ago
United States
Full-time
Mid Level
Data Engineer
Data Science and Analytics
Agile Apache Airflow Apache Spark AWS CI/CD Databricks Feature Engineering Generative AI HIPAA Kafka Machine Learning MLflow MLOps Prefect Python SQL Terraform
Apply Now
RevStar

RevStar

RevStar Consulting is a client-centric cloud consulting firm that specializes in developing modern, user-focused, cloud-native web and mobile applications. They offer custom integrations, implementations, and solutions using the latest cloud technologi...

Internet Software & Services
51-250
Founded 2009
View All Jobs 6

Description

  • Develop and optimize data pipelines using Apache Spark and Delta Lake within Databricks.
  • Implement ETL/ELT workflows for data ingestion, transformation, and storage.
  • Design scalable Lakehouse architecture solutions across structured and unstructured data sources.
  • Integrate Databricks with cloud storage platforms such as Azure Data Lake, AWS S3, and Google Cloud Storage.
  • Optimize Spark jobs for scalability, cost efficiency, and low latency.
  • Implement monitoring, alerting, automated validation, and data quality processes.
  • Support ML model training and deployment within Databricks using MLflow for tracking and versioning.
  • Collaborate with data scientists, ML engineers, architects, and business stakeholders to deliver aligned solutions.
  • Implement feature engineering pipelines and help integrate models into production environments.
  • Ensure data security, access control, governance, compliance, and documentation best practices.

Requirements

  • 3+ years of hands-on experience in data engineering with big data processing and cloud-native architectures.
  • 2+ years of hands-on experience with Databricks, including Apache Spark, Delta Lake, and MLflow.
  • Databricks Certified Data Engineer Associate or higher certification is mandatory.
  • Proficiency in Python, SQL, and Spark-based frameworks.
  • Experience developing and optimizing large-scale ETL/ELT pipelines.
  • Strong understanding of Lakehouse architecture and cloud-agnostic data solutions.
  • Familiarity with CI/CD pipelines and Infrastructure-as-Code tools for Databricks, such as Terraform and Databricks CLI.
  • Knowledge of data governance, security, and compliance best practices.
  • Experience working in Agile environments and following DevOps/MLOps best practices.
  • Preferred: additional Databricks certifications, such as Databricks Certified Machine Learning Associate.
  • Preferred: experience with real-time streaming tools such as Kafka, Kinesis, or Event Hub.
  • Preferred: familiarity with orchestration tools such as Apache Airflow or Prefect.
  • Preferred: background in AI/ML integration within Databricks, including feature engineering and model deployment.
  • Preferred: experience in client-facing roles or consulting environments.

Benefits

  • Paid time off.
  • Remote-first working environment.
  • Comprehensive health coverage including medical, dental, and vision.
  • 401(k) retirement plan.
  • Annual learning and development stipend for conferences, certifications, or courses.
  • Peer mentorship and coaching.
  • Professional growth opportunities with exposure to AWS GenAI, data, and cloud technologies.
  • Company outings and volunteer opportunities.
  • Collaborative, innovative culture.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Aker Systems

Senior Data Platform Engineer

Aker Systems 51-250 IT Services

Aker Systems is hiring a Senior Data Platform Engineer to build and operate secure, scalable cloud-native data platforms for public-sector and utilities clients.

United Kingdom Full-time Senior Data Engineer
Agile Ansible Apache Airflow Apache Spark Argo CD AWS Azure Bash CI/CD Datadog dbt Docker Elasticsearch ELK Stack Git GitHub GitHub Actions GitLab GitLab CI Grafana Helm Java Jenkins Kafka Kubernetes OpenSearch Packer Prometheus Python SQL Terraform
22 hours, 7 minutes ago
Capco

Engenheiro de Dados Senior - Remoto

Capco 5K-10K Internet Software & Services

A Capco é uma consultoria global de tecnologia e negócios que atua nos setores financeiro e de energia, oferecendo oportunidades de carreira em um ambiente colaborativo, inclusivo e orientado à inovação.

Brazil Full-time Senior Data Engineer
22 hours, 7 minutes ago
Karat

Senior Data Engineer (India)

Karat 251-1K Internet Software & Services

Karat is hiring a Senior Data Engineer in Bengaluru to build reliable cloud-based data infrastructure, pipelines, and models that turn interview and performance data into actionable insights across the company.

India Full-time Senior Data Engineer
Apache Airflow AWS Docker Git PostgreSQL Python SQL Terraform
22 hours, 52 minutes ago
phData

Senior Data Engineer

phData 251-1K IT Services

Data Engineer at phData, a remote-first global data and AI consultancy, designing and delivering secure, scalable cloud data solutions for enterprise clients.

India Full-time Mid Level Data Engineer
Apache Airflow Apache Spark AWS Azure Cassandra Databricks dbt Elasticsearch GCP Hadoop HDFS Kafka Luigi Python REST API Scala Snowflake Solr SQL System Design
2 days, 22 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers