RemoteLeaf Logo
Log in Sign up

Databricks Solution Architect

1 month, 1 week ago
United States
Full-time
Lead
Data Architect
DevOps and Infrastructure
Apache Spark AWS Azure CI/CD Databricks GCP Git Kafka MLflow Python Scala SQL Terraform
Apply Now
Bounteous

Bounteous

Bounteous collaborates with ambitious brands across various industries, including finance and healthcare, to drive digital transformation by creating innovative and seamless customer experiences through advanced technologies such as AI, cloud, and data...

Internet Software & Services
1K-5K
Founded 2003
View All Jobs 14

Description

  • Architect and lead the implementation of an enterprise lakehouse on Databricks across one or more major cloud platforms.
  • Design and build scalable batch and streaming data pipelines for ingestion, transformation, and processing.
  • Define and enforce standards for data modeling, CI/CD, testing, code quality, observability, and cost optimization.
  • Lead the governance strategy for data access, lineage, auditability, and PII handling in partnership with security and compliance teams.
  • Optimize Spark workloads for performance and cost through cluster sizing, autoscaling, file layout, Z-ordering, caching, and query tuning.
  • Partner with ML engineers and data scientists to productionize models using MLflow, feature stores, and model serving.
  • Own the cloud infrastructure for the platform, including networking, IAM, secrets, encryption, and Infrastructure as Code.
  • Mentor data engineers and lead architecture reviews, code reviews, and technical design sessions.
  • Work with stakeholders across analytics, product, and finance to translate business needs into a data platform roadmap.
  • Promote and enforce information security and data privacy practices across the information lifecycle.

Requirements

  • 8+ years of data engineering experience, including 4+ years building production workloads on Databricks.
  • Deep expertise in Apache Spark, including PySpark, Spark SQL, performance tuning, partitioning strategy, and the Catalyst/Photon execution model.
  • Strong hands-on experience with Delta Lake, Unity Catalog, Databricks Workflows, and Delta Live Tables.
  • Production experience on at least one major cloud platform: AWS, Azure, or GCP.
  • Experience with cloud networking, IAM, storage services such as S3, ADLS, or GCS, and compute primitives.
  • Proficiency in Python and SQL; Scala experience is a plus.
  • Experience designing medallion architectures and dimensional models for analytics.
  • Strong CI/CD and DevOps experience with Git, Terraform, Databricks Asset Bundles or dbx, and automated testing of data pipelines.
  • Track record of leading technical projects end to end and mentoring engineers.
  • Excellent written and verbal communication skills for working with technical and business stakeholders.
  • Knowledge of data privacy and protection standards such as GDPR and CCPA.
  • Willingness to sponsor eligible candidates for employment visas.
  • Experience with MLflow, feature stores, and Databricks model serving is preferred.

Benefits

  • Competitive salary range of $102,000 to $133,000 per year.
  • Employment visa sponsorship available for eligible candidates.
  • Remote work arrangement.
  • Opportunity to work with a large global digital transformation consultancy and major enterprise clients.
  • Equal opportunity employer environment.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Dan.com - a GoDaddy brand

Data Engineer IV (6249)

Dan.com - a GoDaddy brand Internet Software & Services

itD is hiring a Data Engineer IV to build and optimize AWS-based data infrastructure and pipelines that support AI infrastructure and inference initiatives in a remote role.

United States Contract Lead Data Engineer
AWS
1 day, 19 hours ago
Xebia

Senior Data Engineer (GCP)

Xebia 1K-5K Internet Software & Services

Xebia is hiring a data engineer to support a BigQuery migration by building and maintaining scalable data pipelines for production reporting, analytics, and business intelligence solutions.

Romania Bulgaria Poland Europe Senior Data Engineer
Apache Airflow CI/CD Databricks dbt Git Python Snowflake SQL
1 day, 19 hours ago
Coupang

Staff Data Engineer (Coupang Pay)

Coupang 1K-5K Internet Software & Services

쿠팡페이 핀테크 데이터 플랫폼 팀의 Staff Data Engineer로서 실시간 결제·주문 데이터를 기반으로 데이터 파이프라인과 분석 인프라를 구축·고도화하는 역할입니다.

South Korea Full-time Lead Data Engineer
Apache Spark AWS ClickHouse Flink Hive Java Presto Python Scala SQL Tableau
1 day, 19 hours ago
Cresteo

Data Migration Engineer (SQL Server / TSQL)

Cresteo 51-200 information technology & services

Cresteo is hiring a Data Migration Engineer to own end-to-end SQL Server data migrations from legacy schemas into reconciled, production-ready data for US-based clients and international teams.

Latin America Full-time Senior Database Administrator Data Engineer
JSON SQL Server
1 day, 19 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers