RemoteLeaf Logo
Log in Sign up

Databricks Solution Architect

5 hours, 45 minutes ago
United States
Full-time
Lead
Data Architect
DevOps and Infrastructure
Apache Spark AWS Azure CI/CD Databricks GCP Git Kafka MLflow Python Scala SQL Terraform
Apply Now
Bounteous

Bounteous

Bounteous collaborates with ambitious brands across various industries, including finance and healthcare, to drive digital transformation by creating innovative and seamless customer experiences through advanced technologies such as AI, cloud, and data...

Internet Software & Services
1K-5K
Founded 2003
View All Jobs 18

Description

  • Architect and lead the implementation of an enterprise lakehouse on Databricks across one or more major cloud platforms.
  • Design and build scalable batch and streaming data pipelines for ingestion, transformation, and processing.
  • Define and enforce standards for data modeling, CI/CD, testing, code quality, observability, and cost optimization.
  • Lead the governance strategy for data access, lineage, auditability, and PII handling in partnership with security and compliance teams.
  • Optimize Spark workloads for performance and cost through cluster sizing, autoscaling, file layout, Z-ordering, caching, and query tuning.
  • Partner with ML engineers and data scientists to productionize models using MLflow, feature stores, and model serving.
  • Own the cloud infrastructure for the platform, including networking, IAM, secrets, encryption, and Infrastructure as Code.
  • Mentor data engineers and lead architecture reviews, code reviews, and technical design sessions.
  • Work with stakeholders across analytics, product, and finance to translate business needs into a data platform roadmap.
  • Promote and enforce information security and data privacy practices across the information lifecycle.

Requirements

  • 8+ years of data engineering experience, including 4+ years building production workloads on Databricks.
  • Deep expertise in Apache Spark, including PySpark, Spark SQL, performance tuning, partitioning strategy, and the Catalyst/Photon execution model.
  • Strong hands-on experience with Delta Lake, Unity Catalog, Databricks Workflows, and Delta Live Tables.
  • Production experience on at least one major cloud platform: AWS, Azure, or GCP.
  • Experience with cloud networking, IAM, storage services such as S3, ADLS, or GCS, and compute primitives.
  • Proficiency in Python and SQL; Scala experience is a plus.
  • Experience designing medallion architectures and dimensional models for analytics.
  • Strong CI/CD and DevOps experience with Git, Terraform, Databricks Asset Bundles or dbx, and automated testing of data pipelines.
  • Track record of leading technical projects end to end and mentoring engineers.
  • Excellent written and verbal communication skills for working with technical and business stakeholders.
  • Knowledge of data privacy and protection standards such as GDPR and CCPA.
  • Willingness to sponsor eligible candidates for employment visas.
  • Experience with MLflow, feature stores, and Databricks model serving is preferred.

Benefits

  • Competitive salary range of $102,000 to $133,000 per year.
  • Employment visa sponsorship available for eligible candidates.
  • Remote work arrangement.
  • Opportunity to work with a large global digital transformation consultancy and major enterprise clients.
  • Equal opportunity employer environment.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

SmithRx

Senior Staff Data Engineer

SmithRx 1K-5K Pharmaceuticals

SmithRx is seeking a Data Engineering leader to design and scale the data platforms that support pharmacy benefits analytics, governance, and machine learning in a fast-growing health-tech environment.

United States Full-time Lead Data Engineer
Apache Airflow Apache Spark C# C++ dbt Go Java LLM Looker Python Scala Snowflake SQL Superset
4 hours, 30 minutes ago
SiriusPoint

AVP, Data & AI Governance Manager

SiriusPoint 251-1K Insurance

SiriusPoint is seeking a leader of Data & AI Governance to build and run an enterprise governance framework across its global insurance and reinsurance operations.

United Kingdom Full-time Lead AI (Artificial Intelligence) Data Architect
AWS Cybersecurity Generative AI
4 hours, 45 minutes ago
3Cloud

Principal Architect – Data & AI Solutions

3Cloud 251-1K Internet Software & Services

3Cloud is hiring a Principal Architect - Data & AI SME to lead Microsoft-based data and AI solution shaping, pre-sales, and client advisory work for enterprise engagements.

United States Full-time Lead AI (Artificial Intelligence) Data Architect
$168k-$218k
Azure Databricks Machine Learning
4 hours, 45 minutes ago
Jenzabar

Data Engineer

Jenzabar 251-1K Internet Software & Services

The Data Engineer V at Jenzabar leads the design and optimization of scalable data pipelines and analytics platforms that support business insights across product, analytics, and engineering teams.

Not Available Full-time Lead Data Engineer
$85k-$100k
Agile Apache Spark Azure Databricks Git Power BI Python Scrum SQL SQL Server
5 hours, 45 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers