Senior Data Architect - Databricks Platform

1 month, 3 weeks ago
Full-time
Senior
DevOps and Infrastructure
Caylent

Caylent

Caylent is a cloud native services company that simplifies AWS cloud adoption with a user-friendly DevOps platform for scalable cloud environments.

IT Services
251-1K
Founded 2015
$16M raised

Description

  • Lead architectural design for Databricks-based data platforms and modernization programs, including workspace consolidation, permissioning/persona models, and configuration-as-code patterns.
  • Design data platform architectures that align with AWS leading practices.
  • Design high-volume batch and real-time streaming pipelines using PySpark and Structured Streaming.
  • Design and integrate Databricks Unity Catalog with third-party metadata repositories.
  • Implement automation, CI/CD, and guardrail patterns that support safe platform extension without compromising governance.
  • Build and maintain certified data products, a semantic layer, and lineage/metadata exposure for self-service and agent-driven use cases.
  • Ensure platform components are technically viable, scalable, and production-ready.
  • Communicate architectural decisions, trade-offs, assessments, and roadmaps to technical and business stakeholders.
  • Partner with client business and engineering teams to translate use-case needs into platform capabilities and guardrails.
  • Lead and facilitate technical discussions across business, technology, and knowledge stakeholders.

Requirements

  • 6+ years of experience in data engineering, database engineering, or architecture roles.
  • 3+ years of experience designing cloud-based data solutions.
  • 3+ years of experience in the AWS data landscape.
  • 3+ years of hands-on Databricks experience.
  • Experience with end-to-end modern data platform architecture, including lakehouse, ingestion, processing, and access layers.
  • Hands-on experience with Databricks workspace/persona architecture, Unity Catalog, job orchestration, governance tooling, lineage, access controls, Spark tuning, clusters, and Delta Lake storage.
  • Experience architecting scalable cloud-native systems on AWS.
  • Experience with advanced distributed processing and streaming architectures.
  • Experience with platform security, networking, automation, cost estimation, and performance optimization.
  • Experience with Terraform and IaC/automation strategy.
  • Ability to develop Databricks roadmaps and implementation roadmaps.
  • Experience leading, developing, and mentoring delivery teams.
  • High business acumen and ability to work directly with client business stakeholders.
  • Advanced English (minimum C1).
  • Experience leading multi-database migration and modernization initiatives, including onto Databricks (preferred).
  • Deep expertise in OLTP/OLAP system design and schema evolution (preferred).
  • Experience with advanced performance tuning, HA/DR, and security patterns (preferred).
  • Experience with cloud-native database architecture across managed services (preferred).

Benefits

  • Pay in USD.
  • 100% remote work from home or a shared workspace.
  • Generous holidays and unlimited flexible PTO.
  • Paid exams and certifications.
  • Peer bonus awards.
  • State-of-the-art laptop and tools.
  • Equipment and office stipend.
  • Individual professional development plan.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Arquiteto de Dados - Remoto

Capco 5K-10K Internet Software & Services

A Capco busca um(a) especialista em Integração de Dados para arquitetar e desenvolver soluções escaláveis com Informatica IDMC/CDI em projetos de tecnologia para serviços financeiros e energia.

Apache Spark Azure CI/CD Databricks dbt Git Kafka Python REST API Snowflake SQL Terraform
1 hour, 31 minutes ago

Solution Architect - Databricks

NTT DATA 100K+ IT Services

NTT DATA is seeking a remote Solution Architect – Databricks in LATAM to support Ovintiv’s enterprise Databricks migration by translating requirements into consistent, buildable solutions across Databricks and Azure.

Azure Databricks
2 hours, 16 minutes ago

Senior Data Architect - Azure & Lakehouse | México&LATAM Only | English Required

DaCodes 51-250 Internet Software & Services

DaCodes is seeking a Senior Data Architect to lead the modernization of legacy ETL workloads into scalable Azure cloud lakehouse architectures for clients across Latin America and the United States.

Apache Spark Azure CI/CD Databricks Git Kafka SQL Server
2 hours, 46 minutes ago

Associate Database Administrator

BHG Financial 1K-5K Diversified Financial Services

BHG Financial is hiring an early-career Database Administrator to support SQL Server databases across on-premises and Azure environments, maintaining reliable, secure, and high-performing systems for a financial services organization.

Azure PowerShell SQL SQL Server
1 day, 1 hour ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers