Senior Data Platform Engineer CR 12/11/26

7 hours, 29 minutes ago
Contract
Senior
DevOps and Infrastructure
Margo Bank

Margo Bank

Unlock excellence with MARGO Consulting: where ambition, expertise, and innovation drive the most complex tech challenges.

Professional Services
$8M raised

Description

  • Own and evolve the Apache Airflow platform, including environments, deployments, upgrades, configuration, connections, pools, and DAG standards.
  • Lead the migration from Azure Data Factory to Airflow and enhance the metadata-driven orchestration framework.
  • Ensure reliable, observable orchestration through SLAs, alerting, retries, backfills, health monitoring, and clear failure ownership.
  • Manage Azure platform resources using Terraform, including Databricks, Data Lake Storage, Data Factory, Key Vault, networking, and supporting services.
  • Own secrets, identity, and access management across Key Vault, managed identities, Azure RBAC, Entra ID, and Unity Catalog.
  • Manage Azure DevOps repositories, CI/CD pipelines, branch policies, environment promotion, deployment validation, and Terraform state.
  • Drive Azure resource governance and cost efficiency through tagging, budgets, right-sizing, and cost attribution.
  • Operate and optimize Databricks, including Unity Catalog, cluster and job policies, Auto Loader ingestion patterns, and workload performance.
  • Contribute to data catalog, data quality, lineage, observability, developer tooling, and AI-assisted engineering workflows.
  • Provide second-line production support, lead incident resolution and root-cause analysis, and improve runbooks and operational processes.

Requirements

  • Hands-on experience managing Apache Airflow platforms and orchestration standards.
  • Experience migrating or modernizing orchestration workloads from Azure Data Factory to Apache Airflow.
  • Experience managing Azure resources with Terraform and infrastructure as code.
  • Experience with Azure services including Databricks, Data Lake Storage, Key Vault, Azure Data Factory, networking, and identity management.
  • Knowledge of Azure RBAC, Entra ID groups, managed identities, service principals, and least-privilege access controls.
  • Experience with Azure DevOps repositories, CI/CD pipelines, branch policies, and multi-environment deployments.
  • Experience operating Databricks and Unity Catalog, including governance, cluster or job policies, and workload optimization.
  • Experience designing observability, monitoring, alerting, and operational processes for data platforms.
  • Ability to assess and communicate architectural trade-offs across performance, cost, reliability, and complexity.
  • Experience troubleshooting production platform incidents, conducting root-cause analyses, and maintaining operational documentation.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Data Engineer III — Database & Platform Engineering

Playlist 11-50 Media

Playlist is hiring a Data Engineer to strengthen and modernize the database platform that supports Mindbody, Booker, and critical wellness-business production systems.

AWS Azure CI/CD Go Linux MongoDB PostgreSQL PowerShell Pulumi Python Snowflake SQL SQL Server TypeScript
7 hours, 29 minutes ago

Data Integration Engineer

D.A. Davidson Companies 1001-5000 Capital Markets

D.A. Davidson Companies is seeking a Data Integration Engineer to build, support, and optimize secure, scalable data integrations and pipelines for operational, regulatory, and analytical needs in its financial services environment.

Agile AWS Azure CI/CD Git PowerShell Python SQL SQL Server
7 hours, 29 minutes ago

Freelance Senior Quantexa Data Engineer

HITCONTRACT 11-50 Professional Services

Join Danske Bank as a fully remote Senior Quantexa Data Engineer on a 2027 contract, building scalable and reliable software solutions in a complex banking environment.

2 days, 7 hours ago

Senior Data Engineer- Data Platform

Vida Health 501-1000 Health Care Providers & Services

Vida is seeking a Senior Data Engineer to own and evolve data systems that support its virtual obesity-care platform and business operations.

Python SQL
2 days, 10 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers