[Job-32197] Tech Lead Data - Databricks

4 hours, 36 minutes ago
Full-time
Lead
Software Development
CI&T

CI&T

CI&T is a global digital technology agency empowering agile growth for leading companies through advanced technologies with a team of 2000 experts worldwide.

Internet Software & Services
5001+
Founded 1995

Description

  • Provide technical leadership, design support, reviews, and decision-making for Data Engineering squads.
  • Co-define the Lakehouse technical vision, coding standards, modeling conventions, and best practices with the Data Architect.
  • Design and oversee Databricks pipelines across Bronze, Silver, and Gold layers using PySpark and Spark SQL.
  • Implement DataOps practices including Git version control, CI/CD, automated testing, and documentation.
  • Optimize pipeline performance, reliability, and cost through partitioning, Spark tuning, observability, and alerting.
  • Ensure security, access controls, and data policies in Azure and Databricks align with governance requirements.
  • Translate business and analytics requirements into scalable solutions and realistic delivery plans.
  • Mentor engineers and create templates and playbooks that promote consistent engineering standards.

Requirements

  • Strong Data Engineering experience, including technical and operational leadership as a Tech Lead or Senior professional.
  • Hands-on experience with Databricks, PySpark, Spark SQL, and batch pipelines; working knowledge of streaming.
  • Mastery of Delta Lake and practical experience applying the Bronze/Silver/Gold Medallion architecture.
  • Experience with Azure Databricks, Azure Data Lake Storage (ADLS), and identity or security services such as Azure Active Directory.
  • Strong SQL and Python skills, with knowledge of dimensional and consumption modeling, data contracts, and schemas.
  • Experience with DataOps practices, including Git, CI/CD for data pipelines, automated testing, and documentation.
  • Ability to review designs and code, resolve performance and scalability issues, and guide engineering teams.
  • Advanced English for stakeholder collaboration and technical documentation.
  • Experience with Unity Catalog, Databricks Workflows/Repos, or Delta Live Tables (preferred).
  • Experience with data-quality frameworks such as Great Expectations or Soda, dbt, Airflow, observability tools, CDC/streaming technologies, consulting, international clients, or Databricks/Azure certifications (preferred).

Benefits

  • Health, dental, and life insurance.
  • Meal and food allowance.
  • Childcare assistance and extended paternity leave.
  • Profit Sharing and Results Participation (PLR).
  • Gym and wellness partnerships through Wellhub (Gympass) and TotalPass.
  • Continuous learning through CI&T University, online learning partnerships, and a language learning platform.
  • Free physical, mental, and overall well-being platform.
  • Discount club and pregnancy and responsible-parenting course.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Senior DBA / Data Modeler / Data Engineer

Pingwind 51-200 Internet Software & Services

PingWind is seeking a remote DBA / Data Modeler / Data Engineer to lead secure data migration, modeling, integration, and database operations for the modernization of FSA IAM systems.

Cybersecurity
4 hours, 21 minutes ago

DBA / Data Modeler / Data Engineer

Pingwind 51-200 Internet Software & Services

PingWind is seeking a remote DBA / Data Modeler / Data Engineer to manage and optimize data for FSA IAM systems, enabling secure identity management, reporting, fraud analysis, and system integrations.

Cybersecurity
4 hours, 21 minutes ago

Senior Data Engineer AWS

Coderio 201-500 Internet Software & Services

Coderio is hiring a Senior Data Engineer in Argentina, Brazil, or Uruguay to build and optimize scalable data platforms for high-volume financial and transactional systems supporting international clients.

Apache Airflow Apache Spark AWS CI/CD Cybersecurity Databricks dbt Docker GCP Git GitHub Actions GitLab CI Java Kafka Python Scala Snowflake SQL Terraform
4 hours, 21 minutes ago

Senior Data Engineer - USA

Coderio 201-500 Internet Software & Services

Coderio is seeking a US-based Senior Data Engineer to build and operate scalable cloud data pipelines that power data products and ML workflows for global software projects.

Apache Spark AWS CI/CD Docker GCP Generative AI Git Linux MLOps Python REST API SQL Terraform Unix
4 hours, 21 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers