RemoteLeaf Logo
Log in Sign up

Lead Data Engineer (Platform)

1 week, 2 days ago
North Macedonia
Full-time
Lead
Data Engineer
Software Development
Apache Spark AWS Azure CI/CD Databricks Git GitHub Actions GitLab CI Microservices Python Secrets Management SQL Terraform
Apply Now
Valtech

Valtech

Valtech is a global business transformation company specializing in technology, marketing, and experience design to help clients innovate, connect with consumers, and optimize ROI.

Professional Services
5K-10K
Founded 1993
View All Jobs 48

Description

  • Design, build, and maintain scalable data engineering frameworks and platform utilities used across engineering teams.
  • Develop reusable patterns, templates, and abstractions to standardize and accelerate delivery.
  • Define and evolve platform architecture decisions to ensure scalability, maintainability, and consistency.
  • Design and implement CI/CD pipelines and automation frameworks to improve engineering velocity.
  • Define and enforce engineering standards for testing, code quality, deployment, and documentation.
  • Identify and eliminate manual or repetitive processes through automation and tooling improvements.
  • Integrate AI-assisted development tools into engineering workflows to improve productivity.
  • Develop and maintain AI engineering assets such as coding guidelines, prompt frameworks, and reusable agent configurations.
  • Lead the development and operational support of core data transformation frameworks, including dbt Core at enterprise scale.
  • Investigate and resolve framework-level issues, including deployment failures, dependency conflicts, and production incidents.

Requirements

  • Strong proficiency in Python for building frameworks, automation tooling, and reusable components.
  • Hands-on experience with Databricks, including notebooks, workflows, jobs, and Unity Catalog.
  • Strong SQL skills and experience with distributed processing frameworks such as Apache Spark.
  • Deep experience with dbt Core, including project structure, models, tests, macros, and deployment at scale.
  • Proven experience designing and maintaining CI/CD pipelines, such as GitHub Actions, Azure DevOps, or GitLab CI.
  • Experience with data engineering platform design, including scalable pipeline and workflow architectures.
  • Strong understanding of software engineering principles, including DRY, SOLID, and modular design.
  • Experience working with version control systems and modern Git workflows.
  • Experience with cloud platforms, preferably AWS, and infrastructure-as-code concepts such as Terraform.
  • Experience implementing automated testing strategies, including unit, integration, and data quality testing.
  • Strong understanding of platform monitoring, logging, and alerting practices.
  • Experience writing technical documentation and developer-facing guidance.
  • Experience working in complex engineering environments with multiple teams.
  • Experience with Databricks Asset Bundles or similar deployment frameworks, preferred.
  • Experience integrating AI-assisted development tools such as GitHub Copilot into engineering workflows, preferred.
  • Experience defining AI coding standards, agent configurations, or prompt engineering frameworks, preferred.
  • Knowledge of Delta Lake and Lakehouse architecture patterns, preferred.
  • Experience working on large-scale platform engineering or developer platform teams, preferred.
  • Understanding of IAM, secrets management, and security compliance in cloud environments, preferred.
  • Experience driving adoption of engineering platforms across multiple teams, preferred.
  • Strong communication and technical leadership skills, preferred.

Benefits

  • Private health insurance.
  • An education program with training and certification opportunities.
  • A wellbeing program focused on healthy work-life balance.
  • Free coffee, drinks, and snacks at work.
  • Access to company events and team activities such as ski trips, carting, laser tag, wine tasting, picnics, and cooking classes.
  • Competitive salary plus 24 days of vacation.
  • Annual company events with the full team.
  • Support for reasonable accommodations during the interview process.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Natera

Head of Data Engineering & Platform, Real-World Data (RWD)

Natera 1K-5K Pharmaceuticals

Natera is hiring a Head of Data Engineering & Platform for Real-World Data to lead the strategy, development, and delivery of a healthcare data platform that supports clinical, research, analytics, and business needs.

United States Full-time Lead Data Engineer
$154k-$193k
Agile AWS CI/CD Machine Learning PostgreSQL Python REST API Snowflake SQL
1 hour, 43 minutes ago
Knak

Senior Data Engineer

Knak 51-250 Internet Software & Services

Knak is hiring its first Data Engineer to architect the governed Snowflake data layer that powers company-wide self-serve analytics, AI agents, and department-specific data access.

Canada Full-time Senior Data Engineer
AWS Databricks dbt GCP Git LinkedIn Ads Looker Mixpanel Mode MySQL Pandas Python Salesforce Snowflake SQL Tableau
2 hours, 24 minutes ago
Lingaro

SAP BW Lead

Lingaro 5K-10K IT Services

SAP BW Lead for Poland’s CC Data Engineering & Management team at SAP, responsible for leading SAP BW-related data engineering work in a full-time remote role.

Poland Full-time Lead Data Engineer
3 hours, 38 minutes ago
Egen.ai

Senior Data Engineer

Egen.ai IT Services

Egen is seeking a Senior Data Engineer to build scalable, client-facing data platforms and API integrations on Google Cloud, with a focus on healthcare data solutions.

Anywhere Full-time Senior Data Engineer
$125k-$150k
Apache Airflow AWS dbt GCP JSON Python REST API Salesforce SQL
4 hours, 18 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers