RemoteLeaf Logo
Log in Sign up

Lead Data Engineer (Platform)

4 weeks, 2 days ago
Poland
Full-time
Lead
Data Engineer
Software Development
Apache Spark AWS Azure CI/CD Databricks Docker Git GitHub Actions GitLab CI Kubernetes Microservices Python REST API Secrets Management Serverless SQL Terraform
Apply Now
Valtech

Valtech

Valtech is a global business transformation company specializing in technology, marketing, and experience design to help clients innovate, connect with consumers, and optimize ROI.

Professional Services
5K-10K
Founded 1993
View All Jobs 47

Description

  • Design, build, and maintain scalable data engineering frameworks and platform utilities used across engineering teams.
  • Develop reusable patterns, templates, and abstractions to standardize and accelerate delivery.
  • Define and evolve platform architecture decisions to ensure scalability, maintainability, and consistency.
  • Design and implement CI/CD pipelines and automation frameworks to improve engineering velocity.
  • Define and enforce engineering standards for testing, code quality, deployment, and documentation.
  • Identify and eliminate manual or repetitive processes through automation and tooling improvements.
  • Integrate AI-assisted development tools into engineering workflows to improve productivity.
  • Develop and maintain AI engineering assets such as coding guidelines, prompt frameworks, and reusable agent configurations.
  • Lead the development and operational support of core data transformation frameworks, including dbt Core at enterprise scale.
  • Investigate and resolve framework-level issues, including deployment failures, dependency conflicts, and production incidents.
  • Support onboarding and enablement of engineering teams adopting platform tooling.
  • Partner with engineering teams to identify pain points and translate them into platform improvements.
  • Ensure platform tooling meets security, compliance, and operational requirements.
  • Conduct and support code and design reviews across platform components.
  • Monitor platform health, performance, and adoption, and iterate based on feedback and metrics.
  • Contribute to documentation, developer guides, and enablement materials to improve usability and adoption.

Requirements

  • Strong proficiency in Python for building frameworks, automation tooling, and reusable components.
  • Hands-on experience with Databricks, including notebooks, workflows, jobs, and Unity Catalog.
  • Strong SQL skills and experience with distributed processing frameworks such as Apache Spark.
  • Deep experience with dbt Core, including project structure, models, tests, macros, and deployment at scale.
  • Proven experience designing and maintaining CI/CD pipelines, such as GitHub Actions, Azure DevOps, or GitLab CI.
  • Experience with data engineering platform design, including scalable pipeline and workflow architectures.
  • Strong understanding of software engineering principles, including DRY, SOLID, and modular design.
  • Experience working with version control systems and modern Git workflows.
  • Experience with cloud platforms, preferably AWS, and infrastructure-as-code concepts such as Terraform.
  • Experience implementing automated testing strategies, including unit, integration, and data quality testing.
  • Strong understanding of platform monitoring, logging, and alerting practices.
  • Experience writing technical documentation and developer-facing guidance.
  • Experience working in complex engineering environments with multiple teams.
  • Experience with Databricks Asset Bundles (DAB) or similar deployment frameworks, preferred.
  • Experience integrating AI-assisted development tools, such as GitHub Copilot or similar, into engineering workflows, preferred.
  • Experience defining AI coding standards, agent configurations, or prompt engineering frameworks, preferred.
  • Knowledge of Delta Lake and Lakehouse architecture patterns, preferred.
  • Experience working on large-scale platform engineering or developer platform teams, preferred.
  • Understanding of IAM, secrets management, and security compliance in cloud environments, preferred.
  • Experience driving adoption of engineering platforms across multiple teams, preferred.
  • Strong communication and technical leadership skills, preferred.

Benefits

  • 24 working days of paid vacation.
  • National holidays covered.
  • Sick leave up to 20 days per year.
  • Unpaid leave up to 20 days per year.
  • Medical insurance.
  • Multisport card or Multikafeteria.
  • Maternity and paternity leave support.
  • Internal workshops and learning initiatives.
  • Professional certifications reimbursement.
  • Participation in local and global professional communities.
  • Growth Framework to support career progression.
  • Mentoring program with the option to become a mentor or mentee.
  • Competitive compensation package.
  • Remote and hybrid work options, depending on country.
  • International mobility and professional development programs.
  • Access to cutting-edge tools, training, and industry experts.
  • Progressive benefit packages that increase with tenure.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

PR Newswire

Lead Data Platform Engineer

PR Newswire 1K-5K Internet Software & Services

INFOnline, part of saas.group, is seeking a Lead Data Platform Engineer to own and evolve its GCP-native data platform that powers digital audience measurement for the German and Austrian media industry.

Europe Full-time Lead Data Engineer
CI/CD dbt Docker GCP Go Serverless SQL Terraform
2 hours, 27 minutes ago
U

OFSAA - Basel Technical Consultant

Unison Group Technology consulting

An experienced OFSAA Basel Technical Consultant is needed to design, develop, and support Basel regulatory reporting solutions for Oracle Financial Services Analytical Applications at a banking environment.

India Full-time Senior Database Administrator Data Engineer
2 hours, 42 minutes ago
Qonto

Data Engineer for AI Product

Qonto 1K-5K Banks

Qonto is hiring a Data Engineer for AI Product to build the data layer and production infrastructure that powers machine learning products for its finance workspace serving SMEs across Europe.

France Germany Spain Italy Full-time Senior Data Engineer
Apache Airflow Apache Spark CI/CD dbt Machine Learning Python
2 hours, 57 minutes ago
Trility Consulting

Senior Azure Data Consultant

Trility Consulting 51-250 Internet Software & Services

Trility Consulting is hiring a Senior Azure Data Consultant to work remotely with U.S. clients and lead data architecture and engineering efforts from initial discovery through production delivery.

United States Full-time Senior Data Engineer
Agile Azure CI/CD Databricks SQL
2 hours, 57 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers