RemoteLeaf Logo
Log in Sign up

Lead Data Engineer

3 hours, 18 minutes ago
United Kingdom
Full-time
Lead
Data Engineer
Software Development
Apache Airflow AWS CI/CD DynamoDB Git Kanban Microservices Pandas PostgreSQL Prefect Python Scrum SQL
Apply Now
AND Digital

AND Digital

AND Digital: Leading digital product development company accelerating world-class capabilities for ambitious businesses.

Internet Software & Services
1K-5K
Founded 2014
$26M raised
View All Jobs 3

Description

  • Lead the technical practice for the Luna data team and own the data platform migration roadmap.
  • Design and implement Python-based ETL pipelines to move data from DynamoDB to Postgres OLTP and then to Postgres OLAP.
  • Optimize Postgres transactional and analytics environments for performance and cost efficiency.
  • Establish and enforce DataOps standards, including Git-based version control, CI/CD, and schema migration practices.
  • Mentor and coach data team members in Python, Git, engineering workflows, and production-ready DataOps practices.
  • Conduct detailed code reviews and provide educational feedback to raise team quality standards.
  • Support the definition and implementation of AI tooling and engineering practices for the data team.
  • Break down larger reporting and migration initiatives into epics and stories within an Agile delivery framework.
  • Work with business stakeholders to gather requirements and translate them into scalable technical roadmaps.
  • Support the Data Product Owner with technical context to prioritize the team backlog.

Requirements

  • 7 to 10 years of experience building and supporting full-stack data pipelines from source to reporting.
  • 5+ years of deep AWS experience, ideally including administration and optimization of RDS/Aurora (Postgres) and Redshift.
  • 5+ years of expert-level Python experience, including ETL pipelines and libraries such as Pandas.
  • Experience with orchestration tools such as Airflow or Prefect.
  • 5+ years of expert-level SQL and PostgreSQL database architecture experience, including partitioning and indexing for OLTP and OLAP.
  • 3+ years of expertise in Git, including branching workflows and version control practices.
  • Experience with schema migration tools such as Liquibase, Flyway, or Drizzle.
  • Experience working in Agile environments using Scrum and Kanban.
  • Ability to work mostly remotely with occasional travel to a clubhouse or office location in London, Birmingham, Manchester, Bristol, Reading, Edinburgh, or Leeds.
  • Experience helping teams adopt DataOps practices or mentor engineers and analysts is preferred.

Benefits

  • 25 days of holiday plus bank holidays.
  • Share scheme.
  • £1,000 flexifund to spend on personalized benefits such as gym membership, Cycle to Work, or health, dental, and optical cash plan.
  • 6% pension contribution.
  • Remote-first blended working model with flexibility across home, clubhouse, and client locations.
  • Annual training and upskilling budget with allocated study days.
  • Monthly and quarterly team socials paid for by the company.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

MUTT DATA

Data Engineer Senior

MUTT DATA 51-250 Internet Software & Services

Mutt Data is hiring a Senior Data Engineer in a remote Argentina-based role to build and improve data infrastructure and machine learning systems for client projects.

Argentina Full-time Senior Data Engineer
Apache Airflow Apache Spark AWS Azure CI/CD Databricks dbt Deep Learning Docker GCP Generative AI Jupyter Machine Learning NumPy Pandas Prefect Python SQL
3 minutes ago
qodea

Data Engineer

qodea

Qodea is hiring a Data Engineer to design, build, and maintain scalable data pipelines and infrastructure for large, complex datasets in a flexible, collaboration-driven environment.

Romania Full-time Mid Level Data Engineer
Apache Airflow Apache Spark AWS Azure GCP Hadoop Hive Java MySQL Oracle PostgreSQL Python Scala SQL
3 minutes ago
qodea

Principal Data Engineering

qodea

Qodea is hiring a Data Engineering Team Lead to lead consulting delivery and guide enterprise clients on modern cloud data architecture using Google Cloud solutions.

Romania Full-time Lead Data Engineer Engineering Manager
Agile Apache Spark GCP Hadoop Java Python Scala Terraform
3 minutes ago
Netomi

SDE II - Python Developer

Netomi 51-250 IT Services

Netomi is hiring an SDE II Python Developer in Gurugram to build and operate scalable software systems and data pipelines that support production AI and customer experience products for major enterprise brands.

India Full-time Mid Level Backend Engineer Data Engineer
Apache Airflow AWS EC2 Elasticsearch GCP Hugging Face Kubernetes Machine Learning MLflow MySQL Python Redis System Design
18 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers