RemoteLeaf Logo
Log in Sign up

Lead Data Engineer

2 weeks, 6 days ago
United Kingdom
Full-time
Lead
Data Engineer
Software Development
Apache Airflow AWS CI/CD DynamoDB Git Kanban Microservices Pandas PostgreSQL Prefect Python Scrum SQL
Apply Now
AND Digital

AND Digital

AND Digital: Leading digital product development company accelerating world-class capabilities for ambitious businesses.

Internet Software & Services
1K-5K
Founded 2014
$26M raised
View All Jobs 6

Description

  • Lead the technical practice for the Luna data team and own the data platform migration roadmap.
  • Design and implement Python-based ETL pipelines to move data from DynamoDB to Postgres OLTP and then to Postgres OLAP.
  • Optimize Postgres transactional and analytics environments for performance and cost efficiency.
  • Establish and enforce DataOps standards, including Git-based version control, CI/CD, and schema migration practices.
  • Mentor and coach data team members in Python, Git, engineering workflows, and production-ready DataOps practices.
  • Conduct detailed code reviews and provide educational feedback to raise team quality standards.
  • Support the definition and implementation of AI tooling and engineering practices for the data team.
  • Break down larger reporting and migration initiatives into epics and stories within an Agile delivery framework.
  • Work with business stakeholders to gather requirements and translate them into scalable technical roadmaps.
  • Support the Data Product Owner with technical context to prioritize the team backlog.

Requirements

  • 7 to 10 years of experience building and supporting full-stack data pipelines from source to reporting.
  • 5+ years of deep AWS experience, ideally including administration and optimization of RDS/Aurora (Postgres) and Redshift.
  • 5+ years of expert-level Python experience, including ETL pipelines and libraries such as Pandas.
  • Experience with orchestration tools such as Airflow or Prefect.
  • 5+ years of expert-level SQL and PostgreSQL database architecture experience, including partitioning and indexing for OLTP and OLAP.
  • 3+ years of expertise in Git, including branching workflows and version control practices.
  • Experience with schema migration tools such as Liquibase, Flyway, or Drizzle.
  • Experience working in Agile environments using Scrum and Kanban.
  • Ability to work mostly remotely with occasional travel to a clubhouse or office location in London, Birmingham, Manchester, Bristol, Reading, Edinburgh, or Leeds.
  • Experience helping teams adopt DataOps practices or mentor engineers and analysts is preferred.

Benefits

  • 25 days of holiday plus bank holidays.
  • Share scheme.
  • £1,000 flexifund to spend on personalized benefits such as gym membership, Cycle to Work, or health, dental, and optical cash plan.
  • 6% pension contribution.
  • Remote-first blended working model with flexibility across home, clubhouse, and client locations.
  • Annual training and upskilling budget with allocated study days.
  • Monthly and quarterly team socials paid for by the company.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

NEORIS

Ingeniero de Datos Senior (Informatica CDI)

NEORIS 5K-10K Internet Software & Services

EPAM/NEORIS busca un Ingeniero de Datos Senior para desarrollar y asegurar soluciones de integración y plataforma de datos en un entorno multicultural con tecnologías cloud y de datos empresariales.

Mexico Full-time Senior Data Engineer
Agile Azure Databricks MongoDB Oracle PostgreSQL Power BI SQL Server
4 hours, 27 minutes ago
phData

Senior Data Engineer

phData 251-1K IT Services

phData is building a pipeline of experienced software, data, and analytics professionals to support future client work delivering production-grade solutions across modern cloud data platforms.

India Full-time Senior Data Engineer
Apache Airflow Apache Spark AWS Azure Cassandra Databricks dbt Elasticsearch GCP Hadoop HDFS Java Kafka Luigi Python Scala Snowflake Solr SQL
5 hours, 21 minutes ago
SecurityScorecard

Staff Data Engineer

SecurityScorecard 251-1K IT Services

SecurityScorecard is hiring a Staff Engineer to lead its Data Engineering team in building and maintaining large-scale data infrastructure that powers cybersecurity ratings for companies worldwide.

Canada Latin America Full-time Lead Data Engineer
$80k-$94k
Apache Airflow Apache Spark AWS ClickHouse Cybersecurity Docker Git Hive Jenkins JSON Kafka PostgreSQL Python Redis Scala Terraform XML
5 hours, 32 minutes ago
Verisian

Data Engineer

Verisian 1-10 Pharmaceuticals

Verisian is hiring a Data Engineer to build the platform and data pipelines that help clinical trial evidence, validation, and regulatory workflows move medical therapies to market faster and more safely.

Anywhere Full-time Senior Data Engineer
Docker Git GitHub Actions Java Kubernetes Next.js Python SQL
6 hours, 32 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers