RemoteLeaf Logo
Log in Sign up

Lead Data Engineer (Contract, Full-Time) [HR208] (UK)

1 month, 2 weeks ago
United Kingdom
Contract
Lead
Data Engineer
Software Development
Apache Airflow Apache Spark AWS Azure Elasticsearch GCP Kafka Machine Learning MongoDB MySQL OpenSearch Pandas PostgreSQL Python
Apply Now
Smart Working

Smart Working

Smart Working is a company that specializes in software development outsourcing and staff augmentation. They offer nearshore software development services, outsourcing solutions, and staff augmentation with a focus on providing highly skilled Indian de...

Internet Software & Services
View All Jobs 15

Description

  • Architect and build scalable data pipelines and infrastructure to support AI and product systems.
  • Design and maintain data ingestion, transformation, and storage architectures for operational and AI workloads.
  • Develop and manage batch and real-time data pipelines.
  • Build and optimize systems for vector search, retrieval, and machine learning data pipelines.
  • Ensure data reliability, security, and governance across the platform.
  • Collaborate with AI, backend, product, and leadership teams to support training, inference, and product features.
  • Implement monitoring, observability, and data quality frameworks.
  • Optimize the performance of large-scale datasets and query systems.
  • Contribute to technical architecture decisions and long-term data strategy.
  • Act as the founding data hire and help define the culture, standards, and hiring bar for the growing data function.

Requirements

  • 7+ years of professional experience, with most of that in dedicated data engineering roles.
  • Strong experience designing and building data pipelines and distributed data systems.
  • Experience with relational databases, with PostgreSQL preferred; MySQL or similar acceptable.
  • Experience working with NoSQL databases.
  • Experience with vector databases used in modern AI systems.
  • Strong programming experience in Python.
  • Demonstrated ability to make and justify architectural decisions.
  • Experience building scalable backend systems.
  • Experience designing data models and storage architectures.
  • Strong understanding of data processing performance and optimization.
  • Experience with Apache Spark, Apache Airflow, Kafka, and Elasticsearch or OpenSearch is highly desirable.
  • Experience with PostgreSQL, MongoDB, and vector databases such as Qdrant, Milvus, or pgvector is highly desirable.
  • Experience with Python data-processing libraries such as Pandas or Polars is highly desirable.
  • Experience working on AI or machine learning platforms (nice to have).
  • Familiarity with stream processing and event-driven architectures (nice to have).
  • Experience with cloud infrastructure such as GCP, AWS, or Azure (nice to have).
  • Experience working in high-growth startups or early-stage companies (nice to have).

Benefits

  • Remote-first work environment.
  • Full-time, long-term role.
  • Opportunity to join a genuine community focused on growth and well-being.
  • Work with outstanding global teams and products.
  • Exposure to a high-rated workplace recognized on Glassdoor.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Aker Systems

Senior Data Platform Engineer

Aker Systems 51-250 IT Services

Aker Systems is hiring a Senior Data Platform Engineer to build and operate secure, scalable cloud-native data platforms for public-sector and utilities clients.

United Kingdom Full-time Senior Data Engineer
Agile Ansible Apache Airflow Apache Spark Argo CD AWS Azure Bash CI/CD Datadog dbt Docker Elasticsearch ELK Stack Git GitHub GitHub Actions GitLab GitLab CI Grafana Helm Java Jenkins Kafka Kubernetes OpenSearch Packer Prometheus Python SQL Terraform
21 hours, 24 minutes ago
Capco

Engenheiro de Dados Senior - Remoto

Capco 5K-10K Internet Software & Services

A Capco é uma consultoria global de tecnologia e negócios que atua nos setores financeiro e de energia, oferecendo oportunidades de carreira em um ambiente colaborativo, inclusivo e orientado à inovação.

Brazil Full-time Senior Data Engineer
21 hours, 24 minutes ago
Karat

Senior Data Engineer (India)

Karat 251-1K Internet Software & Services

Karat is hiring a Senior Data Engineer in Bengaluru to build reliable cloud-based data infrastructure, pipelines, and models that turn interview and performance data into actionable insights across the company.

India Full-time Senior Data Engineer
Apache Airflow AWS Docker Git PostgreSQL Python SQL Terraform
22 hours, 9 minutes ago
RevStar

Databricks Engineer

RevStar 51-250 Internet Software & Services

RevStar is seeking a remote, US-based Databricks Engineer to build and optimize cloud-agnostic Lakehouse data pipelines and MLOps frameworks for enterprise AI projects across AWS, Azure, and GCP.

United States Mexico Argentina Brazil Chile Colombia Venezuela Contract Mid Level Data Engineer
Agile Apache Airflow Apache Spark AWS Azure CI/CD Databricks Feature Engineering HIPAA Kafka MLflow MLOps Prefect Python SQL Terraform
1 day, 22 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers