RemoteLeaf Logo
Log in Sign up

Senior Data Engineer

2 months, 3 weeks ago
South Africa
Full-time
Senior
Data Engineer
Software Development
Apache Spark Azure Databricks Flink IoT Kafka Machine Learning MQTT Python SFTP SQL
Apply Now
Sand Technologies

Sand Technologies

Sand Technologies is a leading provider of enterprise AI solutions, offering cutting-edge technology to solve complex challenges. With a focus on outcomes over hype, we work with global enterprises to identify meaningful AI use cases, develop custom al...

Internet Software & Services
View All Jobs 5

Description

  • Design and build reusable data ingestion pipelines from operational source systems into the Azure data platform.
  • Define canonical data models and transformation logic for analytical indices and the intelligence layer.
  • Design high-performance schemas using lakehouse patterns for storage, retrieval, and analysis.
  • Establish and document ingestion and modelling standards that other engineers can reuse consistently.
  • Partner with client data owners, stewards, and enterprise architects to agree access, security, and integration approaches.
  • Own data quality, lineage, observability, and pipeline reliability across the platform.
  • Implement governance and security controls appropriate for a regulated critical-infrastructure environment.
  • Mentor and unblock mid-level and junior data engineers and review their work.
  • Promote engineering best practice across the squad.
  • Contribute across a portfolio of water and energy utility clients as the platform grows.

Requirements

  • Proven experience as a Senior Data Engineer with hands-on production pipeline and data architecture experience.
  • Strong commercial experience with the Azure data platform, such as Data Factory, Synapse, Fabric, ADLS, or Databricks.
  • Expert-level SQL and Python skills.
  • Solid data modelling experience across dimensional and/or lakehouse patterns.
  • Experience integrating heterogeneous source data via API, file transfer, batch, or streaming.
  • Experience setting standards and patterns and mentoring or leading other engineers.
  • Ability to work directly with client data owners, stewards, and architects and communicate with non-technical stakeholders.
  • Knowledge of data governance, quality frameworks, and security practices in regulated environments.
  • Exposure to operational technology (OT)/SCADA, IoT/asset telemetry, or MQTT systems is desirable.
  • Utilities, water, or other asset-intensive industry experience is desirable.
  • Experience with Kafka, Spark/Spark Streaming, or Flink is desirable.
  • Understanding of machine learning workflows and how to support them with robust data pipelines is desirable.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

RevStar

Databricks Engineer

RevStar 51-250 Internet Software & Services

RevStar is seeking a remote, US-based Databricks Engineer to build and optimize cloud-agnostic Lakehouse data pipelines and MLOps frameworks for enterprise AI projects across AWS, Azure, and GCP.

United States Mexico Argentina Brazil Chile Colombia Venezuela Contract Mid Level Data Engineer
Agile Apache Airflow Apache Spark AWS Azure CI/CD Databricks Feature Engineering HIPAA Kafka MLflow MLOps Prefect Python SQL Terraform
4 hours, 59 minutes ago
phData

Senior Data Engineer

phData 251-1K IT Services

Data Engineer at phData, a remote-first global data and AI consultancy, designing and delivering secure, scalable cloud data solutions for enterprise clients.

India Full-time Mid Level Data Engineer
Apache Airflow Apache Spark AWS Azure Cassandra Databricks dbt Elasticsearch GCP Hadoop HDFS Kafka Luigi Python REST API Scala Snowflake Solr SQL System Design
1 day, 4 hours ago
Abacus Insights

Forward Deployed Data Engineer

Abacus Insights 51-250 Insurance

Abacus Insights is hiring a Forward Deployed Data Engineer to work alongside health plan clients, building governed data foundations, pipelines, and integrations that turn fragmented payer data into analytics-ready products.

United States Full-time Senior Data Engineer
Apache Spark Databricks Snowflake SQL
1 day, 4 hours ago
Azumo

Data Engineer

Azumo 51-250 Internet Software & Services

Azumo is hiring a fully remote Latin America-based Data Engineer to build and operate production ingestion, transformation, storage, warehouse, and retrieval infrastructure that supports client AI systems.

Mexico Argentina Brazil Chile Colombia Peru Full-time Senior Data Engineer
Apache Airflow Apache Spark AWS Azure CI/CD Dagster Databricks dbt Docker Flink GCP Git GitHub Actions HIPAA Jupyter Kafka Prefect Python Snowflake SQL Terraform
1 day, 4 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers