RemoteLeaf Logo
Log in Sign up

Data Engineer

1 month, 4 weeks ago
India
Full-time
Mid Level
Data Engineer
Software Development
Apache Airflow Apache Spark AWS Azure CI/CD Databricks Encryption GCP Git Machine Learning Power BI SQL
Apply Now
Irth

Irth

Irth Solutions is a leading provider of SaaS cloud-based asset protection solutions that focus on damage prevention, risk analysis, and network infrastructure resilience. With nearly three decades of experience, Irth Solutions offers a wide range of fe...

Diversified Telecommunication Services
51-250
Founded 1985
View All Jobs 9

Description

  • Build and maintain ingestion pipelines across AWS, Azure, and GCP.
  • Implement batch and streaming data pipelines using Databricks, Spark/PySpark, SQL, Delta Live Tables, and Lakeflow.
  • Apply medallion architecture patterns to transform structured data.
  • Implement CDC, SCD Type 1/2, schema evolution, and data validation processes.
  • Manage Delta Lake storage, tables, partitions, and performance optimization tasks such as OPTIMIZE, Z-ORDER, and VACUUM.
  • Support metadata management, lineage tracking, and cataloging using Unity Catalog or similar tools.
  • Assist with multi-cloud integrations between cloud storage platforms and Databricks.
  • Implement data quality validation, profiling, monitoring, and governance controls.
  • Build and manage orchestration workflows and support CI/CD and deployment automation.
  • Troubleshoot pipeline issues, recover failed jobs, and support performance tuning.
  • Work closely with the Senior Data Architect on architecture implementation and design reviews.
  • Document pipelines, data models, transformation logic, and operational procedures.

Requirements

  • 3–5 years of experience in data engineering, ETL, or cloud data platforms.
  • Experience with Databricks, Spark, PySpark, or distributed data processing.
  • Strong SQL and structured data transformation skills.
  • Experience with at least one cloud platform, with Azure preferred and AWS or GCP acceptable.
  • Knowledge of data modeling, schema evolution, pipeline troubleshooting, and data quality.
  • Understanding of security practices including RBAC, encryption, and credential management.
  • Experience with Delta Lake, medallion architecture, and lakehouse design is preferred.
  • Familiarity with Unity Catalog, Purview, Glue Catalog, or similar metadata tools is preferred.
  • Experience with orchestration tools such as ADF, Airflow, or Databricks Workflows is preferred.
  • Experience with Git, CI/CD, and DevOps practices is preferred.
  • Knowledge of Power BI, geospatial data, or AI/ML data preparation is a plus.
  • Relevant cloud or Databricks certifications such as DP-203 or Data Engineer Associate are preferred.
  • Bachelor’s or master’s degree in computer science, Software Engineering, or a related field, or equivalent professional experience.

Benefits

  • Being an integral part of a dynamic, growing company that is well respected in its industry.
  • Competitive pay based on experience.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

PR Newswire

Lead Data Platform Engineer

PR Newswire 1K-5K Internet Software & Services

INFOnline, part of saas.group, is seeking a Lead Data Platform Engineer to own and evolve its GCP-native data platform that powers digital audience measurement for the German and Austrian media industry.

Europe Full-time Lead Data Engineer
CI/CD dbt Docker GCP Go Serverless SQL Terraform
2 hours, 27 minutes ago
U

OFSAA - Basel Technical Consultant

Unison Group Technology consulting

An experienced OFSAA Basel Technical Consultant is needed to design, develop, and support Basel regulatory reporting solutions for Oracle Financial Services Analytical Applications at a banking environment.

India Full-time Senior Database Administrator Data Engineer
2 hours, 42 minutes ago
Qonto

Data Engineer for AI Product

Qonto 1K-5K Banks

Qonto is hiring a Data Engineer for AI Product to build the data layer and production infrastructure that powers machine learning products for its finance workspace serving SMEs across Europe.

France Germany Spain Italy Full-time Senior Data Engineer
Apache Airflow Apache Spark CI/CD dbt Machine Learning Python
2 hours, 57 minutes ago
Trility Consulting

Senior Azure Data Consultant

Trility Consulting 51-250 Internet Software & Services

Trility Consulting is hiring a Senior Azure Data Consultant to work remotely with U.S. clients and lead data architecture and engineering efforts from initial discovery through production delivery.

United States Full-time Senior Data Engineer
Agile Azure CI/CD Databricks SQL
2 hours, 57 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers