RemoteLeaf Logo
Log in Sign up

Data Engineer

2 months, 2 weeks ago
India
Full-time
Mid Level
Data Engineer
Software Development
Apache Airflow Apache Spark AWS Azure CI/CD Databricks Encryption GCP Git Machine Learning Power BI SQL
Apply Now
Irth

Irth

Irth Solutions is a leading provider of SaaS cloud-based asset protection solutions that focus on damage prevention, risk analysis, and network infrastructure resilience. With nearly three decades of experience, Irth Solutions offers a wide range of fe...

Diversified Telecommunication Services
51-250
Founded 1985
View All Jobs 13

Description

  • Build and maintain ingestion pipelines across AWS, Azure, and GCP.
  • Implement batch and streaming data pipelines using Databricks, Spark/PySpark, SQL, Delta Live Tables, and Lakeflow.
  • Apply medallion architecture patterns to transform structured data.
  • Implement CDC, SCD Type 1/2, schema evolution, and data validation processes.
  • Manage Delta Lake storage, tables, partitions, and performance optimization tasks such as OPTIMIZE, Z-ORDER, and VACUUM.
  • Support metadata management, lineage tracking, and cataloging using Unity Catalog or similar tools.
  • Assist with multi-cloud integrations between cloud storage platforms and Databricks.
  • Implement data quality validation, profiling, monitoring, and governance controls.
  • Build and manage orchestration workflows and support CI/CD and deployment automation.
  • Troubleshoot pipeline issues, recover failed jobs, and support performance tuning.
  • Work closely with the Senior Data Architect on architecture implementation and design reviews.
  • Document pipelines, data models, transformation logic, and operational procedures.

Requirements

  • 3–5 years of experience in data engineering, ETL, or cloud data platforms.
  • Experience with Databricks, Spark, PySpark, or distributed data processing.
  • Strong SQL and structured data transformation skills.
  • Experience with at least one cloud platform, with Azure preferred and AWS or GCP acceptable.
  • Knowledge of data modeling, schema evolution, pipeline troubleshooting, and data quality.
  • Understanding of security practices including RBAC, encryption, and credential management.
  • Experience with Delta Lake, medallion architecture, and lakehouse design is preferred.
  • Familiarity with Unity Catalog, Purview, Glue Catalog, or similar metadata tools is preferred.
  • Experience with orchestration tools such as ADF, Airflow, or Databricks Workflows is preferred.
  • Experience with Git, CI/CD, and DevOps practices is preferred.
  • Knowledge of Power BI, geospatial data, or AI/ML data preparation is a plus.
  • Relevant cloud or Databricks certifications such as DP-203 or Data Engineer Associate are preferred.
  • Bachelor’s or master’s degree in computer science, Software Engineering, or a related field, or equivalent professional experience.

Benefits

  • Being an integral part of a dynamic, growing company that is well respected in its industry.
  • Competitive pay based on experience.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Career TEAM

Data Conversion Software Engineer

Career TEAM 251-1K Professional Services

Career Team is hiring a Data Conversion Software Engineer to build data transformation and integration software for government-funded workforce development programs across the United States.

United States Full-time Mid Level Data Engineer Software Engineer
$60k-$100k
Agile Angular CI/CD Docker Express.js JavaScript JSON MongoDB NestJS Next.js Node.js React Scrum TypeScript XML
12 hours, 59 minutes ago
TWO95 International

Sr. Associate Data Platform - Remote

TWO95 International 51-250 Internet Software & Services

Sr. Associate Data Platform is a contract role with a Los Angeles-based team supporting Adobe analytics and data platform implementation work across on-site and remote locations.

United States Contract Senior Data Engineer
CSS Digital Marketing HTML JavaScript jQuery Vue.js
13 hours, 14 minutes ago
Mindrift.ai: Be the “I” in AI

Freelance Data Scraping Engineer (Python)

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is hiring a part-time remote Python Data Scraping Engineer for the Tendem project to deliver accurate, structured data extraction and processing within a hybrid AI-plus-human workflow.

Kuwait Qatar Saudi Arabia Turkey Israel Jordan Part-time Junior Backend Engineer Data Engineer
Up to $37k
AJAX GitHub JavaScript JSON LLM Python Selenium
13 hours, 14 minutes ago
Mindrift.ai: Be the “I” in AI

Freelance Data Scraping Engineer (Python)

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is hiring a part-time freelance Python Data Scraping Engineer for the Tendem project to build and oversee data extraction workflows in a hybrid AI + human environment.

United States Part-time Mid Level Backend Engineer Data Engineer
AJAX GitHub JavaScript JSON LLM Python Selenium
13 hours, 14 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers