RemoteLeaf Logo
Log in Sign up

Senior Software Engineer (Data Platform)

4 months ago
India
Full-time
Senior
Software Engineer
Software Development
Apache Spark Databricks Kafka
Apply Now
G-P/Globalization Partners

G-P/Globalization Partners

G P offers Employer of Record solutions for global team management in 180+ countries, combining EOR, Payroll, and HCM solutions for streamlined and compliant workforce expansion.

Professional Services
1K-5K
View All Jobs 9

Description

  • Lead the design and implementation of internal SDKs and self-service frameworks for distributed engineering teams.
  • Build reusable patterns for batch and real-time event processing as part of a broader platform engineering approach.
  • Own the performance and cost-effectiveness of the Databricks ecosystem, including Spark tuning and autoscaling strategies.
  • Optimize Spark execution plans, shuffle partitions, and DBU consumption to support platform scalability.
  • Ensure the data platform remains reliable and efficient as data volume and service count grow.
  • Implement schema-on-write validation and data contracts to enforce data quality before the Bronze layer.
  • Partner with data architects and data stewards to enforce data privacy, security, and metadata lineage standards.
  • Champion AI-assisted development tools to improve engineering speed and code quality.
  • Mentor engineers on distributed computing best practices.
  • Conduct deep code reviews focused on scalability and maintainability.

Requirements

  • 5+ years of experience building and operating production-grade data systems at massive scale.
  • Deep hands-on expertise with Databricks and Spark, including Delta Lake, DLT, Spark UI debugging, and performance tuning.
  • Proven experience building real-time or streaming architectures using Spark Structured Streaming, Kafka, or Kinesis.
  • Experience managing and optimizing cloud costs in a high-growth environment.
  • Experience building APIs, tools, or frameworks used by other internal engineering teams.
  • Experience working with AI-assisted development tools such as GitHub Copilot or Cursor (preferred).
  • Experience with data governance, privacy, security, and metadata lineage practices (preferred).

Benefits

  • Competitive compensation with base salary plus annual bonus for non-sales roles or commission structure for sales roles.
  • Generous paid parental leave.
  • Flexible time off.
  • Medical, dental, and vision insurance.
  • Sabbatical after 5 years.
  • Additional compensation may vary based on qualifications, experience, education, and geographic location.
  • Reasonable accommodations for individuals with disabilities during the interview process.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Oklo

Software Engineer (Applied AI/ML)

Oklo 51-250 Electric Utilities

Oklo is hiring a Software Engineer to develop AI and machine-learning systems that support the design, analysis, and deployment of advanced fission power technology.

United States Full-time Lead Software Engineer
$200k-$250k
CI/CD Git JavaScript Machine Learning Python Rust Statistics Svelte TypeScript
11 hours, 21 minutes ago
Databricks

Sr. Forward Deployed Engineer

Databricks 1K-5K IT Services

Databricks is hiring a Forward Deployed Engineer to partner with customers on designing, building, and productionizing end-to-end data and AI solutions using the Databricks platform.

United States Full-time Senior Software Engineer
$182k-$250k
Apache Spark AWS Azure CI/CD Databricks GCP JavaScript Machine Learning MLOps Python Scala TypeScript
11 hours, 21 minutes ago
Impiricus - The HCP-Preferred Engagement Platform

Software Engineer (Full-stack)

Impiricus - The HCP-Preferred Engagement Platform 11-50 Pharmaceuticals

Impiricus is hiring a full-stack Software Engineer to build and optimize scalable web and AI-powered healthcare solutions for life sciences and healthcare users from New York, Atlanta, or remotely in the United States.

United States Full-time Junior Software Engineer
$110k-$135k
AWS Azure CI/CD GCP JavaScript LLM Next.js Node.js Python React REST API SQL TypeScript
11 hours, 21 minutes ago
Elastic

Principal Software Developer - Vector Search - Elasticsearch

Elastic 1K-5K Internet Software & Services

Elastic is hiring a Principal Software Engineer for its globally distributed Elasticsearch Search team to advance vector similarity search features, performance, relevance, and reliability.

Canada Full-time Lead Software Engineer
$114k-$180k
Cassandra CI/CD Elasticsearch Java MongoDB PostgreSQL Solr
11 hours, 36 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers