RemoteLeaf Logo
Log in Sign up

Senior Staff Software Engineer, Data Infrastructure

1 day, 19 hours ago
United States
Full-time
Lead
Software Engineer
Software Development
Apache Airflow Apache Spark AWS ClickHouse Databricks dbt Kafka Microservices PostgreSQL Python Scala Snowflake SQL Trino
Apply Now
instacart.careers

instacart.careers

Instacart is a leading grocery technology company in North America that works with grocers and retailers to transform how people shop. They partner with over 1,000 national, regional, and local retail banners to facilitate online shopping, delivery, an...

Internet Software & Services
1K-5K
View All Jobs 124

Description

  • Own the multi-year technical vision and roadmap for Instacart’s core data platform.
  • Translate company data strategy into an actionable architecture plan for storage, compute, streaming, orchestration, and serving.
  • Lead build-vs-buy decisions, including managed services versus open-source self-hosting, for major platform investments.
  • Develop business cases, cost-benefit models, and risk analyses for infrastructure initiatives.
  • Drive the architecture and delivery of the open data lakehouse foundation, including table formats, compute engines, storage governance, and data sharing standards.
  • Own the architecture for streaming data, event-driven pipelines, stream processing, and real-time serving for critical business use cases.
  • Lead adoption of AI and LLM tools across the data platform lifecycle and partner on AI-powered capabilities.
  • Set system design and reliability standards, lead architecture reviews, and mentor senior engineers.
  • Collaborate with Data Science, ML Platform, Ads Infra, Product Engineering, Finance Engineering, and Security on self-serve infrastructure needs.
  • Represent Data Infrastructure in technical and executive forums and communicate trade-offs clearly to stakeholders.

Requirements

  • 10+ years of software engineering experience focused on data infrastructure or distributed systems at scale.
  • Experience setting technical direction for large-scale data platforms and defining multi-year architecture roadmaps.
  • Experience in high-growth, data-intensive environments with significant infrastructure scale and spend.
  • Expertise in modern data lakehouse architectures and open table formats such as Iceberg, Delta Lake, or Hudi.
  • Experience with distributed query and compute systems such as Trino, Spark, or ClickHouse.
  • Experience with event-driven infrastructure such as Kafka or Flink.
  • Proven track record leading major infrastructure transitions, including build-vs-buy decisions, migration design, and risk management.
  • Experience building business cases for infrastructure investments, including cost-benefit analysis and TCO modeling.
  • Strong technical communication skills for architecture documents, strategy memos, and proposals.
  • Preferred experience with data governance and compliance frameworks such as SOX, CPRA, or GDPR.
  • Preferred experience with FinOps, cost optimization, multi-million-dollar infrastructure budgets, and vendor contract negotiation.
  • Deep knowledge of SQL and strong proficiency in Python or Scala.
  • Preferred experience with orchestration tools such as Apache Airflow and transformation tools such as dbt.
  • Track record of building and growing high-performing data infrastructure teams.
  • Bachelor’s, Master’s, or PhD in Computer Science, Computer Engineering, Electrical Engineering, or equivalent practical experience.

Benefits

  • Remote Flex First work environment with flexibility to work from home, an office, or another location.
  • Highly market-competitive compensation.
  • Base salary range of $254,000 to $321,000 USD depending on location.
  • Eligible for a new hire equity grant.
  • Eligible for annual refresh equity grants.
  • Access to Instacart benefits offerings and employee perks.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

SoFi

Senior Software Engineer, Protect

SoFi 1K-5K Capital Markets

SoFi is hiring a Senior Software Engineer for its Protect team to help build a next-generation insurance platform and shape the technical direction of a greenfield, high-impact business area.

United States Full-time Senior Software Engineer
AWS CI/CD Docker DynamoDB Git Java JavaScript Kafka Kotlin Kubernetes LLM Microservices PostgreSQL React Spring TypeScript
1 day, 5 hours ago
Coinbase

Software Engineer, Developer (Wallets and Onchain Tools)

Coinbase 1K-5K Capital Markets

Coinbase is hiring a software engineer for its CDP Wallets & Onchain Tools team to build developer-focused APIs, SDKs, and documentation that help accelerate crypto application development onchain.

Canada Full-time Mid Level Software Engineer
$154k-$154k
Android Encryption Flutter GitHub Go iOS Microservices OpenAPI React React Native Solana TypeScript Unity
1 day, 6 hours ago
Veracyte

Software Engineer II

Veracyte 251-1K Pharmaceuticals

Veracyte is hiring a cloud engineering and application development professional for its Bioinformatics & Data Science Development team to build scalable cloud-native applications that support cancer diagnostics products and productionize research workflows.

United States Full-time Senior Software Engineer
$132k-$189k
Agile AWS AWS CDK CloudFormation Docker EC2 Kubernetes Machine Learning Microservices Node.js Python React REST API Scrum SQL Terraform Vue.js
1 day, 6 hours ago
Cribl

Staff Software Engineer, C021 Security

Cribl 251-1K IT Services

Cribl is hiring a Staff Engineer for its C021 new product initiative to help design and build an emerging data platform that processes large volumes of streaming data in a fully remote, remote-first environment.

United States Full-time Lead Software Engineer
$170k-$280k
Apache Spark AWS Azure Docker Druid Flink GCP JavaScript Kafka Kubernetes Linux LLM Node.js
1 day, 6 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers