RemoteLeaf Logo
Log in Sign up

Senior Data Engineer

3 weeks ago
United States
Full-time
Lead
Data Engineer
Software Development
Apache Spark AWS Databricks Git GitLab Machine Learning Python SQL
Apply Now
540

540

540 is a government-trusted tech consulting firm that breaks barriers to innovation and delivers impactful solutions like start-ups.

Internet Software & Services
51-250
Founded 2013
View All Jobs 6

Description

  • Design, develop, and maintain scalable production-ready data pipelines and data products in Spark and Databricks.
  • Lead the integration and transformation of complex data from DoW and federal health systems into reusable data products.
  • Design scalable data ingestion, integration, and exchange approaches, including API-based services.
  • Establish reusable data engineering patterns, standards, and best practices across the platform.
  • Provide technical guidance on data architecture, pipeline design, data modeling, integration, and engineering practices.
  • Monitor, troubleshoot, and optimize production workflows for performance, reliability, and scalability.
  • Define and implement data validation, quality, and governance practices.
  • Troubleshoot complex data integration issues, identify root causes, and drive sustainable solutions.
  • Collaborate with engineers, architects, analysts, and stakeholders to translate data needs into technical solutions.
  • Mentor other engineers and help improve engineering tools, processes, and patterns.

Requirements

  • 10+ years of data engineering, software engineering, or related technical experience.
  • Extensive hands-on experience designing, building, and operating production data pipelines and data products.
  • Advanced proficiency with Python and SQL.
  • Strong experience with Apache Spark and distributed data processing.
  • Experience with Databricks or similar modern data platforms.
  • Experience designing and maintaining ETL/ELT processes for complex, large-scale datasets.
  • Experience integrating data across disparate systems and building or consuming API-based integrations.
  • Strong understanding of data modeling, data architecture, data quality, and data governance principles.
  • Experience troubleshooting and optimizing complex production data pipelines for performance, reliability, and scalability.
  • Experience working with Git-based development workflows and terminal/command-line environments.
  • Demonstrated experience providing technical guidance, mentoring engineers, and influencing engineering practices.
  • Strong client and stakeholder communication skills, with the ability to explain technical concepts to technical and non-technical audiences.
  • Ability to work independently through ambiguity, identify technical risks, and drive issues to resolution.
  • Experience identifying security, privacy, and compliance issues and working with relevant stakeholders.
  • U.S. Citizenship and an active DoW Secret clearance or higher are required.
  • Bachelor’s degree in Computer Science, Engineering, Data Science, or a related technical field is preferred.
  • AWS cloud experience is preferred.
  • Experience with very large datasets, including billions of records, is preferred.
  • Experience with Palantir Foundry is preferred.
  • GitLab experience is preferred.
  • Experience with Advana or similar DoW data environments is preferred.
  • Experience working with federal health, financial, or other regulated and sensitive data is preferred.
  • Experience using AI/ML to accelerate work, including automation and debugging, is preferred.

Benefits

  • Flexible PTO plus all Federal holidays off.
  • Health, dental, and vision insurance plans.
  • Flexible Spending Account (FSA).
  • 401(k) with employer match.
  • Company-sponsored life insurance plus short- and long-term disability coverage.
  • Professional development support for training, certifications, and conferences.
  • Paid cloud developer accounts.
  • Referral bonuses.
  • HQ office perks including parking or metro reimbursement, nitro coffee, and lunches.
  • Access to Washington Capitals and Nationals tickets.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Inca Digital

Mid/Senior-level GoLang Data Engineer

Inca Digital 11-50 Internet Software & Services

Inca Digital is hiring a backend engineer to build and operate data infrastructure that transforms blockchain, financial, social, and web data into intelligence products for digital-asset clients.

United States Full-time Mid Level Backend Engineer Data Engineer
AWS CI/CD Docker Git Go gRPC Kafka Kubernetes Microservices MongoDB PostgreSQL RabbitMQ Redis REST API SQL Terraform
21 hours, 26 minutes ago
Xebia

Senior AI/Data Engineer

Xebia 1K-5K Internet Software & Services

Xebia is seeking a senior data or analytics engineer to build an AI-powered Analytics Agent in Devin, integrating dbt, Databricks, GitHub, and related data-platform tools for client engineering teams.

Bulgaria Poland Romania Senior AI (Artificial Intelligence) Data Engineer
AWS Azure CI/CD Databricks dbt GCP Git GitHub MySQL Snowflake SQL
21 hours, 56 minutes ago
Trinetix

Databricks Developer

Trinetix 251-1K Internet Software & Services

Trinetix is seeking a Databricks Developer to build and maintain scalable data ingestion, transformation, and analytics pipelines for enterprise clients across its global delivery organization.

Poland Portugal Romania Bulgaria Spain Senior Data Engineer
Apache Spark CI/CD Databricks Pandas Python
1 day, 22 hours ago
8th Light

Principal Data Engineer (Remote)

8th Light 51-250 Internet Software & Services

8th Light is seeking a Principal Engineer to lead client-facing data platform architecture and delivery, turning complex software and data challenges into scalable solutions.

United States Full-time Lead Data Engineer
$152k-$221k
Apache Spark AWS CI/CD Databricks Kafka MLOps Python SQL
2 days, 21 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers