Data Engineer

2 weeks, 5 days ago
Full-time
Mid Level
Software Development
Egen.ai

Egen.ai

Egen.ai specializes in providing technology services that leverage cloud computing, data analytics, and artificial intelligence to enhance document intelligence and drive productivity and growth for its clients.

IT Services
Founded 2000

Description

  • Design, develop, and deploy large-scale batch and streaming data processing pipelines.
  • Build data pipelines using technologies such as Dataflow, Apache Beam, Spark, Akka, and Pub/Sub.
  • Work with stakeholders to understand business problems and translate them into technical solutions.
  • Design appropriate data schemas based on domain understanding.
  • Manage data lineage and help ensure data security using appropriate tools and methodologies.
  • Collaborate with data scientists, architects, and other stakeholders to align technical work with business strategy.
  • Monitor, refine, and report on the performance of data management systems.
  • Mentor junior data engineers and review their work.
  • Support data storage and processing needs across technologies such as Bigtable/HBase, BigQuery, Spanner, and Cloud SQL/Postgres.

Requirements

  • 2-5 years of experience in data engineering, with a focus on designing and developing data pipelines.
  • Proven expertise with Dataflow, Apache Beam, Spark, Akka, and Pub/Sub.
  • Experience with data storage technologies such as Bigtable/HBase, BigQuery, Spanner, and Cloud SQL/Postgres.
  • Ability to design data schemas based on an understanding of the domain problem.
  • Experience with data security and data lineage methodologies and tools is preferred.
  • Familiarity with agile development methodologies.
  • Exceptional communication skills and the ability to explain complex technical concepts clearly.
  • Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent work experience.
  • Nice to have: experience with data migration projects.
  • Nice to have: knowledge of dbt, Airflow, or similar orchestration tools, multi-cloud environments, and data modeling or analytics use cases.

Benefits

  • Competitive salary of $76,500 - $105,300 per year.
  • Comprehensive health insurance.
  • Paid leave, including vacation/PTO.
  • Paid holidays.
  • Sick leave.
  • Parental leave.
  • Bereavement leave.
  • 401(k) employer match.
  • Employee referral bonuses.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Python Data Engineer

Flatgigs Professional Services

FlatGigs is hiring a Python Data Engineer to build and maintain reliable, scalable data pipelines and ETL workflows that support analytics and data-driven decision-making.

Apache Airflow AWS Azure Docker GCP Kubernetes MongoDB NumPy Pandas PostgreSQL Python SQL
3 hours, 17 minutes ago

Senior Software Engineer - Data Integration & JVM Ecosystem

ClickHouse 51-250 IT Services

ClickHouse is hiring a Senior Software Engineer to own JVM-based data connectors and integrations that connect its high-performance analytics platform to the broader data ecosystem.

Apache Airflow Apache Spark ClickHouse dbt Grafana HTTP Java Kafka Metabase Pandas Power BI Python SQL Tableau TCP/IP
4 hours, 35 minutes ago

SAP BW Lead

Lingaro 5K-10K IT Services

SAP BW Lead for Poland’s CC Data Engineering & Management team at SAP, responsible for leading SAP BW-related data engineering work in a full-time remote role.

5 hours, 53 minutes ago

Lead Data / Rating Engineer

Alex Staff Agency 11-50 Professional Services

Rating Production Lead at a product company building scalable rating systems to improve creator monetization and information navigation, with the goal of turning existing algorithms and data methods into a production-ready system.

Machine Learning Python SQL Statistics
7 hours, 12 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers