RemoteLeaf Logo
Log in Sign up

数据开发工程师（实时计算方向）

3 weeks, 3 days ago
Asia, Oceania
Full-time
Senior
Data Engineer
Software Development
Apache Spark ClickHouse Elasticsearch Flink Hadoop Hive Java Kafka MySQL PostgreSQL Python Scala SQL
Apply Now
Gate.com

Gate.com

Gate.com offers premium Managed WordPress hosting with top-notch security, management tools, and 24/7 support for small and medium-sized businesses, ensuring an enterprise-grade experience at an affordable price.

IT Services
51-250
Founded 1996
View All Jobs 23

Description

  • 分析、设计并交付基于 Flink、Kafka 的实时数据处理链路，为实时分析、监控、指标和业务决策提供数据支持。
  • 参与实时数仓架构、数据分层、指标口径、维度模型和数据服务的设计与落地。
  • 优化实时任务，解决数据延迟、反压、Checkpoint 异常、状态膨胀、资源利用率不合理和数据不一致等问题。
  • 参与任务依赖梳理、元数据管理、数据质量监控、告警机制、开发规范和问题排查体系建设。
  • 维护大数据平台及计算引擎，协同上下游团队保障数据链路的稳定性、准确性和时效性。
  • 通过技术分享、代码评审和方案评审沉淀实时开发最佳实践，提升团队研发能力。

Requirements

  • 本科及以上学历，计算机、软件工程、信息技术或相关专业，具备 5 年及以上数据开发或大数据研发经验；优秀者可适当放宽。
  • 具备完整的数据仓库项目经验，熟悉实时数仓架构、数据分层和数据模型设计。
  • 熟练掌握 Flink，具备 Flink SQL、DataStream API 实际项目经验，理解 State、Checkpoint、Watermark、窗口、反压和容错机制。
  • 具备实时任务开发与优化经验，能够解决任务延迟、资源消耗、状态异常、数据倾斜和数据一致性问题。
  • 熟悉 Kafka 及实时数据链路，理解数据采集、传输、计算、存储和服务流程。
  • 熟悉 Hadoop、Hive、Spark，并具备离线数据开发能力。
  • 熟练掌握 SQL，能够进行复杂数据加工和性能优化；至少掌握 Python、Java 或 Scala 之一。
  • 熟悉 MySQL、PostgreSQL、Elasticsearch 等数据库或存储系统。
  • 具备良好的业务理解、沟通协作和书面表达能力，能够主动推动问题解决和项目落地。
  • 具备大规模 Flink 任务调优、实时数仓治理、Hologres/ClickHouse/Doris 或实时指标、监控、风控、推荐场景经验者优先。

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

RevStar

Databricks Engineer

RevStar 51-250 Internet Software & Services

RevStar is seeking a remote, US-based Databricks Engineer to build and optimize cloud-agnostic Lakehouse data pipelines and MLOps frameworks for enterprise AI projects across AWS, Azure, and GCP.

United States Mexico Argentina Brazil Chile Colombia Venezuela Contract Mid Level Data Engineer
Agile Apache Airflow Apache Spark AWS Azure CI/CD Databricks Feature Engineering HIPAA Kafka MLflow MLOps Prefect Python SQL Terraform
4 hours, 57 minutes ago
phData

Senior Data Engineer

phData 251-1K IT Services

Data Engineer at phData, a remote-first global data and AI consultancy, designing and delivering secure, scalable cloud data solutions for enterprise clients.

India Full-time Mid Level Data Engineer
Apache Airflow Apache Spark AWS Azure Cassandra Databricks dbt Elasticsearch GCP Hadoop HDFS Kafka Luigi Python REST API Scala Snowflake Solr SQL System Design
1 day, 4 hours ago
Abacus Insights

Forward Deployed Data Engineer

Abacus Insights 51-250 Insurance

Abacus Insights is hiring a Forward Deployed Data Engineer to work alongside health plan clients, building governed data foundations, pipelines, and integrations that turn fragmented payer data into analytics-ready products.

United States Full-time Senior Data Engineer
Apache Spark Databricks Snowflake SQL
1 day, 4 hours ago
Azumo

Data Engineer

Azumo 51-250 Internet Software & Services

Azumo is hiring a fully remote Latin America-based Data Engineer to build and operate production ingestion, transformation, storage, warehouse, and retrieval infrastructure that supports client AI systems.

Mexico Argentina Brazil Chile Colombia Peru Full-time Senior Data Engineer
Apache Airflow Apache Spark AWS Azure CI/CD Dagster Databricks dbt Docker Flink GCP Git GitHub Actions HIPAA Jupyter Kafka Prefect Python Snowflake SQL Terraform
1 day, 4 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers