数据开发工程师(实时计算方向)

21 hours, 55 minutes ago
Full-time
Senior
Software Development
Gate.com

Gate.com

Gate.com offers premium Managed WordPress hosting with top-notch security, management tools, and 24/7 support for small and medium-sized businesses, ensuring an enterprise-grade experience at an affordable price.

IT Services
51-200
Founded 1996

Description

  • 分析实时数据需求,设计方案并基于 Flink、Kafka 等技术交付实时数据处理链路。
  • 参与实时数据仓库建设、数据分层、指标口径、维度模型和数据服务的设计与落地。
  • 优化实时任务的延迟、反压、Checkpoint 异常、状态膨胀、资源利用率和数据一致性问题。
  • 推进任务依赖梳理、元数据管理、数据质量监控、告警机制、开发规范和问题排查体系建设。
  • 参与大数据平台及计算引擎的日常维护和故障处理,保障数据链路稳定性、准确性和时效性。
  • 协同上下游团队解决数据链路问题并推动项目落地。
  • 通过技术分享、代码评审和方案评审沉淀实时开发最佳实践,提升团队研发能力。
  • 参与离线数据仓库建设与开发。

Requirements

  • 本科及以上学历,计算机、软件工程、信息技术或相关专业;具备 5 年及以上数据开发或大数据研发经验,优秀者可适当放宽。
  • 具备完整的数据仓库项目经验,熟悉实时数仓架构、数据分层和数据模型设计。
  • 熟练掌握 Flink,具有 Flink SQL 和 DataStream API 项目经验,理解 State、Checkpoint、Watermark、窗口、反压和容错机制。
  • 具备实时任务开发与优化经验,能够独立排查延迟、资源消耗、状态异常、数据倾斜和数据一致性问题。
  • 熟悉 Kafka 及实时数据链路,理解数据采集、传输、计算、存储和服务全流程。
  • 熟悉 Hadoop、Hive、Spark 等大数据技术,具备离线数据开发能力。
  • 熟练掌握 SQL,能够进行复杂数据加工和性能优化;至少掌握 Python、Java 或 Scala 之一。
  • 熟悉 MySQL、PostgreSQL、Elasticsearch 等数据库或存储系统。
  • 具备良好的业务理解、沟通协作和书面表达能力,能够主动推动问题解决与项目落地。
  • 具有大规模 Flink 调优或集群治理、实时数仓或链路治理、Hologres/ClickHouse/Doris,以及实时指标、监控、风控或推荐场景经验者优先。

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Head of Fund Analytics & Automation — Credit Fund

Xsolla 201-500 Internet Software & Services

Xsolla is seeking a Head of Fund Analytics & Automation to build and operate the data, automation, underwriting, and reporting platform for its emerging Credit Fund.

Apache Airflow PostgreSQL Power BI Python React SQL Tableau TypeScript
21 hours, 25 minutes ago

Data Engineer, Marketing Science

Power Digital is seeking a Marketing Data Engineer to build reliable, model-ready data foundations for its Media Mix Modeling and incrementality measurement products.

CI/CD CRM dbt Git Google Ads Python Shopify Snowflake SQL TikTok
21 hours, 40 minutes ago

Data Engineer, Marketing Science

Power Digital is hiring a Marketing Data Engineer to build and maintain trusted Snowflake data foundations for Media Mix Modeling and incrementality measurement products.

CI/CD dbt Facebook Ads Git Google Ads Python Shopify Snowflake SQL TikTok
21 hours, 40 minutes ago

Data Engineer, Marketing Science

Power Digital is seeking a Marketing Data Engineer to build reliable, model-ready data foundations for Media Mix Modeling and incrementality testing across its client portfolio.

CI/CD dbt Facebook Ads Git Google Ads Python Shopify Snowflake SQL TikTok
21 hours, 40 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers