RemoteLeaf Logo
Log in Sign up

Senior/Staff/Principal SWE- Observability Engineering

3 months ago
United States
Full-time
Lead
Technical Lead
Software Development
Datadog DNS Elasticsearch Go Honeycomb JSON LLM New Relic OpenTelemetry Prometheus Rust Splunk TLS
Apply Now
Appgate

Appgate

Appgate is the secure access company empowering secure connections with Zero Trust principles for people, devices, and systems.

Professional Services
251-1K
Founded 2020
View All Jobs 11

Description

  • Architect the observability platform, including telemetry schema, correlation model, transport, and sampling strategies across client devices, controllers, and gateways.
  • Build telemetry SDKs and the LogForwarder pipeline using OpenTelemetry.
  • Implement enrichment, redaction, batching, and tail-sampling for high-volume telemetry processing.
  • Design and support a high-cardinality ingestion and routing layer that can scale to 100M+ events per day.
  • Develop end-to-end distributed tracing across login and session establishment flows.
  • Enable on-demand packet capture workflows that are observable through logs and traces.
  • Define and validate integration standards for OTLP, Prometheus, and JSON logs.
  • Test and validate the observability system at customer-scale environments with large deployments and high concurrency.
  • Collaborate with product, R&D, and customers to shape requirements and deliver outcomes.
  • Mentor engineers, establish engineering patterns and review practices, and help grow the observability discipline at AppGate.

Requirements

  • 8+ years of engineering experience, including at least 4 years in observability, telemetry, or large-scale data infrastructure.
  • Deep OpenTelemetry expertise, including OTLP, the OTel Collector, semantic conventions, context propagation, and head/tail sampling.
  • Experience designing or significantly contributing to a distributed tracing system in production.
  • Hands-on experience with high-throughput systems ingesting 100M+ events per day.
  • Strong systems programming experience with production Go and/or Rust preferred.
  • Comfort working across the stack from agent code to backend services.
  • Knowledge of networking and security fundamentals, including TLS, DNS, TCP, and identity protocols.
  • Prior ZTNA, SASE, or SD-WAN experience is a strong plus.
  • Pragmatic, opinionated, and impact-driven mindset.
  • Experience with observability platforms such as Datadog, Splunk, Elastic, Honeycomb, New Relic, or Grafana Labs, or equivalent.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

2K

Lead Lighting Artist

2K 10K-50K Internet Software & Services

Cloud Chamber is hiring a Lead Lighting Artist to lead real-time lighting for BioShock, shaping the game’s visual atmosphere and supporting immersive storytelling on a AAA development team.

United States Full-time Lead Game Developer Technical Lead
$150k-$223k
6 hours, 12 minutes ago
Apply Digital

Principal Agentic Engineer - Talent Pipeline

Apply Digital 251-1K Professional Services

APPLY is seeking a Principal Agentic Engineer to lead full-stack architecture and AI-native delivery for enterprise-scale client work in a hybrid/remote-friendly, Canada-based role.

Canada Full-time Lead AI Engineer Technical Lead
$126k-$163k
Agile AWS Azure CI/CD Contentful Docker GCP Git GraphQL JavaScript Kubernetes LLM Microservices Next.js Node.js React REST API Serverless Terraform TypeScript Vertex AI
6 hours, 27 minutes ago
May Mobility

Tech Lead, Simulation Engine

May Mobility 251-1K Transportation

May Mobility is seeking a Lead Simulation Engineer to architect and scale the autonomous vehicle simulation platform that supports vehicle dynamics, physics, rendering, sensor simulation, and evaluation for its AV stack.

United States Full-time Lead Technical Lead
$189k-$272k
C++ Machine Learning Python
1 day, 5 hours ago
Kargo

Lead Data Engineer - Identity

Kargo 251-1K Media

Kargo is hiring a senior data engineering leader to own its identity graph, onboarding pipelines, and audience data layer for adtech products across web, CTV, mobile, and in-app inventory.

Europe Full-time Lead Data Engineer Technical Lead
Apache Airflow Apache Spark Argo CD AWS ClickHouse GitHub Actions Grafana Kafka Kubernetes Prometheus Python Snowflake SQL
1 day, 6 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers