RemoteLeaf Logo
Log in Sign up

Senior AI Platform Engineer - HexCore & Eval Systems - OPS00071

4 weeks ago
Argentina, Brazil, Colombia
Full-time
Senior
AI Engineer
Software Development
AWS CI/CD EC2 FastAPI GitHub Actions Notion PostgreSQL Pytest Python Twilio YAML
Apply Now
Dev.Pro

Dev.Pro

Dev.Pro is a globally distributed software development partner, specializing in custom outsourced software development for innovative technology companies to scale their businesses efficiently.

Internet Software & Services
251-1K
Founded 2011
View All Jobs 32

Description

  • Own and evolve the core Python platform repository that implements the modular agent architecture across orchestration, tools, state, retrieval, configuration, and extensibility layers.
  • Design and maintain customer configuration schemas with versioning, lineage tracking, and component provenance fields.
  • Implement backward-compatible schema extensions and enforce schema validation at all node inputs and outputs.
  • Build and maintain multi-tenant isolation across customer configuration, persistent state, conversation logs, eval datasets, and RAG pipelines.
  • Design config-driven deployment parameterization so new customers can be onboarded without code changes.
  • Own the end-to-end conversation logging system, including unified schema, capture format, metadata persistence to PostgreSQL and S3, and data tagging.
  • Maintain and extend knowledge base ingestion pipelines, including scraping, embedding, vector indexing, and retrieval validation.
  • Own the evaluation suite end to end, including scenario design, dataset curation, automated scoring, regression CI gates, and production readiness thresholds.
  • Build and maintain onboarding automation scripts that deploy a new customer in under 30 minutes.
  • Ensure platform APIs meet latency targets and maintain structured logging, CI/CD validation, and observability for incident diagnosis.

Requirements

  • Bachelor’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
  • 4+ years of experience building and owning production-grade backend systems in Python.
  • Proficient or advanced use of agentic coding workflows in tools like Cursor AI or Claude Code.
  • Proven experience owning a core platform or shared infrastructure layer used by multiple teams or customers.
  • Hands-on experience with multi-tenant system design, including schema isolation, config-driven parameterization, and deployment automation.
  • Experience building evaluation harnesses for LLM-based systems with quantitative metrics.
  • Strong proficiency in Python, including async I/O, FastAPI, Pydantic, pytest, type hints, and dataclasses.
  • Experience with LangGraph, including state machines, conditional edges, node composition, and shared state management.
  • Experience with PostgreSQL and pgvector for relational schema design, state persistence, and multi-tenant data isolation.
  • Experience with RAG pipelines, vector databases such as Pinecone or equivalent, embedding pipelines, and retrieval evaluation.
  • Experience with eval and tracing frameworks, automated scoring pipelines, and CI/CD gates such as GitHub Actions.
  • Experience with AWS services such as EC2, S3, RDS, and IAM.
  • Experience with YAML and config-driven deployment, including customer configuration templating and parameterized onboarding scripts.
  • Strong systems thinking, schema discipline, autonomy, written communication, and ability to work across platform, data, eval, and ops.
  • Nice to have: experience with IP-aware architecture decisions or software patent documentation.
  • Nice to have: familiarity with voice agent systems such as Twilio, PSTN, and LiveKit in latency-constrained deployments.
  • Nice to have: experience with multi-model evaluation across providers such as OpenAI, Anthropic, and Mistral.
  • Nice to have: prior work in self-storage, property management, or other regulated verticals with privacy and auditability requirements.
  • Nice to have: experience contributing to modular, clean-architecture codebases across multiple bounded contexts.
  • Nice to have: prior experience in a fast-growing startup where you owned infrastructure other engineers depended on daily.

Benefits

  • 30 paid days off each year for vacation, holidays, or personal time.
  • 5 paid sick days, up to 60 days of medical leave, and 6 paid days off for family events.
  • Partially covered health insurance after probation.
  • Wellness bonus for gym memberships, sports nutrition, and similar needs.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Elastic

Senior Java Engineer - Distributed Systems - Elasticsearch

Elastic 1K-5K Internet Software & Services

Elastic is hiring a Senior Software Engineer for the Elasticsearch Distributed Systems team to improve cluster-scale indexing, coordination, and resilience across a highly distributed search platform.

Spain Full-time Senior Backend Engineer
$72k-$114k
Elasticsearch Java Lucene
2 hours, 37 minutes ago
Elastic

Senior Java Engineer - Distributed Systems - Elasticsearch

Elastic 1K-5K Internet Software & Services

Elastic is hiring a Senior Software Engineer for its Elasticsearch Distributed Systems team to improve the scale, performance, and resilience of clustered search infrastructure.

Portugal Full-time Senior Backend Engineer
$65k-$103k
Elasticsearch Java Lucene
2 hours, 37 minutes ago
Elastic

Senior Java Engineer - Distributed Systems - Elasticsearch

Elastic 1K-5K Internet Software & Services

Elastic is hiring a Senior Software Engineer for its Elasticsearch Distributed Systems team to help improve the scale, performance, and resilience of the cluster systems that handle indexing, allocation, replication, and node coordination.

Hungary Full-time Senior Backend Engineer
$20k-$32k
Elasticsearch Java Lucene
2 hours, 37 minutes ago
New Relic

Senior Software Engineer - Fullstack (Backend Focused)

New Relic 1K-5K Internet Software & Services

New Relic is hiring a backend engineer to help build a new observability experience and next-generation platform services for distributed systems in an AI-first environment.

India Full-time Senior Backend Engineer
Agile CI/CD Docker Git GraphQL Java Kafka Kubernetes Microservices React REST API Spring Boot TypeScript
2 hours, 37 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers