RemoteLeaf Logo
Log in Sign up

Senior AI Platform Engineer - HexCore & Eval Systems - OPS00071

1 week ago
Argentina, Brazil, Colombia
Full-time
Senior
AI Engineer
Software Development
AWS CI/CD EC2 FastAPI GitHub Actions Notion PostgreSQL Pytest Python Twilio YAML
Apply Now
Dev.Pro

Dev.Pro

Dev.Pro is a globally distributed software development partner, specializing in custom outsourced software development for innovative technology companies to scale their businesses efficiently.

Internet Software & Services
251-1K
Founded 2011
View All Jobs 29

Description

  • Own and evolve the core Python platform repository that implements the modular agent architecture across orchestration, tools, state, retrieval, configuration, and extensibility layers.
  • Design and maintain customer configuration schemas with versioning, lineage tracking, and component provenance fields.
  • Implement backward-compatible schema extensions and enforce schema validation at all node inputs and outputs.
  • Build and maintain multi-tenant isolation across customer configuration, persistent state, conversation logs, eval datasets, and RAG pipelines.
  • Design config-driven deployment parameterization so new customers can be onboarded without code changes.
  • Own the end-to-end conversation logging system, including unified schema, capture format, metadata persistence to PostgreSQL and S3, and data tagging.
  • Maintain and extend knowledge base ingestion pipelines, including scraping, embedding, vector indexing, and retrieval validation.
  • Own the evaluation suite end to end, including scenario design, dataset curation, automated scoring, regression CI gates, and production readiness thresholds.
  • Build and maintain onboarding automation scripts that deploy a new customer in under 30 minutes.
  • Ensure platform APIs meet latency targets and maintain structured logging, CI/CD validation, and observability for incident diagnosis.

Requirements

  • Bachelor’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
  • 4+ years of experience building and owning production-grade backend systems in Python.
  • Proficient or advanced use of agentic coding workflows in tools like Cursor AI or Claude Code.
  • Proven experience owning a core platform or shared infrastructure layer used by multiple teams or customers.
  • Hands-on experience with multi-tenant system design, including schema isolation, config-driven parameterization, and deployment automation.
  • Experience building evaluation harnesses for LLM-based systems with quantitative metrics.
  • Strong proficiency in Python, including async I/O, FastAPI, Pydantic, pytest, type hints, and dataclasses.
  • Experience with LangGraph, including state machines, conditional edges, node composition, and shared state management.
  • Experience with PostgreSQL and pgvector for relational schema design, state persistence, and multi-tenant data isolation.
  • Experience with RAG pipelines, vector databases such as Pinecone or equivalent, embedding pipelines, and retrieval evaluation.
  • Experience with eval and tracing frameworks, automated scoring pipelines, and CI/CD gates such as GitHub Actions.
  • Experience with AWS services such as EC2, S3, RDS, and IAM.
  • Experience with YAML and config-driven deployment, including customer configuration templating and parameterized onboarding scripts.
  • Strong systems thinking, schema discipline, autonomy, written communication, and ability to work across platform, data, eval, and ops.
  • Nice to have: experience with IP-aware architecture decisions or software patent documentation.
  • Nice to have: familiarity with voice agent systems such as Twilio, PSTN, and LiveKit in latency-constrained deployments.
  • Nice to have: experience with multi-model evaluation across providers such as OpenAI, Anthropic, and Mistral.
  • Nice to have: prior work in self-storage, property management, or other regulated verticals with privacy and auditability requirements.
  • Nice to have: experience contributing to modular, clean-architecture codebases across multiple bounded contexts.
  • Nice to have: prior experience in a fast-growing startup where you owned infrastructure other engineers depended on daily.

Benefits

  • 30 paid days off each year for vacation, holidays, or personal time.
  • 5 paid sick days, up to 60 days of medical leave, and 6 paid days off for family events.
  • Partially covered health insurance after probation.
  • Wellness bonus for gym memberships, sports nutrition, and similar needs.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

R

Senior AI Engineer

Rockstar 1-10 Professional Services

Rockstar is hiring a Senior AI Engineer / Senior Machine Learning Engineer to build and operationalize production-grade AI systems for a data intelligence platform, taking AI features from prototype to reliable deployment.

United States Full-time Senior AI Engineer
CI/CD Deep Learning Docker Generative AI Kubernetes LLM NLP Python PyTorch Scikit-learn TensorFlow
31 minutes ago
Leadtech

AI Automation Engineer

Leadtech 251-1K IT Services

Leadtech is hiring an AI Automation Engineer to design and operate AI-driven customer support workflows, API integrations, and voice/agent automation systems that improve operational efficiency and customer experience at scale.

Spain Full-time Junior AI Engineer Technical Support Engineer
Generative AI GPT HTTP JavaScript JSON Postman Python REST API
1 hour, 22 minutes ago
Remote

Senior Backend Engineer (Elixir)

Remote 251-1K Professional Services

Remote is hiring a full-time engineer to help build tools, APIs, and integrations for its global HR and Payroll products in a fully remote, async environment.

Remote
Full-time Senior Backend Engineer
$53k-$120k
Angular AWS CI/CD Docker Elixir GitHub GitLab Jenkins Kubernetes Next.js Phoenix PostgreSQL React Vue.js
1 hour, 27 minutes ago
Grafana

Senior Backend Engineer - Grafana Search & Storage | Spain | Remote

Grafana 1K-5K IT Services

Grafana Labs is hiring a Senior Backend Engineer to help build and operate its next-generation search and storage platform for Grafana Cloud and related products.

United Kingdom Spain Sweden Germany Ireland Full-time Senior Backend Engineer
$89k-$111k
Go Grafana Kubernetes Prometheus
1 hour, 56 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers