RemoteLeaf Logo
Log in Sign up

Peach Pilot —Sr QA Engineer (AI Systems & Platform) Remote — Latin America

1 hour, 52 minutes ago
Latin America, Argentina
Contract
Senior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Azure CI/CD Docker FastAPI Firebase GCP GitHub Actions Neo4j Next.js Node.js Plotly PostgreSQL Python React Redis REST API Tailwind CSS TypeScript
Apply Now

Hive Financial Systems

Hive Financial Systems operates in financial services, with a focus on the sub-prime consumer lending vertical. It says its core advantages come from extensive industry experience and cutting-edge technology for marketing, underwriting, scoring, and loan management, and it positions itself as building alternative credit and consumer lending systems using data-driven underwriting and automation.

Financial Services
11-50
Founded 2017
View All Jobs 2

Description

  • Partner with full-stack and backend engineers to understand shipped features, write tests, and identify gaps early.
  • Reproduce, triage, and document bugs with enough detail for engineers to act without follow-up.
  • Contribute to and evolve automated test suites across unit, integration, and end-to-end coverage.
  • Build and run evaluation pipelines for non-deterministic LLM outputs, prompt regressions, model drift, and quality scoring.
  • Test agent orchestration behavior, including governance audit trails, human-in-the-loop overrides, and cross-agent handoffs.
  • Validate retrieval quality and failure modes against the Company Brain using real enterprise data scenarios.
  • Test the Nango-based integration layer and file ingestion pipeline for edge cases, encryption, formatting, and audit continuity.
  • Verify streaming response handling, latency thresholds, and graceful degradation when models are unavailable or slow.
  • Test multi-model routing logic to ensure correct cost-optimized allocation and faithful outputs across providers.
  • Evaluate trust-layer UX flows, uncertainty states, and governance interfaces for non-technical enterprise users.

Requirements

  • 5+ years of QA engineering experience, with meaningful time spent writing test code.
  • Hands-on experience testing LLM-powered applications, including prompt sensitivity and output variance.
  • Strong Python test automation skills; Python is the primary tool.
  • Experience contributing to CI/CD-integrated test suites.
  • Comfort testing complex API chains, async/streaming responses, and multi-service workflows.
  • Collaborative, self-directed working style with strong partnership skills.
  • Strong written and verbal English communication skills.
  • Available during US Eastern business hours with at least 5 hours of daily overlap.
  • Experience with LLM evaluation frameworks such as LangSmith, PromptFlow, or custom eval pipelines (preferred).
  • Experience testing agent frameworks such as LangChain, CrewAI, or similar (preferred).
  • Experience with graph databases like Memgraph and Neo4j, or vector stores like Qdrant (preferred).
  • Background in enterprise software or regulated industries with audit trail integrity requirements (preferred).
  • Insurance industry background is a plus.

Benefits

  • Fully remote contract role based in Latin America.
  • Competitive contractor rate commensurate with experience.
  • Paid monthly via Deel in USD.
  • Opportunity to work with a funded early-stage AI startup on a live production platform.
  • Access to production data, live workflows, and real compliance requirements from day one.
  • Direct impact on a first client engagement that is already scoped and funded.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

BrainRocket

VIP Quality Assurance Manager - French Speaking

BrainRocket 251-1K Internet Software & Services

BrainRocket is hiring a VIP Quality Assurance (QA) Manager to oversee the quality of VIP programs and player services within its iGaming operations.

Poland Full-time Mid Level QA Engineer
22 minutes ago
Twilio

Staff Engineer, AI Security

Twilio 5K-10K Diversified Telecommunication Services

Twilio is hiring a remote-based Staff Engineer, AI Security in Ireland to lead security for AI and machine learning systems across the app security team and help shape a secure-by-default AI lifecycle.

Ireland Full-time Lead AI (Artificial Intelligence) Application Security Engineer
CI/CD Go LLM Machine Learning Python Twilio
22 minutes ago
Databricks

Sr. Manager, AI FDE

Databricks 1K-5K IT Services

Databricks is seeking a Senior Manager, AI FDE to lead its customer-facing AI professional services team in the Global Delivery Center, guiding enterprise AI transformations and scalable GenAI solution delivery.

India Full-time Lead AI (Artificial Intelligence) Program Manager
Apache Spark AWS Azure Databricks GCP Generative AI Google Tag Manager Machine Learning MLflow
52 minutes ago
Xometry

Quality Assurance Engineer, Manufacturing

Xometry 251-1K Industrial Conglomerates

Xometry is hiring a remote Quality Assurance Engineer to support customized part manufacturing by strengthening supplier quality, resolving nonconformances, and maintaining AS9100-compliant quality systems.

United States Full-time Mid Level QA Engineer
Asana Statistics Tableau
52 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers