RemoteLeaf Logo
Log in Sign up

SDET - AI

1 month ago
India
Full-time
Senior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Feature Engineering Generative AI MySQL Playwright PostgreSQL Python Selenium SQL
Apply Now
Resil

Resil

Resil specializes in providing AI-powered supply chain risk management solutions that enable organizations to detect threats in real time and take proactive measures to enhance supply chain resiliency.

Internet Software & Services
251-1K
Founded 2010
View All Jobs 3

Description

  • Develop and implement QA strategies for AI-powered applications with emphasis on accuracy, bias, fairness, robustness, and performance.
  • Design and execute automated and manual test cases to validate AI agents, LLM models, APIs, and data pipelines, ensuring data integrity and correct data models.
  • Assess AI models using quality metrics such as precision/recall and hallucination detection, and monitor for model drift and adversarial vulnerabilities.
  • Test for bias, fairness, explainability (XAI), and ethical considerations in model outputs and model-generated responses.
  • Validate prompt engineering approaches, fine-tuning techniques, and model-generated responses for accuracy and ethical compliance.
  • Design, develop, and maintain automation scripts and frameworks for API and web testing using tools such as Selenium and Playwright.
  • Conduct scalability, latency, and performance testing for AI-driven services and related tooling.
  • Collaborate with data engineers to validate data pipelines, feature engineering processes, and model outputs across the ML lifecycle.
  • Identify, document, track bugs, and perform detailed regression testing while integrating automation best practices into the development lifecycle.

Requirements

  • Proven expertise testing AI models, LLMs, and Generative AI applications, including hands-on use of AI evaluation metrics and testing methodologies.
  • Hands-on experience with AI testing tools such as Arize, MAIHEM, LangTest and automated testing workflows (Playwright, Selenium).
  • Strong proficiency in Python for writing test scripts and automating model validation.
  • Experience with prompt engineering, fine-tuning techniques, and validating model-generated outputs for accuracy and ethical considerations.
  • Deep understanding of AI bias detection, adversarial testing, model explainability (XAI), robustness, and drift detection.
  • Strong SQL skills for validating data integrity and backend processes, particularly with PostgreSQL and MySQL.
  • Experience conducting scalability, latency, and performance testing for production services.
  • Strong analytical and problem-solving skills with keen attention to detail, and excellent communication and documentation abilities.
  • Ability to work remotely and collaborate effectively with cross-functional teams (engineering, data engineering, product).

Benefits

  • Fully remote work environment with opportunities to connect in person.
  • Full-stack benefits for health, wealth, and wellbeing.
  • Opportunities for technical growth, ownership, and influence in shaping impactful technology.
  • Work on high-impact AI systems trusted by global enterprises, within a mission-driven organization.
  • Organizational stability backed by Vista Equity Partners and support for applicants needing accommodations (contact HR).

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Prolific

AI Training Experts - Bogotá

Prolific 51-250 Professional Services

Prolific is hiring AI Training Experts to join its participant platform to perform paid tasks that train and evaluate cutting‑edge AI models by providing human annotations and performance judgments.

Anywhere Freelance Junior AI (Artificial Intelligence)
Up to $52k
1 month ago
Remote Talent Latam

Automations & AI Specialist | Remote | LATAM Only | 85142

Remote Talent Latam 51-250 Professional Services

Automations/AI Expert for a U.S.-focused digital marketing agency (hired via Remote Talent LATAM) to design and implement a highly automated operational environment that eliminates repetitive tasks and scales internal operations.

Mexico Argentina Brazil Chile Colombia Costa Rica Dominican Republic Ecuador El Salvador Guatemala Honduras Nicaragua Panama Paraguay Peru Uruguay Venezuela Contract Mid Level AI (Artificial Intelligence) Operations Specialist
$36k-$36k
ClickUp GPT HubSpot JSON LLM SEO
1 month ago
Veeam Software

Software Development Engineer in Test (SDET), Kasten 

Veeam Software 1K-5K Internet Software & Services

Software Development Engineer in Test at Veeam Kasten working on the infrastructure and test frameworks for the Kubernetes-focused Veeam Kasten data management platform to ensure high-quality, secure backup and recovery capabilities.

Poland Full-time Mid Level DevOps Engineer SDET (Software Development Engineer in Test)
AWS Bash CI/CD Docker Git Go Helm Kubernetes OpenShift Python Rancher Shell Scripting
1 month ago
Toloka

AI Trainer - Freelance Annotator (Portuguese)

Toloka 251-1K Internet Software & Services

Toloka is hiring remote freelance Annotators to evaluate and label text, image, and video data for Generative AI projects, helping improve AI systems by providing human judgments on content.

France Germany Spain Freelance Entry Level AI (Artificial Intelligence)
Up to $46k
Generative AI
1 month ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers