RemoteLeaf Logo
Log in Sign up

AI Evaluator - POLISH

4 weeks ago
Poland
Contract
Junior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Apply Now
Gramian Consultancy Group

Gramian Consultancy Group

Professional Services
View All Jobs 20

Description

  • Design and run short multi-turn conversations to test AI personalization behavior.
  • Create prompts based on realistic personal scenarios and lived context.
  • Review AI responses to assess whether personalization is applied correctly.
  • Check grounding quality to ensure the model does not invent unsupported claims about the user.
  • Evaluate whether personal signals are used naturally and appropriately in responses.
  • Compare two responses side by side and determine which is more helpful, natural, and relevant.
  • Write clear, structured rationales that explain rankings and reference specific conversation turns.
  • Verify debug information to confirm the correct data sources were used.
  • Maintain strict workflow hygiene, including deleting evaluation conversations when required.

Requirements

  • Strong Polish proficiency in reading and writing; Polish is the primary evaluation language.
  • BS/BA degree or equivalent experience in a relevant analytical field such as Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or a related discipline.
  • Strong analytical thinking and ability to assess nuanced AI outputs.
  • Excellent written communication skills with the ability to produce structured evaluation notes.
  • High attention to detail when comparing similar responses.
  • Ability to work independently in a fully remote environment.
  • Reliable desktop or laptop computer and stable internet connection.
  • Willingness to use a primary personal Google account and enable personal data sources for evaluation purposes.
  • Availability to work 30-40 hours per week during local time zone hours.
  • Experience in AI evaluation, annotation, content review, or analytical research roles is preferred.

Benefits

  • 100% remote work.
  • Working hours aligned with your local time zone.
  • 30-40 hours per week commitment.
  • Paid hourly based on hours logged and approved.
  • 1-month contracting engagement with possible extension.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Mindrift.ai: Be the “I” in AI

Mortgage Underwriter - Freelance AI Trainer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking mortgage underwriting and loan origination professionals for project-based AI evaluation work focused on testing and improving mortgage-related AI outputs and compliance decisions.

Greece Portugal Italy Part-time Mid Level AI (Artificial Intelligence) Financial Analyst
Up to $104k
13 hours, 2 minutes ago
Mindrift.ai: Be the “I” in AI

Claims Processing Agent - Freelance AI Trainer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking part-time project-based insurance and claims specialists to evaluate and improve AI systems for auto insurance decision-making, fraud detection, and subrogation testing.

United States Part-time Mid Level AI (Artificial Intelligence)
Up to $125k
13 hours, 2 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance opportunity with an AI training platform recording first-person videos of everyday household activities to help train AI systems and robots.

Guatemala Freelance Entry Level AI (Artificial Intelligence)
$12k-$12k
13 hours, 3 minutes ago
Mindrift.ai: Be the “I” in AI

Freelance Agent Evaluation Engineer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking a project-based software specialist to create realistic coding evaluation tasks and tests for AI agents in simulated development environments.

France Part-time Senior AI (Artificial Intelligence) Software Engineer
Up to $104k
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
13 hours, 3 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers