RemoteLeaf Logo
Log in Sign up

AI Evaluator - POLISH

4 days, 21 hours ago
Poland
Contract
Junior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Apply Now
Gramian Consultancy Group

Gramian Consultancy Group

Professional Services
View All Jobs 26

Description

  • Design and run short multi-turn conversations to test AI personalization behavior.
  • Create prompts based on realistic personal scenarios and lived context.
  • Review AI responses to assess whether personalization is applied correctly.
  • Check grounding quality to ensure the model does not invent unsupported claims about the user.
  • Evaluate whether personal signals are used naturally and appropriately in responses.
  • Compare two responses side by side and determine which is more helpful, natural, and relevant.
  • Write clear, structured rationales that explain rankings and reference specific conversation turns.
  • Verify debug information to confirm the correct data sources were used.
  • Maintain strict workflow hygiene, including deleting evaluation conversations when required.

Requirements

  • Strong Polish proficiency in reading and writing; Polish is the primary evaluation language.
  • BS/BA degree or equivalent experience in a relevant analytical field such as Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or a related discipline.
  • Strong analytical thinking and ability to assess nuanced AI outputs.
  • Excellent written communication skills with the ability to produce structured evaluation notes.
  • High attention to detail when comparing similar responses.
  • Ability to work independently in a fully remote environment.
  • Reliable desktop or laptop computer and stable internet connection.
  • Willingness to use a primary personal Google account and enable personal data sources for evaluation purposes.
  • Availability to work 30-40 hours per week during local time zone hours.
  • Experience in AI evaluation, annotation, content review, or analytical research roles is preferred.

Benefits

  • 100% remote work.
  • Working hours aligned with your local time zone.
  • 30-40 hours per week commitment.
  • Paid hourly based on hours logged and approved.
  • 1-month contracting engagement with possible extension.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

FutureSight

Co-Founder & CEO - AI Communication Agents for Freight & Logistics

FutureSight 11-50 Internet Software & Services

FutureSight is seeking a Co-Founder & CEO to build and lead HawkAI, an enterprise AI venture for US logistics brokerages, carriers, and warehouse operators focused on automating high-volume freight communications.

Canada Full-time Executive AI (Artificial Intelligence) Head of Sales
LLM
2 hours, 6 minutes ago
ELVTR

AI Product Manager

ELVTR 51-250 Diversified Consumer Services

ELVTR is hiring a Middle Product Manager to build AI-native internal tools and, over time, student-facing products that improve key business workflows and drive adoption across the company.

Ukraine Full-time Mid Level AI (Artificial Intelligence) Product Manager
HubSpot Microservices Supabase UI Design UX Design Vercel
2 hours, 10 minutes ago
Toloka

AI Training Specialist (Egocentric Video)

Toloka 251-1K Internet Software & Services

Project-based freelance opportunity on an AI training platform for recording first-person videos of everyday household activities to help train AI systems and robots.

Chile Colombia Bolivia Uruguay Part-time Entry Level AI (Artificial Intelligence)
2 hours, 27 minutes ago
Aspire Software

AI Marketing Assistant

Aspire Software 251-1K Internet Software & Services

Cybertill is seeking an AI Marketing Assistant to help its Marketing team use AI-driven workflows to create content, run campaigns, and support both the Cybertill and Kudos Software brands across digital and traditional channels.

United Kingdom Full-time Junior AI (Artificial Intelligence)
Copywriting Email Marketing Google Ads HubSpot Keyword Research SEO
2 hours, 37 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers