AI Agent Evaluator

$25–45/hr

rate as listed

Workload: ~15h/week
Engagement: Project-based
Location: US
Field: Generalist

Evaluate two AI agents side by side as they attempt everyday errands such as bookings, returns, shopping and form completion. You will approve key actions, record each phone session and score the agents on completion, quality, control, satisfaction and time.

The work

This contractor role involves running real-life scenarios with two AI agents using your own accounts and smartphone. Scenarios may include bills, returns, email follow-ups, bookings, shopping, groceries, travel, family logistics, outings and forms. For each run, you will approve or stop key actions, screen record the full attempt, score both agents and provide written evidence about what worked and what failed.

What the screening covers

micro1 is looking for U.S.-based candidates with a Bachelor’s degree, 2-3 years of experience and an existing habit of using AI in daily life. Engineering, business management and office or operations experience is preferred, while generalists are also welcome. No coding or formal AI background is required. Expect the screening to test whether you can follow a scenario exactly, distinguish task completion from task quality, identify control problems and support a score with observable evidence.

Practical details

The work is remote, phone-based and scheduled around your availability. A run takes approximately 1.5 hours, including evidence collection and scoring. You need a smartphone capable of screen recording and an active personal account on a popular AI service at the Free, Go or Plus tier, not Pro. The listing gives conflicting kickoff dates, so confirm the current onboarding and start schedule directly with micro1.

Pay band: $25–45/hr

What they look for

ChatGPT
Muse
AI agents
Attention to Detail
Following Instructions
Screen Recording
Dots
Mobile

Included with this listing

Interview prep — 10 questions

Matched to what this platform's screening actually measures. Answer each out loud in 60–90 seconds: claim → concrete example → implication.

What it tests: This verifies the stated education and experience requirements while testing whether your work involved concrete processes, decisions or customer-facing outcomes that transfer to agent evaluation.
A strong answer: Name the degree, employer or work setting, dates and specific responsibilities. Connect one or two recurring workflows to careful execution, documentation or review without claiming that an AI background is required.
Where people lose it: Candidates lose points by giving only a job title or describing broad responsibilities without enough dates, tasks or ownership details to verify the experience.

Summary, eligibility guidance and screening preparation are prepared by Sidequest from the platform's posting. Check the source listing before applying.