Baseball Fan – Live MLB Game AI Evaluator (US)

$100–120/task

rate as listed

Workload: ~15h/week
Engagement: Project-based
Location: US
Field: Other

Evaluate AI assistants live during MLB postseason games by asking the questions a fan actually asks, then rating two products side by side on helpfulness, accuracy and presentation. Work is event-based in October, roughly 2 to 2.5 hours per game while it is being played.

What the work involves

On scheduled MLB postseason game days, you follow the game live and put real fan questions to two AI assistants at the same time: live score and win probability, player and team stats, rules and challenge situations, schedules and broadcast details, highlights, and expert-style analysis. Each prompt runs as either a single question or a multi-turn conversation of 2 to 10 turns, with the same question put to both products. You capture share links and full screenshots of every conversation, rate each response for helpfulness, factual accuracy and presentation, pick which product did better, and write a rationale that explains the call in plain terms.

The accuracy judgment is the hard part. A model can produce a confident, well-formatted answer about a pitcher's postseason ERA or the runner-on-second extra-innings rule that is simply wrong, and your job is to notice it while the game is still moving. Knowing where to verify quickly, and knowing when an answer is stale rather than false, matters more than writing length.

What the platform screens for

Mercor's screen is AI-led and leans on verifiable specifics. Expect questions about US location, at least one year of professional work experience, and genuine baseball depth: postseason seeding and format, replay review and challenge rules, how you would read a win probability swing. Comfort with AI chat assistants and web search, strong written English and a demonstrated eye for factual error all come up. Prior AI rating or evaluation experience is a plus, not a requirement. Being an avid MLB fan or broader sports enthusiast is also treated as a plus.

Logistics

Remote and US-based, but not async: tasks are completed live while the game is being played
Event-based across October, Division Series through the World Series, with the first game day as early as Saturday, October 3
Roughly 2 to 2.5 hours per task, scheduled around game windows including evenings and weekends
Requires a setup that can run two AI products side by side, follow the game, and capture share links and full screenshots

Pay band: $100–120/task.

What they look for

Solid baseball knowledge: rules, teams, players, postseason format
Ability to spot factual errors in confident-sounding text
Comfort using AI chat assistants and web search
Strong written English for clear side-by-side rationales
At least 1 year of professional work experience
Availability to work live during scheduled game windows

Included with this listing

Interview prep — 10 questions

Matched to what this platform's screening actually measures. Answer each out loud in 60–90 seconds: claim → concrete example → implication.

What it tests: Whether your fandom is current and detailed enough to generate natural in-game questions, rather than general sports familiarity.
A strong answer: Specifics that only a regular follower would have: teams you track, how you consume games (broadcast, Gameday, box scores), and a recent postseason situation you remember in detail.
Where people lose it: Claiming to be a lifelong fan but naming nothing from the current or most recent season, which the follow-up question will expose immediately.

Summary, eligibility guidance and screening preparation are prepared by Sidequest from the platform's posting. Check the source listing before applying.