STEM Expert Task Pilot

$2000/task

rate as listed

Workload: ~15h/week
Engagement: Project-based
Location: Location not stated
Field: Sciences

Design one difficult, open-ended STEM task that tests an AI agent on realistic expert work. The finished task must support automatic scoring from 0 to 1 with partial credit and is completed only after the proposed idea is approved.

The work

You will propose and, if approved, build one hard real-world task in your STEM field. The task should require expert-level technical work, permit multiple valid approaches, and produce results that an automatic scorer can grade from 0 to 1 with meaningful partial credit.

What the platform screens for

The application asks you to select suitable task types and explain one task idea in a few paragraphs. Reviewers are likely to examine whether the task reflects genuine domain difficulty, remains open-ended without becoming ungradable, and provides enough observable evidence to distinguish weak, partial, and strong AI-agent performance. You do not build the task until the idea is approved, and the listing says ideas are reviewed within 12 hours.

Logistics

This is a project-based submission with no stated location restriction or scheduled working hours, so the workflow appears remote and largely asynchronous. The listing estimates about 20 hours of work. Applications are due Oct 4, 2026 at 11:59pm PT, and the finished task is due Oct 6, 2026 at 9pm PT.

Pay band: $2000/task

Payment is contingent on acceptance of the finished task.

What they look for

Expert-level knowledge in a relevant STEM field
Design of difficult, realistic technical tasks
Construction of open-ended problems with multiple valid approaches
Development of automatic scoring criteria with partial credit
Concise technical proposal writing
Ability to complete a task on a short deadline

Included with this listing

Interview prep — 10 questions

Matched to what this platform's screening actually measures. Answer each out loud in 60–90 seconds: claim → concrete example → implication.

What it tests: This tests whether your claimed expertise is grounded in a specific project whose technical decisions and outcomes can withstand follow-up questions.
A strong answer: Name a concrete project, your exact responsibility, the difficult technical choices you made, and an observable result. Be prepared to explain methods, constraints, and failure modes rather than relying on a title or broad field label.
Where people lose it: Candidates lose points by naming a prestigious field or employer without identifying what they personally designed, analyzed, implemented, or validated.

Summary, eligibility guidance and screening preparation are prepared by Sidequest from the platform's posting. Check the source listing before applying.