The work
You will turn real software repositories into challenging evaluation tasks for frontier AI models. Day-to-day work includes defining a concrete engineering problem, preparing a reproducible environment, writing a reference solution, and building deterministic verifiers that accept different correct implementations while rejecting incomplete, incorrect, or hard-coded submissions. Tasks may involve subtle bugs, non-trivial features, large refactors, legacy modernization, or meaningful performance improvements.
What the screening covers
The AI-led screen is likely to ask for verifiable examples of work in substantial codebases, then probe how you debugged, refactored, optimized, or delivered a feature. It will also test evaluation judgment: whether you can identify ambiguity, control environmental variables, define observable correctness, and design tests that do not overfit to one implementation. The snapshot says you’ll need hands-on expertise in at least one of Python, Java, Rust, Go, C++, or TypeScript. The Preferred Qualifications section separately describes significant hands-on expertise in at least one of Python3, Java, Rust, Go, C++, or TypeScript as preferred; previous AI experience is not required.
Practical details
The contractor engagement is fully remote and project-based, with an expected commitment of about 15 hours per week. The listing estimates five hours per trainer task and 1.5 hours per reviewer task, although actual time may vary by experience and workflow. It does not state that the engagement is async or specify fixed working hours. Selected candidates are expected to begin their first tasks within 24 to 48 hours after onboarding.
Pay band: $50–100/hr
Compensation is output-based and paid per task that meets project specifications, so task approval and verifier quality matter directly.