What the work involves

You connect your own Google account to Gemini and then evaluate what it does with that context. A prompt might ask the model to summarize a thread you actually received, find a document you actually wrote, or recommend something based on your calendar. Your job is to judge whether the retrieval was correct, whether the response used your data appropriately, and whether it invented facts or made assumptions your history doesn't support. Much of the day is side-by-side comparison — two candidate responses, and a written rationale for which one serves the user better and why.

The failure modes you're hunting are specific to personalization: the model pulls the right email but the wrong date, confuses two people with similar names, surfaces a stale calendar entry as current, or produces something plausible that has no basis in your actual data. Because only you can see your own account, you are the only ground truth available — which is why written rationales matter more here than on generic rating projects.

What the platform screens for

  • Genuine account depth. Projects like this fail on thin accounts. Screeners ask about real, sustained use of Gmail, Calendar, Drive, and Photos — enough history that retrieval prompts have something to retrieve.
  • Willingness to grant permissions. This is a hard gate. If connecting your personal Google apps to an evaluation system is not something you're comfortable with, the role is not workable, and saying so early saves everyone time.
  • Rationale quality. Expect to be asked to justify a rating in writing. Vague verdicts ("response B was better") score poorly; specific, checkable reasoning scores well.
  • Rubric discipline. Whether you can apply someone else's guidelines rather than your own taste.

Logistics

Fully remote, US-based only. Contractor engagement of up to 16 weeks, with a stated commitment of at least 4 hours per day and up to 40 hours per week, including 4 hours of overlap with Pacific time. A bachelor's degree in any field or equivalent practical experience is listed; prior annotation or QA work is preferred but not required. Desktop or laptop and reliable internet. Pay is not disclosed in the listing. Shortlisted candidates receive a Job Interest Form, then consent forms and NDAs before onboarding.