Reviewers for agent trajectories and evaluations.
Reviewers who can say exactly what went wrong in a long agent run, at which step, and why it matters.
What these contributors do.
Reading multi-step, tool-using runs end to end and scoring them against a rubric, step by step.
Criteria specific enough that two reviewers agree, and general enough to cover the runs they have not seen yet.
Classifying where and how agents fail, so a team knows what to fix first.
Pushing agents toward unsafe, wrong or costly actions and recording what gets through.
Evaluated on the work itself.
Contributors score real agent trajectories against a rubric and explain each judgement. A reviewer checks their scores against a reference and reads the reasoning, not only the number.
Who we are looking for.
People with hands-on experience of agents, tools or evaluation, who write clearly and can defend a judgement. Joining is free, and we never charge contributors a fee.
Need Agent Evaluation experts?
Tell us what you need in a line or two. A person on our team reads every request and replies by email.
Other areas: Coding & Software Engineering · RL Environments · Technical Review