/devloop

An unattended dev loop that works your issue queue

Open project

--No baseline yetNot run

Recent runs

No runs have been recorded for this target.

Runner not connectedNo benchmark yet — battle-tested running this site's own backlog.

The build must pass, and be idempotent, before a PR opens

Add your own case

Describe a real task and the behavior you expect. The case is checked for clarity and testability before it can join a run.

Functional dimension

RoutingWrite two requests that look similar but should take different paths.GroundingAsk for a factual answer whose evidence exists in one known source.Tool useDescribe a request that cannot be completed correctly without one specific tool.Resource deliveryAsk for a specific file or destination and state exactly what should be returned.Answer qualityProvide the important facts the answer must include and anything it must avoid.Multi-turn contextWrite a short conversation where the final turn depends on one earlier detail.LatencyChoose a common request and set a realistic first-token or completion budget.ReliabilityRewrite the same intent in another language or with a different phrasing.

Language User request Expected behavior Answer must include Optional Answer must not include Optional Email for the report Optional

If you add an email, a report is sent after this accepted case completes a production run.

Complete the case, then run the quality review.

Cases added in this browser

No user-authored cases yet.