AgentGem Benchmark — Feedback Loop

producers contribute outcomes (consent-gated) → hosted aggregator gates + k-anonymises → public roll-up and org-scoped view read back. Loop closed; admins govern participation.

api.agentgem.ai — Fly (iad) hosted aggregator + Neon Postgres · the ONLY DB with real data POST /ingest ✓ write k-anon read roster read admin-only · org-internal same-origin contribute_allowed → forbid gate feedback loop · the aggregate informs "which model / which gem should I use?" AgentGem Desktop PRODUCER · consent-gated • scan .claude transcripts • judge outcomes (ACP · opt-in) • sign attestation (ed25519) POST /api/aggregator/ingest sign_and_publish → per-model outcomes bulk contribute → ingredients-only gate: benchmarkContribute() → hostedIngestEndpoint() Aggregator · /ingest verify → gate → project trust + governance gates signature · anti-inflation quarantine · trust score k-anon: ≥ k distinct producers org forbid → org-forbidden Neon Postgres attestations · model_outcomes account_bindings org_members · org_settings LIVE: producers accruing consent-gated ingest forbid = server-enforced, forward-only (no delete) /benchmarks · /effectiveness k-anon roll-up public read · no auth /api/orgs/:scope/benchmark admin-only · org_members roster NO k-floor · org-internal BenchmarkProxyController /api/benchmark anon server-side GET → hosted Desktop · Benchmark tab reads via proxy ✓ (client-mode) Marketplace SPA app.agentgem.ai · /effectiveness NEW Marketplace · /orgs/:scope Benchmark tab (org-internal) beside Team Pulse usage NEW Admin · Governance POST /benchmark/settings (App-gated) contribute_allowed · view_enabled most-restrictive-org-wins Legend client / org view hosted API Postgres / k-anon governance / forbid feedback

Loop closed (both breaks fixed)

  • Producer → prod. Publishing now ingests when benchmarkContribute() is on: the interactive path uses hostedIngestEndpoint(); bulk contribute refreshes producer + ingredients-only usage. Opt-in, anonymous attestations.
  • Consumer → read. BenchmarkProxyController serves an anonymous server-side GET → hosted; the desktop tab reads via the proxy (client-mode, no local PGlite).

Org-scoped view

  • Admin-only /api/orgs/:scope/benchmark — a Benchmark tab on the marketplace org hub, beside Team Pulse.
  • Org-filtered aggregates: producer's bound account_login ∈ the App-synced org_members roster, no k-floor (de-anonymised within the org).
  • Non-App orgs (membership via account_scopes only) show empty — deferred.

Governance (admin)

  • Two org_settings flags: contribute_allowed (server-enforced forbid) + benchmark_view_enabled.
  • Forbid is enforced at ingest via producerForbidden (most-restrictive-org-wins) and is forward-only — it stops new contributions, it does not delete data already sent (purge = Spec B).
  • The settings write is App-gated (via === "app", else 409) so the write surface equals the enforcement surface.

AgentGem benchmark feedback loop · loop closed (producer wiring + read-fix) · org-scoped view + admin governance shipped · #396/#398/#404/#406 · #411/#412 · #413/#414