Can QM Keep Team Decisions in the Right Scope?

Pilot Y Combinator’s QM with synthetic decisions in Slack and on the web. Check that teammates retrieve the current decision, outsiders receive none of its markers, and the same checks pass after your intended idle interval.

Cross the surface and the permission boundary

QM documents personal and shared scopes and a common identity and configuration across Slack and the web. Turn that into a test with two accounts, one shared project, and a second project without a grant. Put a made-up decision such as `ORBIT-72` in the shared project and an unrelated private marker in the owner’s personal scope. Ask a neutral question from Slack and the web, first as a collaborator and then from the ungranted project or account. Save the raw replies and any visible tool results as separate observations. A collaborator should recover the shared marker without the private one, while the ungranted probe should contain neither. Repeat after your target idle period without putting the answers into the new prompts. This distinguishes a scope failure from a retrieval miss and shows whether one surface behaves differently for the same task.

GitHub - yc-software/qm: Multiplayer agent harness for teams · GitHub

Use a decision log as the answer key

GBrain’s repository describes a facts page and a decisions log as reference material for agents. Borrow the decisions-log pattern to make a QM pilot auditable independently of the harness. Write a synthetic decision with a unique marker, then append a correction with a different marker and an explicit supersession link. Keep both entries in the test source so retrieval has to resolve the conflict. Ask for the currently approved choice in a fresh conversation and again after the idle interval. For a question that asks only for the current choice, count any retired marker as stale output, including replies that give both values. If both fresh conversations fail while an existing thread passes, mark continuity as dependent on the tested thread until you can reproduce a clean-session pass. Keep scope checks separate so a correct answer from the wrong audience still fails.

GitHub - garrytan/gbrain: Garry's Opinionated OpenClaw/Hermes Agent Brain · GitHub

Check whether a correction sticks

Use a fictional decision that changes once. In this illustrative test, a facilities coordinator initially assigns the east entrance to a delivery crew, then corrects the plan to the west entrance and gives a reason. Ask QM to record the current entrance and reason in the conversation. Close the browser tab, reopen the same conversation, and ask for the current decision without supplying either entrance.

Pass if the answer gives the west entrance and the original reason. If it gives the east entrance, inspect the saved conversation for the correction. When the correction is visible, retry with an explicit request for the most recent decision. If it is missing, check that you reopened the same conversation and account. Test a fresh session separately if your workflow requires recall across sessions.

GitHub - yc-software/qm: Multiplayer agent harness for teams · GitHub

Test the actual Slack audience

Upstream QM describes a shared identity and configuration across Slack and the web. Check whether your hosted workspace offers Slack setup and which workspace permissions it requires. If you can use a test channel, put a fictional volunteer roster in the shared project and a distinct fictional planning note in your personal scope. Ask a second tester to retrieve the roster in the intended Slack channel without quoting the roster in the request.

Pass if the tester sees the expected roster and the interaction reveals none of the distinctive details from the personal note. Record who asked, where they asked, and which scope held each item. If the roster is missing, check the Slack connection and project membership before changing access. If personal material appears, pause the rollout and reproduce the result using fictional data.

GitHub - yc-software/qm: Multiplayer agent harness for teams · GitHub

Sources