What Changed Watch
Keeps a dated log of changes to the AI tools you use, and tells you which roles to re-check.
The monthly re-check
Once a month, re-test each role on your last ten cases after checking what your vendors changed. Every approved edit goes in a dated changelog.
The ECHO line
Nothing in a role changes without your OK, and each change is logged with its date, its reason and the version it replaced.
Monthly upkeep for any business running AI roles. The What Changed Watch lists vendor updates and new defaults, your last ten cases run again through each role, and Granted vs Reached compares what each agent can reach with what you think it can. You approve every edit, and each one goes in a dated changelog.
TO BE TESTED — RESULTS WILL BE SHAREDFrom how we run our own AI agent teamSetup: about 60 minAdvanced5 of 5 free opens left
At a glance
- Your last ten cases run through each role every month.
- Anything an agent can reach that you didn't grant comes to you right away.
- Role edits wait for your OK and go in a dated changelog.
Where the lines fall
AI may prepare
- A dated list of vendor changes, your last ten cases re-run through each role with the differences marked, the granted vs reached lists, and proposed edits.
Only after a person says yes
- Any edit to a role's text or settings.
- Removing or shortening a permission.
- Switching a tool or model.
Human reserved
- Deciding whether a changed behavior is acceptable for your business.
Stops when
A re-run does something a role forbids, an agent can reach more than you granted, or a vendor change can't be confirmed from the vendor's own pages.
Takes over
The owner
Steps
Reads each vendor's update notes and settings pages, and lists what changed this month, with dates and links.
AIA person
Runs the Upgrade Re-check (CK-05): the last ten real cases through each role, names and private details removed, on the model and settings it uses now.
AIA person
Runs Granted vs Reached (CK-02) for each agent: what you think it can do, next to what its settings let it reach.
AIYou
Decide what each change in behavior or reach means for your business.
YouYou
Approve each edit to a role, and each permission you remove or shorten.
Your OKWrites the dated changelog: what changed, why, the version it replaced, and the model and date it was tested on.
AI
Model fit
Best fit
- Everyday assistant
- Deep reasoning
Also fits
- Fast and low-cost
Avoid
- Personal agent (acts for you)
Reading vendor notes suits an assistant with web browsing; comparing re-runs with the originals suits a deep-reasoning model. Re-test each role on the model and settings it uses day to day, and note the model name and date, or the test tells you little.
Where it runs
- Web browser
- Documents and files
Risk
- Access
- Relationship
The monthly re-check · RL-WF-13
5 of 5 free opens left- ① Orientation
- ② Permission
- ③ Constraint ★
- ④ Invitation
The paste didn't load. Check your connection and try again.
These are samples. Fill in the [INSERT] blanks and adapt each one to your business.
A pasted role guides the agent. Your tool's settings are what stop an action.
Sample handoff note (fictional business)
Settings that enforce it
- Keep one current copy of each role and its settings, with a version date, and archive old ones where agents can't reach them.
- Keep a test mode or test contacts for re-runs, so no re-run reaches a real customer.
- During Granted vs Reached, review saved "always allow" approvals and any permission with no end date.
- Save each month's changelog in one place you can read.
- If your tool can't do one of these, leave that part of the job with a person.
One check before you trust it
Before the first run, change one setting yourself (for example, give one agent access to a folder it doesn't need). The re-check should report that the agent can now reach more than you granted, and stop for your decision.
Want it set up in your own tools? Book a Working Session · $450Working Sessions run in English or Spanish.