Monitor Usage, Audit Logs, and Risk

After launch, admins need an evidence trail for what users ran, what tools changed, which configuration changed, and whether agents are helping or creating risk. Use Tool Executions, Audit Log, conversations, feedback, and analytics as separate signals alongside workflow history and token usage.

What To Monitor

Signal	Use it for
Tool Executions	Tool calls, status, arguments, results, approvals, failures, and execution time
Audit Log	Configuration and access changes with user, entity, timestamp, correlation ID, and changed fields
Conversations	Real user behavior, prompt gaps, incorrect answers, and sensitive-data exposure
Feedback	Negative ratings, user corrections, and improvement candidates
Agent Analytics	Usage, tokens, active users, performance, and adoption trends
Workflow History	Changes to workflow logic and run behavior
Token limits	Budget control for shared model connections by organization, team, or user

Tool Executions

Use Tool Executions whenever an agent or workflow uses a Connection function, REST tool, MCP tool, Platform Tool, or sub-agent-like action.

Review these fields during troubleshooting:

• agent and conversation,
• action/function,
• status: Pending, Success, Failed, or PendingApproval,
• input arguments,
• result and error message,
• whether approval was required,
• who approved or rejected,
• resolved time,
• execution time where available.

If a user says "the agent did the wrong thing", start here before editing prompts. Tool Executions show what was actually sent to the tool.

Audit Log

Use Audit Log for configuration and access changes. Audit entries include the changed entity, change type, user, organization, timestamp, correlation ID, and changed properties. Pair Siesta audit data with logs from the connected external system when investigating record changes.

Audit review is especially important after:

• SSO or organization security changes,
• team membership changes,
• access-policy changes,
• connection credential changes,
• agent prompt/tool/data changes,
• workflow edits,
• API key or webhook changes.

Incident Playbooks

A Tool Failed

1. Open Tool Executions and find the failed action.
2. Check arguments, result, error message, and connection.
3. Verify the external credential still works.
4. Check whether the organization disabled the provider or function.
5. Retest with safe input.
6. If the failure is user-specific, check private connection ownership.

A Write Action Needs Approval

1. Confirm the function is intentionally in confirmation mode.
2. Review the arguments and target record before approving.
3. If approvals are too noisy, split safe read/draft functions from write functions.
4. Do not remove confirmation from customer, financial, delete, publish, or permission-changing actions without owner approval.

Token Usage Spiked

1. Identify the model connection.
2. Check usage by agent, workflow, team, and user.
3. Review recent workflow changes and automated runs.
4. Add or tighten org/team/user token limits.
5. Pause high-volume workflows if needed.
6. Review whether the agent is using unnecessary long context, files, or repeated tool calls.

A Public Agent Answered Poorly

1. Check the conversation and feedback.
2. Confirm the public agent has the right prompt and allowed tools.
3. Review uploaded files, public page context, and data collections.
4. Check whether public chat settings allow feedback, uploads, and privacy behavior as intended.
5. Patch the prompt or data source, then retest through the public/widget path.

Operating Rhythm

For the first week after launch, review Tool Executions and feedback daily. After stabilization, review weekly for production teams and after every major change.

Use a simple monthly review:

• remove stale users and teams,
• rotate or delete unused API keys,
• disable unused webhooks,
• review shared connections and function confirmations,
• check token limits against usage,
• review high-risk agents and workflows,
• confirm public features are still intended.

For the detailed investigation flow, continue with Review Tool Executions. Feed the results into Deployment Reporting and Monthly Operations Review.