Scrapbook: Audit Intelligence Features Design
Date: 2026-02-16T17:00:00Z
Goal: Define 5–10 concrete MVP features that generate immediate business value for the audit overlay. Each feature must:
- List required data signals (which BusinessEvents or raw events)
- Provide a simple heuristic baseline (rule‑based)
- Explain how AI improves it (summaries, explanations, semantic search)
- Suggest how to evaluate success in a pilot (metrics)
Audience: Antonio (needs clear, practical features that sell).
Feature ideas:
1. Daily Audit Summary – Natural language digest of the day’s key events.
- Signals: all BusinessEvents for the day.
- Heuristic: concatenate top events (by entity importance) into bullet list, then summarize with LLM.
- AI improvement: LLM writes coherent narrative, highlights anomalies, explains impact.
- Metrics: time saved for auditor; adoption rate; qualitative feedback.
2. Inventory Anomaly Detection – Flag suspicious inventory changes.
- Signals: `inventory.adjustment` events, especially negative adjustments outside normal sale flow.
- Heuristic: Z‑score of daily adjustment volume per branch/product; after‑hour adjustments (outside 08:00–20:00); adjustments with reason `THEFT` or `OTHER`.
- AI: LLM generates explanation for each anomaly, linking related events (e.g., “Three large negative adjustments occurred after closing; possible theft.”).
- Metrics: number of true positives confirmed by manual review; reduction inundetected shrinkage.
3. Suspicious User Activity – Detect risky user behavior.
- Signals: `user.role_changed`, `user.created`, `user.deactivated`, plus bulk operations (many updates by same user in short time).
- Heuristic: count of changes per user per hour; role changes outside business hours; new users granted elevated roles immediately.
- AI: LLM profiles the user’s activity pattern and writes a short risk note.
- Metrics: audit time reduction; prevention of unauthorized access.
4. Branch-to-Branch Variance – Identify branches that deviate from norms.
- Signals: aggregated daily sales, inventory turns, adjustment rates per branch.
- Heuristic: compare each branch to rolling 30‑day average; flag >2σ deviation.
- AI: LLM explains possible reasons (e.g., “Branch 5’s sales dropped 40% while theft adjustments spiked.”).
- Metrics: early detection of operational issues; management satisfaction.
5. Entity Traceability Report – “Show me everything that happened to X.”
- Signals: any event affecting a given entity (product ID, user ID, branch ID).
- Heuristic: timeline of all BusinessEvents with that entity ID.
- AI: LLM summarizes the timeline, highlights unusual patterns (e.g., “Product 123 had three price changes in one day.”).
- Metrics: time saved during investigations; completeness of audit trail.
6. Sales Anomaly Detection – Spot abnormal sales patterns.
- Signals: `sale.completed` events.
- Heuristic: very high‑value sales, many small sales in rapid succession (possible money laundering), refunds without original sale.
- AI: LLM explains context (“High‑value sale occurred just after inventory adjustment.”).
- Metrics: fraud detection rate; false positive tuning.
7. Purchase Order Fraud – Flag suspicious POs.
- Signals: `purchase_order.received` with quantity discrepancies; new suppliers; large orders.
- Heuristic: orders >3σ above historical average for that supplier; new supplier orders over threshold.
- AI: LLM generates investigative summary.
- Metrics: cost savings from catching overbilling.
8. Role Change Alert – Watch for unauthorized privilege escalations.
- Signals: `user.role_changed`.
- Heuristic: any role change where new role has higher privileges (based on privilege matrix) and not performed by admin.
- AI: LLM provides natural‑language alert: “User X promoted to manager by Y, outside normal hours.”
- Metrics: security incidents prevented.
Pilot evaluation: We’ll instrument each feature with counters: number of alerts, user acknowledges, manual review outcomes (true/false positive). Success = auditors find at least one material issue they would have missed manually, and overall time to complete daily audit checklist is reduced by 30% within 2 weeks of deployment.
---
This will be structured into Report 4.