AI Governance Framework: Step-by-Step Guide
A practical AI governance framework has six components: a written acceptable use policy, an approved AI vendor inventory with DPAs, technical controls at the endpoint and browser layer, audit logging, employee training, and a review cycle. Below is a step-by-step guide to building each component, the order to build them in, and how each maps to SOC 2, HIPAA, and GDPR controls.
This framework is what works in practice across mid-market and MSP-served organizations. Larger enterprises may layer additional structure (AI Governance Councils, model risk committees), but the six components are foundational regardless of size.
The Six Components
| Component | Purpose | Maps to |
|---|---|---|
| 1. Acceptable Use Policy | Defines rules | SOC 2 CC1, HIPAA §164.316, GDPR Art 32 |
| 2. Vendor Inventory + DPAs | Lists approved AI tools | SOC 2 CC9.2, GDPR Art 28 |
| 3. Endpoint + browser controls | Enforces the policy | SOC 2 CC6.1, HIPAA §164.312 |
| 4. Audit logging | Evidence the controls work | SOC 2 CC7.2, HIPAA §164.312(b), GDPR Art 30 |
| 5. Employee training | Policy awareness | SOC 2 CC2.2, HIPAA §164.308 |
| 6. Review cycle | Keeps the program current | SOC 2 CC4.1 |
Skipping any of the six produces a gap that auditors increasingly find.
Step 1: Publish an AI Acceptable Use Policy
Effort: Low, 1-2 weeks. Owner: CISO / IT Director, with HR and legal review.
The AUP is the foundation. Without it, the rest of the program is operating without a stated rulebook. The policy defines: which AI tools are approved, what data is prohibited, how personal accounts are treated, how violations are handled.
Use our free AI acceptable use policy template as a starting point. Customize the placeholders, have legal and HR review, distribute via your HR system, and collect acknowledgements.
For a deeper breakdown of what to include and how to structure it, see our AI acceptable use policy guide.
Step 2: Build an AI Vendor Inventory
Effort: Low to moderate, 2-4 weeks depending on existing vendor management maturity. Owner: Procurement or vendor management, with security review.
The vendor inventory pillar is straightforward but often skipped. For each AI tool your organization formally uses, capture:
- Vendor name and product
- Link to the DPA or data processing terms
- Completed security questionnaire or SOC 2 report from the vendor
- Date of last review
- Approved data categories (what content can be sent to this vendor)
Common starting entries:
- OpenAI Enterprise (DPA via enterprise agreement)
- Anthropic Business (DPA via business agreement)
- Google Workspace AI features (covered under Google’s data processing amendment)
- Microsoft Copilot for M365 (covered under Microsoft’s data protection agreement)
- GitHub Copilot Business (covered under Microsoft’s enterprise terms)
This step alone closes a major SOC 2 CC9.2 gap.
Step 3: Deploy Endpoint and Browser Controls
Effort: Moderate, 2-4 weeks including POC. Owner: IT / security operations.
The technical enforcement layer is what turns the AUP from a document into a working program. The control needs to cover three layers:
- Browser layer, a managed browser extension that watches paste events on AI tool domains
- Endpoint layer, an agent that detects desktop AI applications and can enforce blocking
- Content classification, recognition of sensitive data types (PII, credentials, source code, PHI, financial)
Deploy in monitor-only mode for two weeks to baseline actual usage. Then enable blocking on the highest-risk classifiers first (credentials, PHI, customer PII). Expand from there.
See our shadow AI detection guide for the technical detection model, or ShadowLock’s AI governance platform for a deployment that handles all three layers.
Step 4: Configure Audit Logging
Effort: Low if your platform handles it; high if you are stitching things together. Owner: Security operations, with compliance review.
Every AI event, sanctioned or unsanctioned, blocked or allowed, should produce an audit record containing: user, timestamp, AI tool, content classifier matches, outcome (allowed / blocked / alerted). Retention should align to your compliance program (90 days minimum for most SOC 2 programs).
The audit logs are what auditors examine. Their absence is the difference between “we have policies” and “we have evidence.”
Step 5: Roll Out Employee Training
Effort: Low, typically rolled into existing annual security training. Owner: HR or security awareness, with IT input.
Training does not need to be elaborate. A 10-minute module covering: what the AUP says, why it exists, what the approved tools are, what the prohibited data categories are, and how to request approval for new tools.
Pair training with the AUP acknowledgement collection. The combined record, “employee acknowledged the AUP and completed training”, is solid audit evidence.
Step 6: Establish a Review Cycle
Effort: Low. Owner: AI governance owner (typically CISO or IT Director).
The review cycle is what keeps the program from going stale. At minimum:
- Quarterly: Review the approved AI tools list. Add new tools that have completed vendor review. Remove tools no longer in use.
- Quarterly: Review the audit log summaries. Identify trends in violations and adjust policies if needed.
- Annually: Full policy review. Update for any regulatory or vendor changes.
- Event-triggered: SOC 2 audit findings, regulatory changes, material vendor changes, or incidents.
Document each review. Audit evidence of the review cycle is itself part of the audit evidence package.
How the Framework Maps to Specific Frameworks
SOC 2 mapping
| Component | SOC 2 Criteria |
|---|---|
| Acceptable Use Policy | CC1.1 (control environment), CC2.2 (communication) |
| Vendor Inventory | CC9.2 (vendor risk management) |
| Endpoint + browser controls | CC6.1 (logical access), CC6.7 (data transmission) |
| Audit logging | CC7.2 (system monitoring), CC7.3 (incident detection) |
| Employee training | CC2.2 (communication), CC1.4 (commitment to competence) |
| Review cycle | CC4.1 (control monitoring) |
HIPAA mapping
The relevant safeguards are administrative (§164.308), physical (§164.310), and technical (§164.312). For AI specifically:
- Audit Controls, §164.312(b), covered by Component 4
- Access Control, §164.312(a)(1), covered by Component 3
- Information Access Management, §164.308(a)(4), covered by Components 1 and 2
- Security Awareness and Training, §164.308(a)(5), covered by Component 5
GDPR mapping
- Article 28 (processor agreements), covered by Component 2 (vendor inventory + DPAs)
- Article 30 (records of processing), covered by Component 4 (audit logging)
- Article 32 (security of processing), covered by Component 3 (technical controls)
- Article 13 (information provided to data subjects), relates to monitoring acknowledgement in Component 1
The Sequencing That Works
The order matters. The fastest path from no program to audit-ready:
- Month 1: Components 1 (AUP) and 2 (vendor inventory). These are the cheapest and most foundational.
- Month 2: Component 3 (technical controls). Deploy in monitor-only first.
- Month 3: Components 4 (audit logging) and 5 (training). Audit logs flow from the technical controls. Training pairs with the AUP acknowledgement.
- Month 4 and ongoing: Component 6 (review cycle).
By month four, most organizations have a working program with all six components in place. The technical controls are by far the highest-leverage piece, they enable the audit logging that auditors actually examine.
Common Implementation Mistakes
Mistake 1: Starting with technical controls before policy. The technical layer needs a policy foundation. Without the AUP, the controls are operating in a vacuum and employees have no clear standard to follow.
Mistake 2: Treating vendor inventory as paperwork. The vendor inventory is the legal foundation for using AI tools at all. Skipping it produces compliance gaps that no technical control can close.
Mistake 3: Skipping monitor-only mode. Going straight to blocking produces user backlash and a flood of exception requests. Two weeks of monitor-only first gives you the baseline data to make informed blocking decisions.
Mistake 4: Treating training as a checkbox. A 10-minute training module that employees actually engage with beats a 60-minute module they click through.
Mistake 5: No review cycle. Programs without review cycles go stale within a year. The AI vendor landscape changes faster than that.
Frequently Asked Questions
What is an AI governance framework?
An AI governance framework is a structured approach to building an AI governance program. The framework defines the components (policy, vendor inventory, technical controls, audit, training, review), the sequence to build them in, and the mappings to compliance frameworks.
Is there an industry-standard AI governance framework?
Not yet. NIST published the AI Risk Management Framework in 2023, which is a useful reference but is primarily focused on AI system risk rather than enterprise AI tool usage governance. ISO/IEC 42001 (AI management system) is the closest formal standard. For most organizations, an internal framework grounded in the six components above is more practical.
How long does it take to implement an AI governance framework?
Three to four months for most mid-market organizations. The first month is policy and vendor inventory (cheap and fast). The second is technical controls deployment (the biggest single piece of work). The third is training, audit log configuration, and refinement. By the fourth month most organizations have a working program.
Does ShadowLock implement an AI governance framework?
ShadowLock provides the technical pieces of the framework, endpoint + browser controls, content classification, and audit logging. The policy, vendor inventory, training, and review cycle remain organizational responsibilities, but ShadowLock customers get a working technical foundation in under an hour.
Can a small organization implement this framework?
Yes, and faster than larger ones. The six components scale down well. A 50-person organization can complete the entire framework in a single month if leadership is engaged. The components are the same; the bureaucratic overhead is less.
How does the framework handle new AI tools?
The exception request process (part of Component 1) is how new tools enter the framework. An employee requests approval, the security team runs vendor review (Component 2), and if approved the tool is added to the AUP and to the technical controls’ allow-list (Component 3). The review cycle (Component 6) is when this catches up if exceptions have piled up.
What if my organization is not regulated?
The framework still applies. Even without specific regulatory requirements, every organization has sensitive data (customer records, source code, financial information, IP) that should not flow to unapproved AI tools. The framework is about controlling data flow, regardless of which specific framework the controls are mapped to.
The six-component framework is what works in practice. It is opinionated, there are other ways to structure the program, but it is grounded in what auditors actually examine and what produces the audit evidence the program is built to support. Start with the policy and vendor inventory, layer in the technical controls, and the rest follows.