A content policy for AI products on Pancake — content standards, moderation, reporting, and enforcement
A complete Acceptable Use Policy for AI-powered (AIGC) products on Pancake. It sets the content standards, moderation, reporting, and enforcement that keep an AI product in good standing — pick what applies, replace the placeholders, publish.
This is a template, not legal advice. AI content rules vary by market and change fast — have a qualified AI-compliance professional review it before publishing if your model, content types, or jurisdiction are unusual.
If you resell or build on top of AI models, account review looks at how you handle the content your users generate. A published AUP plus real moderation is what keeps an AI product approved — and keeps it from being suspended later. The four things review cares about most:
Independent branding
Required: your own product name and identity. Not allowed: marketing yourself as an AI model brand (e.g., “the official GPT app”) or using a model’s name/logo as your own.
Honest marketing
Required: describe what your product actually does. Not allowed: overstating capabilities, implying endorsement by a model provider, or hiding that output is AI-generated.
Moderation in place
Required: run a content-moderation step (a moderation API or equivalent) on inputs and outputs. Not allowed: shipping an unmoderated raw-model passthrough.
Published content policy
Required: a public AUP (this template) covering prohibited content, reporting, and enforcement. Not allowed: no policy, or one that doesn’t match your actual product.
🔴 items decide whether an AI product is approved. 🟡 items strengthen the case.
1
🔴 Must be done
Product uses independent branding — not an AI model’s name or logo as your own
Marketing is accurate — no overstated capabilities, no implied provider endorsement
A moderation step runs on inputs and outputs (name the tool in Section 3.1)
The “Absolutely Prohibited” list (2.1) is published and enforced — CSAM, terrorism, WMD, non-consensual deepfakes
Prohibited and restricted categories match what your product can actually generate
At least one real, monitored reporting channel exists (Section 5)
Enforcement actions (Section 7) are ones you can actually carry out
2
🟡 Strongly recommended
Output is labeled or otherwise disclosed as AI-generated
Severity tiers (3.4) map to concrete response times (Section 6)
An appeals path exists for wrongly-actioned users (Section 8)
Age verification in place if you allow any restricted (e.g., adult) content
3
Final pass
Every [placeholder] replaced with real text
“Last Updated” date filled in
Categories/channels that don’t apply to your product are deleted, not left as stubs
This AUP is the standalone content policy. It complements Section 10 (Acceptable Use) of the Terms of Service template — the T&C points users at your full content policy; this is that policy.
Copy the block below and replace every [placeholder] with your real text. Delete any category, channel, or action that doesn’t apply to your product.
The template is in English. Translate or localize as your market requires — the content standards should stay the same.
Acceptable Use Policy — AI-Generated Content (replace every [placeholder])
[Platform / Company Name]Acceptable Use Policy — AI-Generated ContentLast Updated: [DATE][Platform / Company Name] ("Platform," "we," "us," or "our") is committed to providing safe, lawful, and responsible AI-generated content services. This Acceptable Use Policy ("Policy") applies to all content generated, published, distributed, or stored through the Platform, and sets out our content standards, moderation, reporting procedures, and enforcement actions. All users, developers, and merchants using the Platform must comply with this Policy.1. ScopeThis Policy applies to all content types, including:- Text: AI-generated text, dialogue, code, articles, etc.- Images: AI-generated or edited images and illustrations.- Audio / Video (if applicable): AI-synthesized voice, music, and video.- User Inputs (Prompts): prompts and instructions submitted by users.- [Other content types supported by the Platform, if applicable.]2. Content Standards & Prohibited Categories2.1 Absolutely Prohibited — Zero ToleranceThe following will be actioned immediately and may be reported to authorities:- Child sexual abuse material (CSAM) or any sexual content involving minors.- Content that promotes or facilitates terrorism, extremism, or mass violence.- Technical instructions for weapons of mass destruction.- Content inciting genocide or ethnic hatred.- Non-consensual deepfake sexual content targeting real individuals.- [Add categories based on applicable local regulations.]2.2 Restricted — Subject to Review- Adult content: only permitted on age-verified, compliant platforms.- Violent content: must have a clear creative or educational context.- Politically sensitive content: content that may be misleading.- Medical / legal / financial advice: must comply with local regulations.2.3 Generally Prohibited Conduct- Generating or spreading misinformation or disproven claims.- Impersonating real individuals, brands, or organizations.- Infringing intellectual property, publicity, or privacy rights.- Deceiving users by passing AI-generated content off as human-created.- Bulk-generating spam or deceptive commercial content.- Jailbreak prompts designed to bypass Platform safety measures.3. Content ModerationWe operate a multi-layer moderation system: automated review, human review, and periodic audits.3.1 Automated Review (real-time)All requests and generated content are scanned in real time by safety filters.- Input filtering: keyword, semantic, and intent detection on prompts.- Output filtering: real-time safety assessment of generated content.- Usage pattern analysis: monitoring for anomalous API usage.- Safety model used: [e.g., OpenAI Moderation API, or a proprietary model].3.2 Human Review- Auto-flagged content enters a human review queue.- User-reported content is reviewed manually by the trust & safety team.- Review team: [e.g., 24/7 on-call moderation team].- Reviewers complete content-safety training and sign NDAs.3.3 Periodic Audits- Random content sampling performed [specify frequency].- Regular evaluation and improvement of classifier accuracy.- [Whether independent third-party audits are conducted, if applicable.]3.4 Content Severity Levels| Level | Description | Review | Action ||---|---|---|---|| L1 — Critical | Child safety, terrorism | Auto-block + human confirmation | Immediate ban + report || L2 — High | Severe violations | Auto-flag + priority review | Remove + suspend account || L3 — Medium | General violations | Human review | Warning + removal || L4 — Low | Minor violations | Report-triggered | Warning + edit required |4. Reporting4.1 When to Report- Content or behavior violating this Policy's content standards.- Critical violations involving child safety or terrorism.- AI impersonation or infringement of your own rights.- [Other situations the Platform deems reportable.]4.2 Reporting Principles- Confidentiality: reporter identity is strictly protected.- Impartiality: all reports receive independent, objective review.- Anti-abuse: malicious or false reports are recorded and actioned.5. Reporting ChannelsRetain the options that apply:- Dedicated report email: [report@yourplatform.com]- Online report form: [https://yourplatform.com/report]- In-product report button: [e.g., Content page → ··· → Report]- Customer support: [live chat or support email]- Emergency contact (critical): [hotline or instant-messaging channel, if applicable]6. Response & Processing Timelines| Violation Type | Initial Response | Resolution ||---|---|---|| L1 — Critical | Within 2 hours | Within 24 hours || L2 — High | Within 24 hours | Within 3 business days || L3 — Medium | Within 3 business days | Within 7 business days || L4 — Low | Within 5 business days | Within 15 business days |If extended processing is required, we will notify the reporter proactively with reasons. Processing workflow: received (auto-acknowledge) → triage (classify L1–L4) → review (auto + human) → decision → enforcement → notification.7. Enforcement Actions| Action | Measure | Applies to ||---|---|---|| 1. Warning | Written notice requiring removal or edit | First-time minor violation || 2. Content removal | Violating content taken down | Confirmed L3–L4 || 3. Feature restriction | Reduced quota or model access | Repeat minor or single L3 || 4. Account suspension | Temporary suspension pending review | L2 or cumulative violations || 5. Permanent ban | Permanent termination of access | L1 or repeat L2 || 6. Legal action | Report to authorities; cooperate with investigation | Criminal-level L1 |8. AppealsYou may appeal an enforcement decision within [specify days] of receiving notice:- Appeal channel: [appeal@yourplatform.com]- Timeline: [e.g., reviewed within 10 business days].- Review method: reviewed by a reviewer independent of the original decision.9. Transparency & Policy Updates- We publish a content-safety transparency report every [specify period].- Material changes are notified at least [e.g., 15] days in advance.- Continued use of the Platform constitutes acceptance of updates.10. Contact Us- Safety Email: [safety@yourplatform.com]- Report Email: [report@yourplatform.com]- Appeal Email: [appeal@yourplatform.com]- Company Name: [full legal name]- Address: [registered address]This Policy is for general reference only and does not constitute legal advice. We recommend having it reviewed by a qualified AI-compliance attorney before publication.
Each section below shows its tier (🔴 load-bearing · 🟡 strongly recommended · ⚪ optional), a one-line summary, and the matching template snippet so you can read the guidance and copy that block together.
🔴 load-bearingNames every content type you generate. A code assistant doesn’t need the image/audio lines; an image generator does. Prompts count as content too — keep that line.
Section 1
1. ScopeThis Policy applies to all content types, including:- Text: AI-generated text, dialogue, code, articles, etc.- Images: AI-generated or edited images and illustrations.- Audio / Video (if applicable): AI-synthesized voice, music, and video.- User Inputs (Prompts): prompts and instructions submitted by users.- [Other content types supported by the Platform, if applicable.]
🔴 load-bearingThe 2.1 zero-tolerance list is the floor, not a menu — keep all of it and add local-law items. 2.2 is “allowed with conditions” (age-gating, context, local rules); if you have no age verification, delete the adult-content line. 2.3’s “passing AI off as human” and “jailbreak” lines are AI-specific and matter for review.
Section 2
2. Content Standards & Prohibited Categories2.1 Absolutely Prohibited — Zero ToleranceThe following will be actioned immediately and may be reported to authorities:- Child sexual abuse material (CSAM) or any sexual content involving minors.- Content that promotes or facilitates terrorism, extremism, or mass violence.- Technical instructions for weapons of mass destruction.- Content inciting genocide or ethnic hatred.- Non-consensual deepfake sexual content targeting real individuals.- [Add categories based on applicable local regulations.]2.2 Restricted — Subject to Review- Adult content: only permitted on age-verified, compliant platforms.- Violent content: must have a clear creative or educational context.- Politically sensitive content: content that may be misleading.- Medical / legal / financial advice: must comply with local regulations.2.3 Generally Prohibited Conduct- Generating or spreading misinformation or disproven claims.- Impersonating real individuals, brands, or organizations.- Infringing intellectual property, publicity, or privacy rights.- Deceiving users by passing AI-generated content off as human-created.- Bulk-generating spam or deceptive commercial content.- Jailbreak prompts designed to bypass Platform safety measures.
🔴 load-bearingHow you actually catch violations — review expects a real moderation step, not a promise. Name a real tool in 3.1 ([OpenAI Moderation API] or your own); a step on inputs and outputs is the single thing review most wants to see. The L1–L4 tiers in 3.4 drive response times and enforcement.
Section 3
3. Content ModerationWe operate a multi-layer moderation system: automated review, human review, and periodic audits.3.1 Automated Review (real-time)All requests and generated content are scanned in real time by safety filters.- Input filtering: keyword, semantic, and intent detection on prompts.- Output filtering: real-time safety assessment of generated content.- Usage pattern analysis: monitoring for anomalous API usage.- Safety model used: [e.g., OpenAI Moderation API, or a proprietary model].3.2 Human Review- Auto-flagged content enters a human review queue.- User-reported content is reviewed manually by the trust & safety team.- Review team: [e.g., 24/7 on-call moderation team].- Reviewers complete content-safety training and sign NDAs.3.3 Periodic Audits- Random content sampling performed [specify frequency].- Regular evaluation and improvement of classifier accuracy.- [Whether independent third-party audits are conducted, if applicable.]3.4 Content Severity Levels| Level | Description | Review | Action ||---|---|---|---|| L1 — Critical | Child safety, terrorism | Auto-block + human confirmation | Immediate ban + report || L2 — High | Severe violations | Auto-flag + priority review | Remove + suspend account || L3 — Medium | General violations | Human review | Warning + removal || L4 — Low | Minor violations | Report-triggered | Warning + edit required |
🔴 load-bearingWhen and how users flag content, plus confidentiality and anti-abuse. The “independent, objective review” and “false reports actioned” lines keep the process credible.
Section 4
4. Reporting4.1 When to Report- Content or behavior violating this Policy's content standards.- Critical violations involving child safety or terrorism.- AI impersonation or infringement of your own rights.- [Other situations the Platform deems reportable.]4.2 Reporting Principles- Confidentiality: reporter identity is strictly protected.- Impartiality: all reports receive independent, objective review.- Anti-abuse: malicious or false reports are recorded and actioned.
🟡 strongly recommendedResponse/resolution windows by severity. Be realistic — you’ll be held to them. Keep L1 fast (hours, not days); child-safety and terrorism content can’t sit in a queue.
Section 6
6. Response & Processing Timelines| Violation Type | Initial Response | Resolution ||---|---|---|| L1 — Critical | Within 2 hours | Within 24 hours || L2 — High | Within 24 hours | Within 3 business days || L3 — Medium | Within 3 business days | Within 7 business days || L4 — Low | Within 5 business days | Within 15 business days |If extended processing is required, we will notify the reporter proactively with reasons. Processing workflow: received (auto-acknowledge) → triage (classify L1–L4) → review (auto + human) → decision → enforcement → notification.
🔴 load-bearingThe escalation ladder from warning to permanent ban to legal referral. List only actions you can actually take — if you can’t suspend at the account level, describe what you can do instead.
Section 7
7. Enforcement Actions| Action | Measure | Applies to ||---|---|---|| 1. Warning | Written notice requiring removal or edit | First-time minor violation || 2. Content removal | Violating content taken down | Confirmed L3–L4 || 3. Feature restriction | Reduced quota or model access | Repeat minor or single L3 || 4. Account suspension | Temporary suspension pending review | L2 or cumulative violations || 5. Permanent ban | Permanent termination of access | L1 or repeat L2 || 6. Legal action | Report to authorities; cooperate with investigation | Criminal-level L1 |
🟡 strongly recommendedA second look by someone uninvolved in the original decision. The “independent reviewer” line is the point — appeals reviewed by the same person who made the call aren’t appeals.
Section 8
8. AppealsYou may appeal an enforcement decision within [specify days] of receiving notice:- Appeal channel: [appeal@yourplatform.com]- Timeline: [e.g., reviewed within 10 business days].- Review method: reviewed by a reviewer independent of the original decision.
⚪ optionalOptional reporting cadence plus a change-notice window. The transparency report is optional; the change-notice window is good practice.
Section 9
9. Transparency & Policy Updates- We publish a content-safety transparency report every [specify period].- Material changes are notified at least [e.g., 15] days in advance.- Continued use of the Platform constitutes acceptance of updates.
Publish the AUP at a stable URL and link it from your product (footer, signup, and the report button). Reference it from Section 10 of your Terms of Service so the two documents point at each other.
