AI Knowledge Ingestion Plan

To give Copilot full knowledge of TDCFlow, product documentation must become a first-class knowledge source alongside workspace data.

Objective

Copilot should answer product questions using TDCFlow docs instead of generic PSA assumptions.

Examples:

• "How do I enable credit notes?"
• "What can clients see in the portal?"
• "Why can't I approve this request?"
• "How does invoice reminder recipient selection work?"
• "What is the difference between semantic search and global search?"

Knowledge Collections

Use separate knowledge collections so Copilot can tell sources apart.

Collection	Contents
Product docs	End-user docs, workflow guides, troubleshooting, module references.
AI system docs	Architecture, agents, skills, proposals, safety, ingestion plan.
Workspace records	Projects, tasks, accounts, invoices, contacts, resources, expenses, credit notes.
Conversation memory	User's current and previous chat context.

Product Doc Indexing

Index each docs page by sections, not only whole files.

Each indexed chunk should include:

• Collection name.
• Docs path.
• Page title.
• Section heading.
• Content chunk.
• Related module.
• Audience.
• Last indexed timestamp.

Recommended chunking:

• One chunk per major heading.
• Keep tables with their heading.
• Keep workflow steps together.
• Keep troubleshooting problem and resolution together.

Retrieval Rules

When a user asks a product/how-to question:

1. Search product docs.
2. Search AI system docs only if the question is about Copilot internals.
3. Use workspace tools only if the user asks about current data.
4. Answer with product-native steps.
5. Offer navigation when relevant.

When a user asks a data question:

1. Use structured tools first.
2. Use semantic workspace memory for discovery.
3. Use product docs only to explain behavior or missing access.

Ingestion Events

Docs ingestion should run when:

• Docs files change.
• A production build or deploy occurs.
• An admin manually reindexes knowledge.

Workspace ingestion should run when:

• Supported records are created.
• Supported records are updated.
• Supported records are archived/restored.
• Supported records are deleted.
• Backfill is requested.

Quality Checklist

Before indexing docs, each page should have:

• Clear title and description.
• Product-specific terminology.
• "When to use" or "What it does" sections.
• Steps for workflows.
• Troubleshooting where relevant.
• Related docs.
• No stale links.
• No API routes or implementation details in end-user pages.

Copilot Prompt Policy

The system prompt should tell Copilot:

• Product docs are authoritative for TDCFlow behavior.
• Workspace tools are authoritative for current customer data.
• Conversation history is useful but lower priority.
• Do not answer from generic PSA knowledge when docs are available.
• If docs and data disagree, say what source was used.

Initial Implementation Steps

1. Add a product_knowledge embedding type or separate docs knowledge table.
2. Build a docs crawler over apps/web/content/docs.
3. Parse frontmatter, headings, tables, and content.
4. Chunk pages by heading.
5. Generate embeddings for chunks.
6. Add a searchProductKnowledge tool.
7. Add tool usage rules for how-to and troubleshooting prompts.
8. Add a reindex script and admin-only reindex endpoint.
9. Add tests for common product questions.

Current Runtime Hooks

Product docs can be indexed into semantic memory with:

pnpm --filter @psa/api ai:knowledge:backfill -- --org-id=<org-id>

For all organizations, use:

pnpm --filter @psa/api ai:knowledge:backfill -- --all

Each docs section is indexed as product_knowledge, with metadata for title, docs path, heading, description, and source file. Copilot can retrieve it with the product knowledge search tool for how-to, settings, workflow, troubleshooting, and Copilot behavior questions.

Evaluation Questions

Use these questions to test product knowledge:

• "Where do I enable credit notes?"
• "How do I create a project from a template?"
• "Why is an approval action unavailable?"
• "What can a client see in the portal?"
• "How do I send an invoice reminder?"
• "Why is Copilot showing a proposal?"
• "What records are included in semantic search?"

Related Docs

• AI Memory and Embeddings
• AI Tooling and Context
• Workflow Guides
• Troubleshooting