Overview

Sources

Sources are the knowledge foundation of Lattice. Add documents, URLs, and repositories to build a searchable knowledge base that grounds AI responses.

Supported Source Types

Type	Description	Processing
pdf	PDF documents	Text extraction, chunking, embeddings
url	Web pages	HTML parsing, content extraction
github	GitHub repositories	README, code files, documentation
youtube	YouTube videos	Transcript extraction
google_docs	Google Documents	API-based content retrieval
markdown	Markdown files	Direct parsing
text	Plain text	Direct chunking
artifact	Promoted artifacts	Converted from Studio

Auto-Classification

Sources are automatically categorized based on content analysis:

• requirements - SLAs, PRDs, RFPs, specifications
• research - Academic papers, blog posts, transcripts
• vendor - Pricing pages, model cards, API documentation
• architecture - System designs, diagrams, technical specs
• benchmarks - Leaderboards, evaluations, comparisons
• tutorial - Guides, tutorials, learning content

Processing Pipeline

1. Ingestion: Content fetched/uploaded
2. Extraction: Text extracted from format
3. Chunking: Split into semantic chunks (~1000 tokens)
4. Embedding: Vector embeddings generated (if enabled)
5. Indexing: Full-text search index updated