Platform Features

Phase 1.5: Pagination, Search, and AI Name Generation

Core infrastructure for scaling to 10,000+ content items with under 200ms response times.

Core Features

• Keyset Pagination - Cursor-based pagination replacing offset/limit pattern
• Fuzzy Search - PostgreSQL pg_trgm for typo-tolerant search
• AI Name Generation - Culture-specific fantasy names with caching
• Performance Monitoring - P95 latency tracking for Wave 2 triggers

Keyset Pagination

Pattern

// Cursor format: {sort_field}:{id} encoded as HMAC-SHA256 signed base64
interface PaginationParams {
  cursor: string | null;
  limit: number; // 10-100, default 20
  sort: {
    field: 'name' | 'created_at' | 'updated_at';
    direction: 'asc' | 'desc';
  };
}

// Response includes next cursor for stable pagination
interface PaginatedResponse<T> {
  items: T[];
  pagination: {
    nextCursor: string | null;
    hasMore: boolean;
    total: number;
  };
}

Security

Cursor Security: HMAC-SHA256 signature prevents tampering, uses CURSOR_SECRET from env. Invalid cursors return 400 error.

Fuzzy Search

Implementation

• PostgreSQL pg_trgm extension: CREATE EXTENSION IF NOT EXISTS pg_trgm;
• GIN index: CREATE INDEX idx_content_name_trgm ON content USING GIN (name gin_trgm_ops);
• Similarity threshold: 0.3 (catches "aldrik" → "aldric")
• Search fields with weights:
  • name: 1.0
  • description: 0.5
  • tags: 0.3
  • setting: 0.2
• Returns sorted by similarity + created_at

AI Name Generation

Configuration

• 8 fantasy cultures: human, elvish, dwarven, orcish, halfling, gnomish, tiefling, dragonborn
• Model: Claude Haiku for speed (under 2s p95)
• Cache: 1-hour TTL
• Validation: SRD 5.2 + Tolkien blocklist (250+ terms)
• Optional context: biome (forest/mountain/desert), tone (mysterious/cheerful/grim)
• Cache key: names:{nameType}:{culture}:{biome}:{tone}

Rate Limits

Performance Monitoring

Tracked Metrics

• pagination_query
• search_query
• name_generation
• pdf_export
• campaign_list

Monitoring Strategy

• P95 latency over 1-hour rolling windows
• Wave 2 triggers:
  • Campaign list P95 >500ms
  • PDF timeout >5%
• In-memory store (use Redis for production)

Wave 1 vs Wave 2 Strategy

Wave 2 triggered by production metrics, not premature optimization.

API Endpoints

POST /api/content/paginate   // Keyset pagination for content library
POST /api/campaigns/paginate // Keyset pagination for campaigns
POST /api/search             // Unified fuzzy search across all content types
POST /api/generate/names     // AI name generation with caching

Database Requirements

-- Enable pg_trgm extension
CREATE EXTENSION IF NOT EXISTS pg_trgm;

-- GIN indexes for fuzzy search
CREATE INDEX idx_content_name_trgm ON content USING GIN (name gin_trgm_ops);

-- Composite indexes for keyset pagination
CREATE INDEX idx_content_name_id ON content (name, id);
CREATE INDEX idx_campaigns_name_id ON campaigns (name, id);

Critical Patterns

• Cursor Validation: ALWAYS verify HMAC signature before parsing cursor
• Cache Warming: Name generator auto-warms cache on first request
• Fuzzy Threshold: 0.3 similarity balances precision vs recall (tested with user data)
• Performance Tracking: Call performanceTracker.track() for all Wave 1 features

Common Gotchas

❌ NEVER:

• Use offset/limit for large tables (O(n) performance)
• Skip cursor signature validation (security hole)
• Use fuzzy search without GIN indexes (table scan on large datasets)
• Cache AI responses without TTL (stale data + memory leak)

✅ ALWAYS:

• Use keyset pagination for >1000 rows
• Check pg_trgm extension enabled before fuzzy search
• Include performance monitoring for new endpoints
• Validate Tolkien names in blocklist (legal risk)

Integration Points

• NameSuggestionDropdown component for forms (NPC, campaign, location, faction)
• useNameGenerator hook for state management
• Auto-generates on dropdown open (UX optimization)

Testing Strategy

• Unit: 80+ tests for culture prompts, cache, SRD/Tolkien validation
• Integration: Full API workflows with mocked AI
• Performance: Verify under 100ms pagination, under 200ms search with 10k rows

Additional Resources

See ConnectionGuide.txt for complete API documentation and performance characteristics.