Feedback Data Storage and Schema Design
You'll learn how to structure a database to store user feedback alongside the full context needed to analyze and improve your AI system over time.
The Core Idea
When you collect feedback (thumbs up/down, ratings, corrections), that feedback is meaningless without context. You need to remember: What prompt was sent? What did the model generate? Which model version? Who gave the feedback? Think of it like reviewing a restaurant—the star rating alone doesn't help; you need to know what dish, which chef, and when.
A well-designed feedback schema captures:
- The prompt (user input, system context)
- The response (model output, possibly multiple candidates)
- Metadata (model ID, version, temperature, timestamp)
- User identifiers (anonymized user ID, session ID)
- The feedback itself (rating, category, free-text comment)
- Contextual flags (A/B test group, feature flags active)
Example Schema Structure
feedback_events:
- id (primary key)
- timestamp
- user_id (anonymized)
- session_id
- prompt_text
- response_text
- model_name
- model_version
- temperature
- feedback_type (rating, binary, correction)
- feedback_value (1-5, true/false, corrected_text)
- metadata_json (flexible additional context)
Store enough detail to replay the interaction months later. If you discover a pattern of negative feedback, you want to trace it back: Was it a specific prompt pattern? A model update? A particular user segment?
Key Takeaway: Design your feedback schema to capture the complete interaction context—prompts, responses, model metadata, and user identifiers—so you can meaningfully analyze feedback and trace issues back to root causes.