Course contentsShow
← AI Engineering
Lesson 861 of 1,88621. Human Evaluation and FeedbackFree lesson

Feedback Data Storage and Schema Design

Structure databases to store feedback with context: prompts, responses, metadata, and user identifiers for analysis.

Feedback Data Storage and Schema Design

You'll learn how to structure a database to store user feedback alongside the full context needed to analyze and improve your AI system over time.

The Core Idea

When you collect feedback (thumbs up/down, ratings, corrections), that feedback is meaningless without context. You need to remember: What prompt was sent? What did the model generate? Which model version? Who gave the feedback? Think of it like reviewing a restaurant—the star rating alone doesn't help; you need to know what dish, which chef, and when.

A well-designed feedback schema captures:

  • The prompt (user input, system context)
  • The response (model output, possibly multiple candidates)
  • Metadata (model ID, version, temperature, timestamp)
  • User identifiers (anonymized user ID, session ID)
  • The feedback itself (rating, category, free-text comment)
  • Contextual flags (A/B test group, feature flags active)

Example Schema Structure

feedback_events:
  - id (primary key)
  - timestamp
  - user_id (anonymized)
  - session_id
  - prompt_text
  - response_text
  - model_name
  - model_version
  - temperature
  - feedback_type (rating, binary, correction)
  - feedback_value (1-5, true/false, corrected_text)
  - metadata_json (flexible additional context)

Store enough detail to replay the interaction months later. If you discover a pattern of negative feedback, you want to trace it back: Was it a specific prompt pattern? A model update? A particular user segment?

Key Takeaway: Design your feedback schema to capture the complete interaction context—prompts, responses, model metadata, and user identifiers—so you can meaningfully analyze feedback and trace issues back to root causes.