# Memory Quality System Guide

> **Version**: 8.50.0
> **Status**: Production Ready
> **Feature**: Memento-Inspired Quality System (Issue #260)
> **Evaluation**: [Quality System Evaluation Report](https://github.com/doobidoo/mcp-memory-service/wiki/Memory-Quality-System-Evaluation)
>
> **⚠️ IMPORTANT (v8.50.0)**: Fallback mode is **not recommended** due to MS-MARCO architectural limitations (query-document relevance model cannot assess absolute quality). See [Recommended Configuration](#recommended-configuration-v850) for the simpler, more robust approach.

## Recommended Configuration (v8.50.0)

### Primary Recommendation: Implicit Signals Only

**✅ For Technical Corpora (RECOMMENDED)**

If your memory base contains:
- Technical fragments, file paths, references
- Code snippets, CLI output, configuration
- Task lists, tickets, abbreviations
- Short notes and checklists

```bash
# Disable AI quality scoring (avoids prose bias)
export MCP_QUALITY_AI_PROVIDER=none

# Quality based on implicit signals only
export MCP_QUALITY_SYSTEM_ENABLED=true
export MCP_QUALITY_BOOST_ENABLED=false
```

**Why Implicit Signals Work Better**:
- ✅ **Access patterns = true quality** - Frequently-used memories are valuable
- ✅ **No prose bias** - Technical fragments treated fairly
- ✅ **Self-learning** - Quality improves based on actual usage
- ✅ **Simpler** - No model loading, zero latency
- ✅ **Works offline** - No external dependencies

**Implicit Signal Components**:
```python
quality_score = (
    access_frequency × 0.40 +  # How often retrieved
    recency × 0.30 +           # When last accessed
    retrieval_ranking × 0.30   # Average position in results
)
```

**Real-World Test Results** (v8.50.0):
- DeBERTa average score: 0.209 (median: 0.165)
- Only 4% of memories scored ≥ 0.6 (DeBERTa's "good" threshold)
- Manual review: 72% classified as "low quality" were valid technical references
- Examples misclassified: file paths, Terraform configs, technical abbreviations

**Conclusion**: DeBERTa trained on Wikipedia/news systematically under-scores technical content.

---

### Alternative: DeBERTa for Prose-Heavy Corpora

**For Long-Form Documentation Only**

If your memory base contains primarily:
- Blog posts, articles, tutorials
- Narrative meeting notes
- Long-form documentation with prose paragraphs

```bash
# AI quality scoring with DeBERTa
export MCP_QUALITY_AI_PROVIDER=local
export MCP_QUALITY_LOCAL_MODEL=nvidia-quality-classifier-deberta

# Lower threshold for technical tolerance
export MCP_QUALITY_DEBERTA_THRESHOLD=0.4  # Or 0.3 for more leniency

# Combine AI + implicit signals
export MCP_QUALITY_BOOST_ENABLED=true
export MCP_QUALITY_BOOST_WEIGHT=0.3
```

**When to Use AI Scoring**:
| Content Type | Use AI? | Reason |
|--------------|---------|--------|
| Narrative documentation | ✅ Yes | DeBERTa trained for prose |
| Blog posts, articles | ✅ Yes | Complete paragraphs |
| Meeting notes (narrative) | ✅ Yes | Story-like structure |
| Technical fragments | ❌ No | Prose bias under-scores |
| File paths, references | ❌ No | Not prose |
| Code snippets, CLI output | ❌ No | Technical, not narrative |
| Task lists, tickets | ❌ No | Fragmented |

**Expected Distribution** (prose-heavy corpus with 0.4 threshold):
- Low quality (< 0.4): ~20% filtered
- Medium (0.4-0.6): ~40% accepted
- Good (0.6-0.8): ~30% accepted
- Excellent (≥ 0.8): ~10% accepted

## Overview

The **Memory Quality System** transforms MCP Memory Service from static storage to a learning memory system. It automatically evaluates memory quality using AI-driven scoring and uses these scores to improve retrieval precision, consolidation efficiency, and overall system intelligence.

### Key Benefits

- ✅ **40-70% improvement** in retrieval precision (top-5 useful rate: 50% → 70-85%)
- ✅ **Zero cost** with local SLM (privacy-preserving, offline-capable)
- ✅ **Smarter consolidation** - Preserve high-quality memories longer
- ✅ **Quality-boosted search** - Prioritize best memories in results
- ✅ **Automatic learning** - System improves from usage patterns

## How It Works

### Multi-Tier AI Scoring (Local-First)

The system evaluates memory quality (0.0-1.0 score) using a multi-tier fallback chain:

| Tier | Provider | Cost | Latency | Privacy | Default |
|------|----------|------|---------|---------|---------|
| **1** | **Local SLM (ONNX)** | **$0** | **50-100ms** | ✅ Full | ✅ Yes |
| 2 | Groq API | ~$0.30/mo | 900ms | ❌ External | ❌ Opt-in |
| 3 | Gemini API | ~$0.40/mo | 2000ms | ❌ External | ❌ Opt-in |
| 4 | Implicit Signals | $0 | 10ms | ✅ Full | Fallback |

**Default setup**: Local SLM only (zero cost, full privacy, no external API calls)

### Quality Score Components

```
quality_score = (
    local_slm_score × 0.50 +      # Cross-encoder evaluation
    implicit_signals × 0.50        # Usage patterns
)

implicit_signals = (
    access_frequency × 0.40 +      # How often retrieved
    recency × 0.30 +              # When last accessed
    retrieval_ranking × 0.30      # Average position in results
)
```

## 🎯 Local SLM Models (Tier 1 - Primary)

**Update (v8.49.0)**: DeBERTa quality classifier is now the default model, eliminating self-matching bias and providing absolute quality assessment.

### Model Options

The quality system supports multiple ONNX models for different use cases:

#### **NVIDIA DeBERTa Quality Classifier** (Default, Recommended)

**Model**: `nvidia-quality-classifier-deberta` (450MB)
**Architecture**: 3-class text classifier (Low/Medium/High quality)
**Type**: Absolute quality assessment (query-independent)

**Key Features**:
- ✅ **Eliminates self-matching bias** - No query needed, evaluates content directly
- ✅ **Absolute quality scores** - Assesses inherent memory quality
- ✅ **Uniform distribution** - More realistic quality spread (mean: 0.60-0.70)
- ✅ **Fewer false positives** - <5% perfect 1.0 scores (vs 20% with MS-MARCO)

**Performance**:
- CPU: 80-150ms per evaluation
- GPU (CUDA/MPS/DirectML): 20-40ms per evaluation
- Model size: 450MB (downloaded once, cached locally)

**Scoring Process**:
1. Tokenize memory content (no query needed)
2. Run DeBERTa inference (local, private)
3. Get 3-class probabilities: P(low), P(medium), P(high)
4. Calculate weighted score: `0.0×P(low) + 0.5×P(medium) + 1.0×P(high)`

**Usage**:
```bash
# Default model (no configuration needed)
export MCP_QUALITY_LOCAL_MODEL=nvidia-quality-classifier-deberta  # Already default in v8.49.0+

# Quality boost works better with DeBERTa (more accurate scores)
export MCP_QUALITY_BOOST_ENABLED=true
export MCP_QUALITY_BOOST_WEIGHT=0.3
```

**Quality Metrics** (Expected vs MS-MARCO):
| Metric | MS-MARCO (Old) | DeBERTa (New) | Improvement |
|--------|---------------|---------------|-------------|
| Mean Score | 0.469 | 0.60-0.70 | +28-49% |
| Perfect 1.0 Scores | 15-20% | <5% | -60% false positives |
| High Quality (≥0.7) | 32.2% | 40-50% | More accurate |
| Distribution | Bimodal | Uniform | Better spread |

#### **MS-MARCO Cross-Encoder** (Legacy, Backward Compatible)

**Model**: `ms-marco-MiniLM-L-6-v2` (23MB)
**Architecture**: Cross-encoder (query-document relevance ranking)
**Type**: Relative relevance assessment (query-dependent)

**Use Cases**:
- Legacy systems requiring MS-MARCO compatibility
- Relative ranking within search results
- When disk space is extremely limited (<100MB)

**Known Limitations**:
- ⚠️ **Self-matching bias** - Tag-based queries match content (~25% false positives)
- ⚠️ **Bimodal distribution** - Scores cluster at extremes (0.0 or 1.0)
- ⚠️ **Requires meaningful queries** - Empty queries return 0.0

**Performance**:
- CPU: 50-100ms per evaluation
- GPU (CUDA/MPS/DirectML): 10-20ms per evaluation
- Model size: 23MB

**Usage** (opt-in for legacy compatibility):
```bash
# Override to use MS-MARCO (not recommended)
export MCP_QUALITY_LOCAL_MODEL=ms-marco-MiniLM-L-6-v2
```

#### **Fallback Quality Scoring** (Best of Both Worlds) 🆕 v8.50.0

**Model**: DeBERTa + MS-MARCO hybrid approach
**Architecture**: Threshold-based fallback for optimal quality assessment
**Type**: Absolute quality with technical content rescue

**Problem Solved**:
DeBERTa has systematic bias toward prose/narrative content over technical documentation:
- **Prose content**: 0.78-0.92 scores (excellent)
- **Technical content**: 0.48-0.60 scores (undervalued)
- **Impact**: 95% of technical memories undervalued in technical databases

**Solution - Threshold-Based Fallback**:
```python
# Always score with DeBERTa first
deberta_score = score_with_deberta(content)

# If DeBERTa confident (high score), use it
if deberta_score >= DEBERTA_THRESHOLD:
    return deberta_score  # MS-MARCO not consulted (fast path)

# DeBERTa scored low - check if MS-MARCO thinks it's good (technical content)
ms_marco_score = score_with_ms_marco(content)
if ms_marco_score >= MS_MARCO_THRESHOLD:
    return ms_marco_score  # Rescue technical content

# Both agree it's low quality
return deberta_score
```

**Key Features**:
- ✅ **Signal Preservation** - No averaging dilution, uses strongest signal
- ✅ **DeBERTa Primary** - Eliminates self-matching bias for prose
- ✅ **MS-MARCO Rescue** - Catches undervalued technical content
- ✅ **Performance Optimized** - MS-MARCO only runs when DeBERTa scores low (~60% of memories)
- ✅ **Simple Configuration** - Just thresholds, no complex weights

**Expected Results**:
| Content Type | DeBERTa-Only | Fallback Mode | Improvement |
|--------------|--------------|---------------|-------------|
| **Technical content** | 0.48-0.60 | 0.70-0.80 | +45-65% ✅ |
| **Prose content** | 0.78-0.92 | 0.78-0.92 | No change ✅ |
| **High quality (≥0.7)** | 0.4% (17 memories) | 20-30% (800-1200) | 50-75x ✅ |

**Performance**:
| Scenario | DeBERTa-Only | Fallback | Time |
|----------|--------------|----------|------|
| **High prose** (~40% of memories) | DeBERTa only | DeBERTa only | 115ms ⚡ |
| **Low technical** (~35% of memories) | DeBERTa only | DeBERTa + MS-MARCO rescue | 155ms |
| **Low garbage** (~25% of memories) | DeBERTa only | DeBERTa + MS-MARCO both low | 155ms |
| **Average** | 115ms | 139ms | +21% acceptable overhead |

**Configuration**:
```bash
# Enable fallback mode (requires both models)
export MCP_QUALITY_FALLBACK_ENABLED=true
export MCP_QUALITY_LOCAL_MODEL="nvidia-quality-classifier-deberta,ms-marco-MiniLM-L-6-v2"

# Thresholds (recommended defaults)
export MCP_QUALITY_DEBERTA_THRESHOLD=0.6   # DeBERTa confidence threshold
export MCP_QUALITY_MSMARCO_THRESHOLD=0.7   # MS-MARCO rescue threshold

# Quality boost works well with fallback
export MCP_QUALITY_BOOST_ENABLED=true
export MCP_QUALITY_BOOST_WEIGHT=0.3
```

**Configuration Tuning**:
```bash
# Technical-heavy systems (more MS-MARCO rescues)
export MCP_QUALITY_DEBERTA_THRESHOLD=0.5   # Lower threshold → more rescues
export MCP_QUALITY_MSMARCO_THRESHOLD=0.6   # Lower bar for technical content

# Quality-focused (stricter standards)
export MCP_QUALITY_DEBERTA_THRESHOLD=0.7   # Higher threshold → DeBERTa must be very confident
export MCP_QUALITY_MSMARCO_THRESHOLD=0.8   # Higher bar for rescue

# Balanced (recommended default)
export MCP_QUALITY_DEBERTA_THRESHOLD=0.6
export MCP_QUALITY_MSMARCO_THRESHOLD=0.7
```

**Decision Tracking**:
Fallback mode stores detailed decision metadata:
```json
{
  "quality_score": 0.78,
  "quality_provider": "fallback_deberta-msmarco",
  "quality_components": {
    "decision": "ms_marco_rescue",
    "deberta_score": 0.52,
    "ms_marco_score": 0.78,
    "final_score": 0.78
  }
}
```

**Decision Types**:
- `deberta_confident`: DeBERTa ≥ threshold, used DeBERTa (fast path, ~40%)
- `ms_marco_rescue`: DeBERTa low, MS-MARCO ≥ threshold, used MS-MARCO (~35%)
- `both_low`: Both below thresholds, used DeBERTa (~25%)
- `deberta_only`: MS-MARCO not loaded, DeBERTa only
- `ms_marco_failed`: MS-MARCO error, used DeBERTa

**Re-scoring Existing Memories**:
```bash
# Dry-run (preview changes)
python scripts/quality/rescore_fallback.py

# Execute re-scoring
python scripts/quality/rescore_fallback.py --execute

# Custom thresholds
python scripts/quality/rescore_fallback.py --execute \
  --deberta-threshold 0.5 \
  --msmarco-threshold 0.6

# Expected time: 3-5 minutes for 4,000-5,000 memories
# Creates backup automatically in ~/backups/mcp-memory-service/
```

**When to Use Fallback Mode**:
| Use Case | Recommended Mode |
|----------|-----------------|
| **Technical documentation** | ✅ Fallback (rescues undervalued content) |
| **Mixed content** (code + prose) | ✅ Fallback (best of both) |
| **Prose-only** (creative writing) | DeBERTa-only (faster, same quality) |
| **Strict quality** (academic) | DeBERTa-only (no rescue for low scores) |
| **Legacy compatibility** | MS-MARCO-only (if required) |

### GPU Acceleration (Automatic)

Both models support automatic GPU detection:
- **CUDA** (NVIDIA GPUs)
- **CoreML/MPS** (Apple Silicon M1/M2/M3)
- **DirectML** (Windows DirectX)
- **ROCm** (AMD GPUs on Linux)
- **CPU** (fallback, always works)

```bash
# GPU detection is automatic
export MCP_QUALITY_LOCAL_DEVICE=auto  # Default (auto-detects best device)

# Or force specific device
export MCP_QUALITY_LOCAL_DEVICE=cpu    # Force CPU
export MCP_QUALITY_LOCAL_DEVICE=cuda   # Force CUDA (NVIDIA)
export MCP_QUALITY_LOCAL_DEVICE=mps    # Force MPS (Apple Silicon)
```

### Migration from MS-MARCO to DeBERTa

If you're upgrading from v8.48.x or earlier and want to re-evaluate existing memories:

```bash
# Export DeBERTa model (one-time, downloads 450MB)
python scripts/quality/export_deberta_onnx.py

# Re-evaluate all memories with DeBERTa
python scripts/quality/migrate_to_deberta.py

# Verify new distribution
curl -ks https://127.0.0.1:8000/api/quality/distribution | python3 -m json.tool
```

**Migration preserves**:
- Original MS-MARCO scores (in `quality_migration` metadata)
- Access patterns and timestamps
- User ratings and feedback

**Migration time**: ~10-20 minutes for 4,000-5,000 memories (depends on CPU/GPU)

## Installation & Setup

### 1. Basic Setup (Local SLM Only)

**Zero configuration required** - The quality system works out of the box with local SLM:

```bash
# Install MCP Memory Service (if not already installed)
pip install mcp-memory-service

# Quality system is enabled by default with local SLM
# No API keys needed, no external calls
```

### 2. Optional: Cloud APIs (Opt-In)

If you want cloud-based scoring (Groq or Gemini):

```bash
# Enable Groq API (fast, cheap)
export GROQ_API_KEY="your-groq-api-key"
export MCP_QUALITY_AI_PROVIDER=groq  # or "auto" to try all tiers

# Enable Gemini API (Google)
export GOOGLE_API_KEY="your-gemini-api-key"
export MCP_QUALITY_AI_PROVIDER=gemini
```

### 3. Configuration Options

```bash
# Quality System Core
export MCP_QUALITY_SYSTEM_ENABLED=true         # Default: true
export MCP_QUALITY_AI_PROVIDER=local           # local|groq|gemini|auto|none

# Local SLM Configuration (Tier 1)
export MCP_QUALITY_LOCAL_MODEL=ms-marco-MiniLM-L-6-v2  # Model name
export MCP_QUALITY_LOCAL_DEVICE=auto           # auto|cpu|cuda|mps|directml

# Quality-Boosted Search (Opt-In)
export MCP_QUALITY_BOOST_ENABLED=false         # Default: false (opt-in)
export MCP_QUALITY_BOOST_WEIGHT=0.3            # 0.0-1.0 (30% quality, 70% semantic)

# Quality-Based Retention (Consolidation)
export MCP_QUALITY_RETENTION_HIGH=365          # Days for quality ≥0.7
export MCP_QUALITY_RETENTION_MEDIUM=180        # Days for quality 0.5-0.7
export MCP_QUALITY_RETENTION_LOW_MIN=30        # Min days for quality <0.5
export MCP_QUALITY_RETENTION_LOW_MAX=90        # Max days for quality <0.5
```

## Using the Quality System

### 1. Automatic Quality Scoring

Quality scores are calculated automatically when memories are retrieved:

```bash
# Normal retrieval - quality scoring happens in background
claude /memory-recall "what did I work on yesterday"

# Quality score is updated in metadata (non-blocking)
```

### 2. Manual Rating (Optional)

Override AI scores with manual ratings:

```bash
# Rate a memory (MCP tool)
rate_memory(
    content_hash="abc123...",
    rating=1,  # -1 (bad), 0 (neutral), 1 (good)
    feedback="This was very helpful!"
)

# Manual ratings weighted 60%, AI scores weighted 40%
```

**HTTP API**:
```bash
curl -X POST http://127.0.0.1:8000/api/quality/memories/{hash}/rate \
  -H "Content-Type: application/json" \
  -d '{"rating": 1, "feedback": "Helpful!"}'
```

### 3. Quality-Boosted Search

Enable quality-based reranking for better results:

**Method 1: Global Configuration**
```bash
export MCP_QUALITY_BOOST_ENABLED=true
claude /memory-recall "search query"  # Uses quality boost
```

**Method 2: Per-Query (MCP Tool)**
```bash
# Search with quality boost (MCP tool)
retrieve_with_quality_boost(
    query="search query",
    n_results=10,
    quality_weight=0.3  # 30% quality, 70% semantic
)
```

**Algorithm**:
1. Over-fetch 3× candidates (30 results for top 10)
2. Rerank by: `0.7 × semantic_similarity + 0.3 × quality_score`
3. Return top N results

**Performance**: <100ms total (50ms semantic search + 20ms reranking + 30ms quality scoring)

### 4. View Quality Metrics

**MCP Tool**:
```bash
get_memory_quality(content_hash="abc123...")

# Returns:
# - quality_score: Current composite score (0.0-1.0)
# - quality_provider: Which tier scored it (ONNXRankerModel, etc.)
# - access_count: Number of retrievals
# - last_accessed_at: Last access timestamp
# - ai_scores: Historical AI evaluation scores
# - user_rating: Manual rating if present
```

**HTTP API**:
```bash
curl http://127.0.0.1:8000/api/quality/memories/{hash}
```

### 5. Quality Analytics

**MCP Tool**:
```bash
analyze_quality_distribution(min_quality=0.0, max_quality=1.0)

# Returns:
# - total_memories: Total count
# - high_quality_count: Score ≥0.7
# - medium_quality_count: 0.5 ≤ score < 0.7
# - low_quality_count: Score < 0.5
# - average_score: Mean quality score
# - provider_breakdown: Count by provider
# - top_10_memories: Highest scoring
# - bottom_10_memories: Lowest scoring
```

**Dashboard** (http://127.0.0.1:8000/):
- Quality badges on all memory cards (color-coded by tier)
- Analytics view with distribution charts
- Provider breakdown pie chart
- Top/bottom performers lists

## Quality-Based Memory Management

### 1. Quality-Based Forgetting (Consolidation)

High-quality memories are preserved longer during consolidation:

| Quality Tier | Score Range | Retention Period |
|--------------|-------------|------------------|
| **High** | ≥0.7 | 365 days inactive |
| **Medium** | 0.5-0.7 | 180 days inactive |
| **Low** | <0.5 | 30-90 days inactive (scaled by score) |

**How it works**:
- Weekly consolidation scans inactive memories
- Applies quality-based thresholds
- Archives low-quality memories sooner
- Preserves high-quality memories longer

### 2. Quality-Weighted Decay

High-quality memories decay slower in relevance scoring:

```
decay_multiplier = 1.0 + (quality_score × 0.5)
# High quality (0.9): 1.45× multiplier
# Medium quality (0.5): 1.25× multiplier
# Low quality (0.2): 1.10× multiplier

final_relevance = base_relevance × decay_multiplier
```

**Effect**: High-quality memories stay relevant longer in search results.

## Privacy & Cost

### Privacy Modes

| Mode | Configuration | Privacy | Cost |
|------|---------------|---------|------|
| **Local Only** | `MCP_QUALITY_AI_PROVIDER=local` | ✅ Full (no external calls) | $0 |
| **Hybrid** | `MCP_QUALITY_AI_PROVIDER=auto` | ⚠️ Cloud fallback | ~$0.30/mo |
| **Cloud** | `MCP_QUALITY_AI_PROVIDER=groq` | ❌ External API | ~$0.30/mo |
| **Implicit Only** | `MCP_QUALITY_AI_PROVIDER=none` | ✅ Full (no AI) | $0 |

### Cost Comparison (3500 memories, 100 retrievals/day)

| Provider | Monthly Cost | Notes |
|----------|--------------|-------|
| **Local SLM** | **$0** | Free forever, runs locally |
| Groq (Kimi K2) | ~$0.30-0.50 | Fast, good quality |
| Gemini Flash | ~$0.40-0.60 | Slower, free tier available |
| Implicit Only | $0 | No AI scoring, usage patterns only |

**Recommendation**: Use default local SLM (zero cost, full privacy, fast).

## Performance Benchmarks

| Operation | Latency | Notes |
|-----------|---------|-------|
| **Local SLM Scoring (CPU)** | 50-100ms | Per memory evaluation |
| **Local SLM Scoring (GPU)** | 10-20ms | With CUDA/MPS/DirectML |
| **Quality-Boosted Search** | <100ms | Over-fetch + rerank |
| **Implicit Signals** | <10ms | Always fast |
| **Quality Metadata Update** | <5ms | Storage backend write |

**Target Metrics**:
- Quality calculation overhead: <10ms
- Search latency with boost: <100ms total
- No user-facing blocking (async scoring)

## Troubleshooting

### Local SLM Not Working

**Symptom**: `quality_provider: ImplicitSignalsEvaluator` (should be `ONNXRankerModel`)

**Fixes**:
1. Check ONNX Runtime installed:
   ```bash
   pip install onnxruntime
   ```

2. Check model downloaded:
   ```bash
   ls ~/.cache/mcp_memory/onnx_models/ms-marco-MiniLM-L-6-v2/
   # Should contain: model.onnx, tokenizer.json
   ```

3. Check logs for errors:
   ```bash
   tail -f logs/mcp_memory_service.log | grep quality
   ```

### Quality Scores Always 0.5

**Symptom**: All memories have `quality_score: 0.5` (neutral default)

**Cause**: Quality scoring not triggered yet (memories haven't been retrieved)

**Fix**: Retrieve memories to trigger scoring:
```bash
claude /memory-recall "any search query"
# Quality scoring happens in background after retrieval
```

### GPU Not Detected

**Symptom**: Local SLM uses CPU despite having GPU

**Fixes**:
1. Install GPU-enabled ONNX Runtime:
   ```bash
   # NVIDIA CUDA
   pip install onnxruntime-gpu

   # DirectML (Windows)
   pip install onnxruntime-directml
   ```

2. Force device selection:
   ```bash
   export MCP_QUALITY_LOCAL_DEVICE=cuda  # or mps, directml
   ```

### Quality Boost Not Working

**Symptom**: Search results don't show quality reranking

**Checks**:
1. Verify enabled:
   ```bash
   echo $MCP_QUALITY_BOOST_ENABLED  # Should be "true"
   ```

2. Use explicit MCP tool:
   ```bash
   retrieve_with_quality_boost(query="test", quality_weight=0.5)
   ```

3. Check debug info in results:
   ```python
   result.debug_info['reranked']  # Should be True
   result.debug_info['quality_score']  # Should exist
   ```

## Best Practices

### 1. Start with Defaults (Updated for v8.48.3)

Use local SLM (default) for:
- Zero cost
- Full privacy
- Offline capability
- **Good for relative ranking** (not absolute quality assessment)

**Important**: Local SLM scores should be **combined with implicit signals** (access patterns, recency) for best results. Do not rely on ONNX scores alone.

### 2. Enable Quality Boost Gradually

```bash
# Week 1: Collect quality scores (boost disabled)
export MCP_QUALITY_BOOST_ENABLED=false

# Week 2: Test with low weight
export MCP_QUALITY_BOOST_ENABLED=true
export MCP_QUALITY_BOOST_WEIGHT=0.2  # 20% quality

# Week 3+: Increase if helpful
export MCP_QUALITY_BOOST_WEIGHT=0.3  # 30% quality (recommended)
```

### 3. Monitor Quality Distribution (With Caution)

Check analytics regularly, but interpret with awareness of limitations:
```bash
analyze_quality_distribution()

# Current observed distribution (v8.48.3):
# - High quality (≥0.7): 32.2% (includes ~25% false positives from self-matching)
# - Medium quality (0.5-0.7): 27.4%
# - Low quality (<0.5): 40.4%

# Ideal distribution (future with hybrid scoring):
# - High quality (≥0.7): 20-30% of memories
# - Medium quality (0.5-0.7): 50-60%
# - Low quality (<0.5): 10-20%
```

**Warning**: High scores may not always indicate high quality due to self-matching bias. Validate manually before making retention decisions.

### 4. Manual Rating for Edge Cases

Rate important memories manually:
```bash
# After finding a very helpful memory
rate_memory(content_hash="abc123...", rating=1, feedback="Critical info!")

# After finding unhelpful memory
rate_memory(content_hash="def456...", rating=-1, feedback="Outdated")
```

Manual ratings weighted 60%, AI scores 40%.

### 5. Periodic Review (Updated for v8.48.3)

Monthly checklist:
- [ ] Check quality distribution (analytics dashboard)
- [ ] **Manually validate** top 10 performers (verify genuinely helpful, not just self-matching bias)
- [ ] **Manually validate** bottom 10 before deletion (may be low-quality or just poor matches)
- [ ] Verify provider breakdown (target: 75%+ local SLM)
- [ ] Check average quality score (current: 0.469, target with hybrid: 0.6+)
- [ ] **Review Issue #268** progress on quality improvements

## Advanced Configuration

### Custom Retention Policy

```bash
# Conservative: Preserve longer
export MCP_QUALITY_RETENTION_HIGH=730       # 2 years for high quality
export MCP_QUALITY_RETENTION_MEDIUM=365     # 1 year for medium
export MCP_QUALITY_RETENTION_LOW_MIN=90     # 90 days minimum for low

# Aggressive: Archive sooner
export MCP_QUALITY_RETENTION_HIGH=180       # 6 months for high
export MCP_QUALITY_RETENTION_MEDIUM=90      # 3 months for medium
export MCP_QUALITY_RETENTION_LOW_MIN=14     # 2 weeks minimum for low
```

### Custom Quality Boost Weight

```bash
# Semantic-first (default)
export MCP_QUALITY_BOOST_WEIGHT=0.3  # 30% quality, 70% semantic

# Balanced
export MCP_QUALITY_BOOST_WEIGHT=0.5  # 50% quality, 50% semantic

# Quality-first
export MCP_QUALITY_BOOST_WEIGHT=0.7  # 70% quality, 30% semantic
```

**Recommendation**: Start with 0.3, increase if quality boost improves results.

### Hybrid Cloud Strategy

Use local SLM primarily, cloud APIs as fallback:

```bash
export MCP_QUALITY_AI_PROVIDER=auto  # Try all available tiers
export GROQ_API_KEY="your-key"       # Groq as Tier 2 fallback
```

**Behavior**:
1. Try local SLM (99% success rate)
2. If fails, try Groq API
3. If fails, try Gemini API
4. Ultimate fallback: Implicit signals only

## Success Metrics (Phase 1 Targets)

From Issue #260 and #261 roadmap:

| Metric | Target | Measurement |
|--------|--------|-------------|
| **Retrieval Precision** | >70% useful (top-5) | Up from ~50% baseline |
| **Quality Coverage** | >30% memories scored | Within 3 months |
| **Quality Distribution** | 20-30% high-quality | Pareto principle |
| **Search Latency** | <100ms with boost | SQLite-vec backend |
| **Monthly Cost** | <$0.50 or $0 | Groq API or local SLM |
| **Local SLM Usage** | >95% of scoring | Tier 1 success rate |

## FAQ

### Q: Do I need API keys for the quality system?

**A**: No! The default local SLM works with zero configuration, no API keys, and no external calls.

### Q: How much does it cost?

**A**: $0 with the default local SLM. Optional cloud APIs cost ~$0.30-0.50/month for typical usage.

### Q: Does quality scoring slow down searches?

**A**: No. Scoring happens asynchronously in the background. Quality-boosted search adds <20ms overhead.

### Q: Can I disable the quality system?

**A**: Yes, set `MCP_QUALITY_SYSTEM_ENABLED=false`. System works normally without quality scores.

### Q: How accurate is the local SLM?

**A**: 80%+ correlation with human quality ratings. Good enough for ranking and retention decisions.

### Q: What if the local SLM fails to download?

**A**: System falls back to implicit signals (access patterns). No failures, degraded gracefully.

### Q: Can I use my own quality scoring model?

**A**: Yes! Implement the `QualityEvaluator` interface and configure via `MCP_QUALITY_AI_PROVIDER`.

### Q: Does this work offline?

**A**: Yes! Local SLM works fully offline. No internet required for quality scoring.

## Related Documentation

- [Issue #260](https://github.com/doobidoo/mcp-memory-service/issues/260) - Quality System Specification
- [Issue #261](https://github.com/doobidoo/mcp-memory-service/issues/261) - Roadmap (Quality → Agentic RAG)
- [Consolidation Guide](./memory-consolidation-guide.md) - Detailed consolidation documentation
- [API Reference](../api/quality-endpoints.md) - HTTP API documentation

## Changelog

**v8.48.3** (2025-12-08):
- **Documentation Update**: Added comprehensive ONNX limitations section
- Documented self-matching bias and bimodal distribution issues
- Updated best practices with manual validation recommendations
- Added performance benchmarks from evaluation (4,762 memories)
- Linked to [Quality System Evaluation Report](https://github.com/doobidoo/mcp-memory-service/wiki/Memory-Quality-System-Evaluation)
- Created [Issue #268](https://github.com/doobidoo/mcp-memory-service/issues/268) for Phase 2 improvements

**v8.45.0** (2025-01-XX):
- Initial release of Memory Quality System
- Local SLM (ONNX) as primary tier
- Quality-based forgetting in consolidation
- Quality-boosted search with reranking
- Dashboard UI with quality badges and analytics
- Comprehensive MCP tools and HTTP API

---

**Need help?** Open an issue at https://github.com/doobidoo/mcp-memory-service/issues

**Quality System Improvements**: See [Issue #268](https://github.com/doobidoo/mcp-memory-service/issues/268) for planned enhancements
