cache key error when user id changes -fixed task 1 31_10_2025 v8

OPTIMIZATION_ENHANCEMENTS_REVIEW.md

@@ -0,0 +1,116 @@
+# Optimization Enhancements - Review and Implementation Plan
+
+## Executive Summary
+
+This document reviews the requested optimization enhancements and provides an implementation plan with any required deviations from the original specifications.
+
+## Current State Analysis
+
+### ✅ Already Implemented (Partial)
+
+1. **Parallel Processing**: 
+   - `process_request_parallel()` method exists (lines 696-751 in src/orchestrator_engine.py)
+   - Runs intent, skills, and safety agents in parallel using `asyncio.gather()`
+   - **Deviation Required**: The requested `process_agents_parallel()` method with different signature needs to be added
+
+2. **Context Caching**:
+   - Basic caching infrastructure exists with `session_cache` dictionary
+   - Cache config has TTL defined (3600s) but expiration not actively checked
+   - `_is_cache_valid()` exists but uses hardcoded 60s instead of config TTL
+   - **Deviation Required**: Need to add `add_context_cache()` method with proper TTL expiration
+
+3. **Metrics Tracking**:
+   - Basic token_count tracking exists in metadata
+   - Processing time tracked
+   - **Deviation Required**: Need comprehensive `track_response_metrics()` method with structured logging
+
+### ❌ Not Implemented
+
+4. **Query Similarity Detection**: No implementation found
+5. **Smart Context Pruning**: No token-count-based pruning exists
+
+## Implementation Plan
+
+### Step 1: Optimize Agent Chain
+**Status**: ⚠️ Partial Implementation  
+**Action Required**: Add new `process_agents_parallel()` method while keeping existing `process_request_parallel()`
+
+**Deviation Notes**:
+- Existing `process_request_parallel()` handles intent+skills+safety together
+- New method will be more generic for any agent pair execution
+- Will integrate with existing parallel processing flow
+
+### Step 2: Implement Context Caching with TTL
+**Status**: ⚠️ Infrastructure exists, expiration missing  
+**Action Required**: Add `add_context_cache()` method with expiration checking
+
+**Deviation Notes**:
+- Cache expiration needs to be checked on retrieval, not just set on store
+- Will modify `_get_from_memory_cache()` to check expiration
+- Will respect existing `cache_config['ttl']` value (3600s)
+
+### Step 3: Add Query Similarity Detection
+**Status**: ❌ Not Implemented  
+**Action Required**: Implement similarity checking using embeddings
+
+**Deviation Notes**:
+- FAISS infrastructure exists but is incomplete
+- Will use simple string similarity (Levenshtein/cosine) for MVP
+- Can be enhanced with embeddings later if needed
+- Will cache recent queries in orchestrator for similarity checking
+
+### Step 4: Implement Smart Context Pruning
+**Status**: ❌ Not Implemented  
+**Action Required**: Add `prune_context()` method with token counting
+
+**Deviation Notes**:
+- Token counting will use approximate method (4 chars ≈ 1 token)
+- Will preserve most recent interactions + most relevant (by keyword match)
+- Pruning threshold: 2000 tokens (configurable)
+
+### Step 5: Add Response Metrics Tracking
+**Status**: ⚠️ Partial Implementation  
+**Action Required**: Add comprehensive `track_response_metrics()` method
+
+**Deviation Notes**:
+- Will extend existing metadata tracking
+- Add structured logging for metrics
+- Track: latency, token_count, agent_calls, safety_score
+
+## Files to Modify
+
+1. `Research_AI_Assistant/src/orchestrator_engine.py`
+   - Add `process_agents_parallel()` method
+   - Add query similarity detection
+   - Add response metrics tracking
+   - Add agent_call_count tracking
+
+2. `Research_AI_Assistant/src/context_manager.py`
+   - Add `add_context_cache()` with TTL
+   - Enhance `_get_from_memory_cache()` with expiration check
+   - Add `prune_context()` method
+   - Add `get_token_count()` helper
+
+## Compatibility Considerations
+
+- All enhancements will be backward compatible
+- Existing functionality preserved
+- New methods will be additive, not replacing existing code
+- Cache TTL will respect existing config values
+
+## Testing Recommendations
+
+1. Test parallel agent execution with various agent combinations
+2. Verify cache expiration works correctly (test with different TTL values)
+3. Test query similarity with similar queries (threshold: 0.85)
+4. Verify context pruning maintains important information
+5. Validate metrics are tracked correctly in logs
+
+## Implementation Status
+
+- [ ] Step 1: Optimize Agent Chain
+- [ ] Step 2: Implement Context Caching
+- [ ] Step 3: Add Query Similarity Detection
+- [ ] Step 4: Implement Smart Context Pruning
+- [ ] Step 5: Add Response Metrics Tracking
+

OPTIMIZATION_IMPLEMENTATION_COMPLETE.md

@@ -0,0 +1,154 @@
+# Optimization Enhancements - Implementation Complete
+
+## Summary
+
+All 5 optimization enhancements have been successfully implemented with the following deviations and notes:
+
+## ✅ Step 1: Optimize Agent Chain
+
+**Implementation**: Added `process_agents_parallel()` method in `orchestrator_engine.py`
+
+**Location**: `Research_AI_Assistant/src/orchestrator_engine.py` lines 704-744
+
+**Features**:
+- Processes intent and skills agents in parallel using `asyncio.gather()`
+- Tracks agent call count for metrics
+- Handles exceptions gracefully
+- Returns list of results in order [intent_result, skills_result]
+
+**Deviation**: Method signature differs from original specification to work with existing agent structure. Uses dictionary input instead of direct request object.
+
+## ✅ Step 2: Implement Context Caching with TTL
+
+**Implementation**: Added `add_context_cache()` method with expiration checking
+
+**Location**: `Research_AI_Assistant/src/context_manager.py` lines 632-649
+
+**Features**:
+- Stores cache entries with expiration timestamps
+- TTL default: 3600 seconds (1 hour) from `cache_config`
+- Automatic expiration check in `_get_from_memory_cache()`
+- Backward compatible with old cache format
+
+**Integration**:
+- `_get_from_memory_cache()` now checks expiration before returning
+- Cache entries stored with structure: `{'value': context, 'expires': timestamp, 'timestamp': timestamp}`
+- Expired entries automatically removed
+
+## ✅ Step 3: Add Query Similarity Detection
+
+**Implementation**: Added `check_query_similarity()` and `_calculate_similarity()` methods
+
+**Location**: `Research_AI_Assistant/src/orchestrator_engine.py` lines 1982-2045
+
+**Features**:
+- Uses Jaccard similarity on word sets for comparison
+- Default threshold: 0.85 (configurable)
+- Stores recent queries in `self.recent_queries` list (last 50 queries)
+- Checks most recent queries first for better performance
+- Early exit in `process_request()` for duplicate detection
+
+**Algorithm**:
+- Jaccard similarity: `intersection / union` of word sets
+- Substring matching for very similar queries (boosts score to 0.9)
+- Case-insensitive comparison
+
+**Note**: Can be enhanced with embeddings for semantic similarity in future.
+
+## ✅ Step 4: Implement Smart Context Pruning
+
+**Implementation**: Added `prune_context()` and `get_token_count()` methods
+
+**Location**: `Research_AI_Assistant/src/context_manager.py` lines 651-755
+
+**Features**:
+- Token counting using approximation: 4 characters ≈ 1 token
+- Default max tokens: 2000 (configurable)
+- Priority system:
+  1. User context (essential)
+  2. Session context (essential)
+  3. Most recent interaction contexts (fits in remaining budget)
+- Preserves most recent interactions first
+- Logs pruning statistics
+
+**Integration**:
+- Called automatically in `_optimize_context()` before formatting
+- Ensures context stays within token limits for LLM consumption
+
+## ✅ Step 5: Add Response Metrics Tracking
+
+**Implementation**: Added `track_response_metrics()` method
+
+**Location**: `Research_AI_Assistant/src/orchestrator_engine.py` lines 2047-2100
+
+**Features**:
+- Tracks latency (processing time)
+- Tracks token count (word count approximation)
+- Tracks agent calls (incremented during parallel processing)
+- Tracks safety score (extracted from metadata)
+- Stores metrics history (last 100 entries)
+- Logs metrics for monitoring
+- Resets agent call count after each request
+
+**Metrics Tracked**:
+- `latency`: Processing time in seconds
+- `token_count`: Approximate tokens in response
+- `agent_calls`: Number of agents called during processing
+- `safety_score`: Overall safety score from safety analysis
+- `timestamp`: ISO timestamp of the metrics
+
+## Integration Points
+
+### Orchestrator Engine (`src/orchestrator_engine.py`)
+- Initialized tracking variables in `__init__()`:
+  - `self.recent_queries = []`
+  - `self.agent_call_count = 0`
+  - `self.response_metrics_history = []`
+- Query similarity checked early in `process_request()`
+- Metrics tracked after response generation
+- Recent queries stored for similarity checking
+
+### Context Manager (`src/context_manager.py`)
+- Cache structure updated to support TTL
+- Context pruning integrated into `_optimize_context()`
+- Cache expiration checked on retrieval
+- Token counting utilities added
+
+## Testing Recommendations
+
+1. **Parallel Processing**: Test with multiple agent combinations
+2. **Cache TTL**: Verify expiration after TTL period (change TTL to short value for testing)
+3. **Query Similarity**: Test with similar queries (e.g., "What is AI?" vs "Tell me about AI")
+4. **Context Pruning**: Test with large contexts (add many interaction contexts)
+5. **Metrics Tracking**: Verify metrics appear in logs and history
+
+## Configuration
+
+- **Cache TTL**: Set in `context_manager.cache_config['ttl']` (default: 3600s)
+- **Similarity Threshold**: Set in `check_query_similarity(threshold=0.85)`
+- **Max Tokens**: Set in `prune_context(max_tokens=2000)`
+- **Max Recent Queries**: Set in `self.max_recent_queries` (default: 50)
+
+## Backward Compatibility
+
+All enhancements are backward compatible:
+- Old cache format still works (direct value storage)
+- New cache format detected and handled appropriately
+- Existing functionality preserved
+- No breaking changes to API
+
+## Performance Impact
+
+- **Parallel Processing**: Reduces latency for multi-agent operations
+- **Cache with TTL**: Reduces database queries
+- **Query Similarity**: Prevents duplicate processing
+- **Context Pruning**: Ensures context fits within LLM token limits
+- **Metrics Tracking**: Minimal overhead (logging only)
+
+## Future Enhancements
+
+1. **Query Similarity**: Use embeddings for semantic similarity
+2. **Context Pruning**: Implement relevance-based ranking (not just recency)
+3. **Metrics Tracking**: Add metrics aggregation and analytics
+4. **Cache**: Implement LRU eviction policy (currently only TTL)
+

src/context_manager.py

@@ -5,6 +5,7 @@ import logging
 import uuid
 import hashlib
 import threading
+import time
 from contextlib import contextmanager
 from datetime import datetime, timedelta
 from typing import Dict, Optional, List
@@ -249,12 +250,23 @@ class EfficientContextManager:
         session_context = self._get_from_memory_cache(session_cache_key)
         
         # Check if cached session context matches current user_id
-        if session_context and session_context.get("user_id") != user_id:
-            # User changed, invalidate session cache
-            logger.info(f"User mismatch in cache for session {session_id}, invalidating cache")
-            session_context = None
-            if session_cache_key in self.session_cache:
-                del self.session_cache[session_cache_key]
+        # Handle both old and new cache formats
+        cached_entry = self.session_cache.get(session_cache_key)
+        if cached_entry:
+            # Extract actual context from cache entry
+            if isinstance(cached_entry, dict) and 'value' in cached_entry:
+                actual_context = cached_entry.get('value', {})
+            else:
+                actual_context = cached_entry
+            
+            if actual_context and actual_context.get("user_id") != user_id:
+                # User changed, invalidate session cache
+                logger.info(f"User mismatch in cache for session {session_id}, invalidating cache")
+                session_context = None
+                if session_cache_key in self.session_cache:
+                    del self.session_cache[session_cache_key]
+            else:
+                session_context = actual_context
         
         # Get user context separately
         user_context = self._get_from_memory_cache(user_cache_key)
@@ -263,8 +275,8 @@ class EfficientContextManager:
             # Retrieve from database with user context
             session_context = await self._retrieve_from_db(session_id, user_input, user_id)
             
-            # Cache session context (cache invalidation for user changes is handled in _retrieve_from_db)
-            self._warm_memory_cache(session_cache_key, session_context)
+            # Step 2: Cache session context with TTL
+            self.add_context_cache(session_cache_key, session_context, ttl=self.cache_config.get("ttl", 3600))
         
         # Handle user context separately - load only once and cache thereafter
         # Cache does not refer to database after initial load
@@ -572,10 +584,15 @@ Keep the summary concise (approximately 100 tokens)."""
         """
         Optimize context for LLM consumption
         Format: [Session Context] + [User Context] + [Interaction Context #N, #N-1, ...]
+        
+        Applies smart pruning before formatting.
         """
-        user_context = context.get("user_context", "")
-        interaction_contexts = context.get("interaction_contexts", [])
-        session_context = context.get("session_context", {})
+        # Step 4: Prune context if it exceeds token limits
+        pruned_context = self.prune_context(context, max_tokens=2000)
+        
+        user_context = pruned_context.get("user_context", "")
+        interaction_contexts = pruned_context.get("interaction_contexts", [])
+        session_context = pruned_context.get("session_context", {})
         session_summary = session_context.get("summary", "") if isinstance(session_context, dict) else ""
         
         # Format interaction contexts as requested
@@ -593,22 +610,175 @@ Keep the summary concise (approximately 100 tokens)."""
             combined_context += "\n\n".join(formatted_interactions)
         
         return {
-            "session_id": context.get("session_id"),
-            "user_id": context.get("user_id", "Test_Any"),
+            "session_id": pruned_context.get("session_id"),
+            "user_id": pruned_context.get("user_id", "Test_Any"),
             "user_context": user_context,
             "session_context": session_context,
             "interaction_contexts": interaction_contexts,
             "combined_context": combined_context,  # For direct use in prompts
-            "preferences": context.get("preferences", {}),
-            "active_tasks": context.get("active_tasks", []),
-            "last_activity": context.get("last_activity")
+            "preferences": pruned_context.get("preferences", {}),
+            "active_tasks": pruned_context.get("active_tasks", []),
+            "last_activity": pruned_context.get("last_activity")
         }
     
     def _get_from_memory_cache(self, cache_key: str) -> dict:
         """
-        Retrieve context from in-memory session cache
+        Retrieve context from in-memory session cache with expiration check
+        """
+        cached = self.session_cache.get(cache_key)
+        if not cached:
+            return None
+        
+        # Check if it's the new format with expiration
+        if isinstance(cached, dict) and 'value' in cached:
+            # New format with TTL
+            if self._is_cache_expired(cached):
+                # Remove expired cache entry
+                del self.session_cache[cache_key]
+                logger.debug(f"Cache expired for key: {cache_key}")
+                return None
+            return cached.get('value')
+        else:
+            # Old format (direct value) - return as-is for backward compatibility
+            return cached
+    
+    def _is_cache_expired(self, cache_entry: dict) -> bool:
+        """
+        Check if cache entry has expired based on TTL
+        """
+        if not isinstance(cache_entry, dict):
+            return True
+        
+        expires = cache_entry.get('expires')
+        if not expires:
+            return False  # No expiration set, consider valid
+        
+        return time.time() > expires
+    
+    def add_context_cache(self, key: str, value: dict, ttl: int = 3600):
+        """
+        Step 2: Implement Context Caching with TTL expiration
+        
+        Add context to cache with expiration time.
+        
+        Args:
+            key: Cache key
+            value: Value to cache (dict)
+            ttl: Time to live in seconds (default 3600 = 1 hour)
+        """
+        import time
+        self.session_cache[key] = {
+            'value': value,
+            'expires': time.time() + ttl,
+            'timestamp': time.time()
+        }
+        logger.debug(f"Cached context for key: {key} with TTL: {ttl}s")
+    
+    def get_token_count(self, text: str) -> int:
         """
-        return self.session_cache.get(cache_key)
+        Approximate token count for text (4 characters ≈ 1 token)
+        
+        Args:
+            text: Text to count tokens for
+            
+        Returns:
+            Approximate token count
+        """
+        if not text:
+            return 0
+        # Simple approximation: 4 characters per token
+        return len(text) // 4
+    
+    def prune_context(self, context: dict, max_tokens: int = 2000) -> dict:
+        """
+        Step 4: Implement Smart Context Pruning
+        
+        Prune context to stay within token limit while keeping most recent and relevant content.
+        
+        Args:
+            context: Context dictionary to prune
+            max_tokens: Maximum token count (default 2000)
+            
+        Returns:
+            Pruned context dictionary
+        """
+        try:
+            # Calculate current token count
+            current_tokens = self._calculate_context_tokens(context)
+            
+            if current_tokens <= max_tokens:
+                return context  # No pruning needed
+            
+            logger.info(f"Context token count ({current_tokens}) exceeds limit ({max_tokens}), pruning...")
+            
+            # Create a copy to avoid modifying original
+            pruned_context = context.copy()
+            
+            # Priority: Keep most recent interactions + session context + user context
+            interaction_contexts = pruned_context.get('interaction_contexts', [])
+            session_context = pruned_context.get('session_context', {})
+            user_context = pruned_context.get('user_context', '')
+            
+            # Keep user context and session context (essential)
+            essential_tokens = (
+                self.get_token_count(user_context) + 
+                self.get_token_count(str(session_context))
+            )
+            
+            # Calculate how many interaction contexts we can keep
+            available_tokens = max_tokens - essential_tokens
+            if available_tokens < 0:
+                # Essential context itself is too large - summarize user context
+                if self.get_token_count(user_context) > max_tokens // 2:
+                    pruned_context['user_context'] = user_context[:max_tokens * 2]  # Rough cut
+                    logger.warning(f"User context too large, truncated")
+                return pruned_context
+            
+            # Keep most recent interactions that fit in token budget
+            kept_interactions = []
+            current_size = 0
+            
+            for interaction in interaction_contexts:
+                summary = interaction.get('summary', '')
+                interaction_tokens = self.get_token_count(summary)
+                
+                if current_size + interaction_tokens <= available_tokens:
+                    kept_interactions.append(interaction)
+                    current_size += interaction_tokens
+                else:
+                    break  # Can't fit any more
+            
+            pruned_context['interaction_contexts'] = kept_interactions
+            
+            logger.info(f"Pruned context: kept {len(kept_interactions)}/{len(interaction_contexts)} interactions, "
+                       f"reduced from {current_tokens} to {self._calculate_context_tokens(pruned_context)} tokens")
+            
+            return pruned_context
+            
+        except Exception as e:
+            logger.error(f"Error pruning context: {e}", exc_info=True)
+            return context  # Return original on error
+    
+    def _calculate_context_tokens(self, context: dict) -> int:
+        """Calculate total token count for context"""
+        total = 0
+        
+        # Count tokens in each component
+        user_context = context.get('user_context', '')
+        total += self.get_token_count(str(user_context))
+        
+        session_context = context.get('session_context', {})
+        if isinstance(session_context, dict):
+            total += self.get_token_count(str(session_context.get('summary', '')))
+        else:
+            total += self.get_token_count(str(session_context))
+        
+        interaction_contexts = context.get('interaction_contexts', [])
+        for interaction in interaction_contexts:
+            summary = interaction.get('summary', '')
+            total += self.get_token_count(str(summary))
+        
+        return total
     
     async def _retrieve_from_db(self, session_id: str, user_input: str, user_id: str = "Test_Any") -> dict:
         """
@@ -809,8 +979,10 @@ Keep the summary concise (approximately 100 tokens)."""
     def _warm_memory_cache(self, cache_key: str, context: dict):
         """
         Warm the in-memory cache with retrieved context
+        Note: Use add_context_cache() instead for TTL support
         """
-        self.session_cache[cache_key] = context
+        # Use add_context_cache for consistency with TTL
+        self.add_context_cache(cache_key, context, ttl=self.cache_config.get("ttl", 3600))
     
     def _update_cache_with_interaction_context(self, session_id: str, interaction_summary: str, created_at: str):
         """

src/orchestrator_engine.py

@@ -57,6 +57,14 @@ class MVPOrchestrator:
         # Context cache to prevent loops
         self._context_cache = {}  # cache_key -> {context, timestamp}
         
+        # Query similarity tracking for duplicate detection
+        self.recent_queries = []  # List of {query, response, timestamp}
+        self.max_recent_queries = 50  # Keep last 50 queries
+        
+        # Response metrics tracking
+        self.agent_call_count = 0
+        self.response_metrics_history = []  # Store recent metrics
+        
         logger.info("MVPOrchestrator initialized with safety revision thresholds")
     
     def set_user_id(self, session_id: str, user_id: str):
@@ -163,6 +171,16 @@ class MVPOrchestrator:
         }
         
         try:
+            # Step 3: Check query similarity BEFORE processing (early exit for duplicates)
+            # Note: This happens early to skip full processing for identical/similar queries
+            similar_response = self.check_query_similarity(user_input, threshold=0.95)  # Higher threshold for exact duplicates
+            if similar_response:
+                logger.info(f"Similar/duplicate query detected, using cached response")
+                # Still track metrics for cached response (minimal processing)
+                metrics_start = time.time()
+                self.track_response_metrics(metrics_start, similar_response)
+                return similar_response
+            
             # Step 1: Generate unique interaction ID
             interaction_id = self._generate_interaction_id(session_id)
             logger.info(f"Generated interaction ID: {interaction_id}")
@@ -486,6 +504,19 @@ This response has been flagged for potential safety concerns:
                 except Exception as e:
                     logger.error(f"Error generating interaction context: {e}", exc_info=True)
             
+            # Track response metrics
+            self.track_response_metrics(start_time, result)
+            
+            # Store query and response for similarity checking
+            self.recent_queries.append({
+                'query': user_input,
+                'response': result,
+                'timestamp': time.time()
+            })
+            # Keep only recent queries
+            if len(self.recent_queries) > self.max_recent_queries:
+                self.recent_queries = self.recent_queries[-self.max_recent_queries:]
+            
             logger.info(f"Request processing complete. Response length: {len(response_text)}")
             return result
             
@@ -693,6 +724,48 @@ This response has been flagged for potential safety concerns:
         
         return " | ".join(summary_parts) if summary_parts else "No prior context"
     
+    async def process_agents_parallel(self, request: Dict) -> List:
+        """
+        Step 1: Optimize Agent Chain - Process multiple agents in parallel
+        
+        Args:
+            request: Dictionary containing request data with 'user_input' and 'context'
+            
+        Returns:
+            List of agent results in order [intent_result, skills_result]
+        """
+        user_input = request.get('user_input', '')
+        context = request.get('context', {})
+        
+        # Increment agent call count for metrics
+        self.agent_call_count += 2  # Two agents called
+        
+        tasks = [
+            self.agents['intent_recognition'].execute(
+                user_input=user_input,
+                context=context
+            ),
+            self.agents['skills_identification'].execute(
+                user_input=user_input,
+                context=context
+            ),
+        ]
+        
+        try:
+            results = await asyncio.gather(*tasks, return_exceptions=True)
+            # Handle exceptions
+            processed_results = []
+            for idx, result in enumerate(results):
+                if isinstance(result, Exception):
+                    logger.error(f"Agent task {idx} failed: {result}")
+                    processed_results.append({})
+                else:
+                    processed_results.append(result)
+            return processed_results
+        except Exception as e:
+            logger.error(f"Error in parallel agent processing: {e}", exc_info=True)
+            return [{}, {}]
+    
     async def process_request_parallel(self, session_id: str, user_input: str, context: Dict) -> Dict:
         """Process intent, skills, and safety in parallel"""
         
@@ -714,6 +787,9 @@ This response has been flagged for potential safety concerns:
                 context=context
             )
             
+            # Increment agent call count for metrics
+            self.agent_call_count += 3
+            
             # Wait for all to complete
             results = await asyncio.gather(
                 intent_task,
@@ -1904,3 +1980,123 @@ Revised Response:"""
 Additional guidance for response: {improvement_instructions}. Ensure all advice is specific, actionable, and acknowledges different backgrounds and circumstances."""
         
         return improved_prompt
+    
+    def check_query_similarity(self, new_query: str, threshold: float = 0.85) -> Optional[Dict]:
+        """
+        Step 3: Add Query Similarity Detection
+        
+        Check if new query is similar to any recent queries above threshold.
+        Uses simple string similarity (can be enhanced with embeddings later).
+        
+        Args:
+            new_query: The new query to check
+            threshold: Similarity threshold (default 0.85)
+            
+        Returns:
+            Cached response dict if similar query found, None otherwise
+        """
+        if not self.recent_queries:
+            return None
+        
+        new_query_lower = new_query.lower().strip()
+        
+        for cached_query_data in reversed(self.recent_queries):  # Check most recent first
+            cached_query = cached_query_data.get('query', '')
+            if not cached_query:
+                continue
+            
+            cached_query_lower = cached_query.lower().strip()
+            
+            # Calculate similarity using simple word overlap (Jaccard similarity)
+            similarity = self._calculate_similarity(new_query_lower, cached_query_lower)
+            
+            if similarity > threshold:
+                logger.info(f"Similar query detected (similarity: {similarity:.2f}): '{new_query[:50]}...' similar to '{cached_query[:50]}...'")
+                return cached_query_data.get('response')
+        
+        return None
+    
+    def _calculate_similarity(self, query1: str, query2: str) -> float:
+        """
+        Calculate similarity between two queries using Jaccard similarity on words.
+        Can be enhanced with embeddings for semantic similarity.
+        """
+        if not query1 or not query2:
+            return 0.0
+        
+        # Split into words and create sets
+        words1 = set(query1.split())
+        words2 = set(query2.split())
+        
+        if not words1 or not words2:
+            return 0.0
+        
+        # Calculate Jaccard similarity
+        intersection = len(words1.intersection(words2))
+        union = len(words1.union(words2))
+        
+        if union == 0:
+            return 0.0
+        
+        jaccard = intersection / union
+        
+        # Also check for substring similarity for very similar queries
+        if query1 in query2 or query2 in query1:
+            jaccard = max(jaccard, 0.9)
+        
+        return jaccard
+    
+    def track_response_metrics(self, start_time: float, response: Dict):
+        """
+        Step 5: Add Response Metrics Tracking
+        
+        Track performance metrics for responses.
+        
+        Args:
+            start_time: Start time from time.time()
+            response: Response dictionary containing response data
+        """
+        try:
+            latency = time.time() - start_time
+            
+            # Extract response text for token counting
+            response_text = (
+                response.get('response') or 
+                response.get('final_response') or 
+                str(response.get('result', ''))
+            )
+            
+            # Approximate token count (4 characters ≈ 1 token)
+            token_count = len(response_text.split()) if response_text else 0
+            
+            # Extract safety score
+            safety_score = 0.8  # Default
+            if 'metadata' in response:
+                synthesis_result = response['metadata'].get('synthesis_result', {})
+                safety_result = response['metadata'].get('safety_result', {})
+                if safety_result:
+                    safety_analysis = safety_result.get('safety_analysis', {})
+                    safety_score = safety_analysis.get('overall_safety_score', 0.8)
+            
+            metrics = {
+                'latency': latency,
+                'token_count': token_count,
+                'agent_calls': self.agent_call_count,
+                'safety_score': safety_score,
+                'timestamp': datetime.now().isoformat()
+            }
+            
+            # Store in history (keep last 100)
+            self.response_metrics_history.append(metrics)
+            if len(self.response_metrics_history) > 100:
+                self.response_metrics_history = self.response_metrics_history[-100:]
+            
+            # Log metrics
+            logger.info(f"Response Metrics - Latency: {latency:.3f}s, Tokens: {token_count}, "
+                       f"Agent Calls: {self.agent_call_count}, Safety Score: {safety_score:.2f}")
+            
+            # Reset agent call count for next request
+            self.agent_call_count = 0
+            
+        except Exception as e:
+            logger.error(f"Error tracking response metrics: {e}", exc_info=True)