api endpoints for UI migration v10

FLASK_API_SETUP.md

@@ -0,0 +1,368 @@
+# Flask API Setup Guide
+
+## Overview
+
+The Flask API (`flask_api.py`) provides a simple REST API interface to the Research AI Assistant orchestrator. It runs on port 5001, separate from Gradio (port 7860).
+
+## Architecture
+
+```
+┌─────────────────┐
+│  Express.js UI  │ Port 3000
+│  (or other      │ - User interface
+│   frontend)     │ - Session management
+└────────┬────────┘
+         │ HTTP POST /api/chat
+         ↓
+┌─────────────────┐
+│  Flask API      │ Port 5001
+│  flask_api.py   │ - REST endpoints
+│                 │ - Imports from app.py
+└────────┬────────┘ - Direct Python access
+         │
+┌─────────────────┐
+│  Orchestrator   │ Python
+│  (from app.py)  │ - AI agents
+│                 │ - Context management
+└─────────────────┘
+
+┌─────────────────┐
+│  Gradio UI      │ Port 7860
+│  (optional)     │ - Alternative UI
+│  app.py         │ - Can run alongside
+└─────────────────┘
+```
+
+## Quick Start
+
+### Step 1: Install Dependencies
+
+```bash
+pip install flask flask-cors
+```
+
+Or install all requirements:
+```bash
+pip install -r requirements.txt
+```
+
+### Step 2: Start Flask API
+
+```bash
+# Default port 5001
+python flask_api.py
+
+# Or specify custom port
+FLASK_PORT=5001 python flask_api.py
+```
+
+You should see:
+```
+STARTING FLASK API
+INITIALIZING AI ORCHESTRATOR
+✓ AI ORCHESTRATOR READY
+Starting Flask API on port 5001
+```
+
+### Step 3: Test the API
+
+```bash
+# Health check
+curl http://localhost:5001/health
+
+# Chat endpoint
+curl -X POST http://localhost:5001/api/chat \
+  -H "Content-Type: application/json" \
+  -d '{
+    "message": "Hello, how are you?",
+    "session_id": "test-session-123",
+    "user_id": "test-user"
+  }'
+```
+
+## API Endpoints
+
+### `GET /health`
+
+Health check endpoint.
+
+**Response:**
+```json
+{
+  "status": "ok",
+  "orchestrator_available": true,
+  "service": "flask_api"
+}
+```
+
+### `POST /api/chat`
+
+Main chat endpoint for processing user messages.
+
+**Request:**
+```json
+{
+  "message": "What is machine learning?",
+  "history": [],  // optional
+  "session_id": "session-123",  // optional, auto-generated if not provided
+  "user_id": "user-123"  // optional, defaults to "Admin_J"
+}
+```
+
+**Response:**
+```json
+{
+  "response": "Machine learning is...",
+  "history": [
+    {"role": "user", "content": "What is machine learning?"},
+    {"role": "assistant", "content": "Machine learning is..."}
+  ],
+  "reasoning": {
+    "chain_of_thought": {...},
+    "confidence_calibration": {...}
+  },
+  "performance": {
+    "processing_time": 1.23,
+    "tokens_used": 456
+  },
+  "context": {
+    "session_context": "...",
+    "relevant_interactions": [...]
+  },
+  "session_id": "session-123",
+  "skills": "<div>...</div>"  // HTML for skills display
+}
+```
+
+### `GET /api/orchestrator/status`
+
+Check if orchestrator is available.
+
+**Response:**
+```json
+{
+  "orchestrator_available": true,
+  "orchestrator_type": "MVPOrchestrator"
+}
+```
+
+## Configuration
+
+### Environment Variables
+
+```bash
+# Flask configuration
+FLASK_PORT=5001      # Port for Flask API (default: 5001)
+FLASK_HOST=0.0.0.0  # Host to bind to (default: 0.0.0.0)
+
+# AI configuration (used by orchestrator)
+HF_TOKEN=your_token_here  # Hugging Face token for API access
+```
+
+### Ports
+
+- **Flask API**: 5001 (configurable via `FLASK_PORT`)
+- **Gradio UI**: 7860 (separate, can run alongside)
+- **Express.js** (external): 3000 (your frontend)
+
+## Running with Gradio
+
+You can run Flask API alongside Gradio:
+
+**Terminal 1: Flask API**
+```bash
+python flask_api.py
+```
+
+**Terminal 2: Gradio UI (optional)**
+```bash
+python app.py
+```
+
+Both can run simultaneously on different ports.
+
+## Integration Examples
+
+### Express.js Integration
+
+See `EXPRESS_FLASK_INTEGRATION.md` for Express.js setup.
+
+### Python Client
+
+```python
+import requests
+
+def chat_with_flask(message, session_id=None):
+    url = "http://localhost:5001/api/chat"
+    data = {
+        "message": message,
+        "session_id": session_id or "python-client-session"
+    }
+    response = requests.post(url, json=data)
+    return response.json()
+
+# Usage
+result = chat_with_flask("Hello, how are you?")
+print(result["response"])
+```
+
+### JavaScript/Node.js Client
+
+```javascript
+const axios = require('axios');
+
+async function chatWithFlask(message, sessionId = null) {
+    const response = await axios.post('http://localhost:5001/api/chat', {
+        message: message,
+        session_id: sessionId || `node-client-session-${Date.now()}`
+    });
+    return response.data;
+}
+
+// Usage
+chatWithFlask("Hello, how are you?")
+    .then(result => console.log(result.response))
+    .catch(error => console.error(error));
+```
+
+## Troubleshooting
+
+### Issue: "Orchestrator not available"
+
+**Check:**
+1. Flask API logs show initialization errors
+2. `HF_TOKEN` environment variable is set
+3. All dependencies are installed
+
+**Solution:**
+```bash
+# Check logs
+# Look for "INITIALIZING AI ORCHESTRATOR" section
+
+# Verify HF token
+echo $HF_TOKEN
+
+# Reinstall dependencies
+pip install -r requirements.txt
+```
+
+### Issue: "Port already in use"
+
+**Solution:**
+```bash
+# Use different port
+FLASK_PORT=5002 python flask_api.py
+```
+
+### Issue: "Module not found"
+
+**Check:**
+1. You're in the correct directory (same as `app.py`)
+2. All imports are available
+
+**Solution:**
+```bash
+# Make sure you're in the project root
+cd /path/to/Research_AI_Assistant
+
+# Verify Python path
+python -c "import app; print('✓ app.py importable')"
+```
+
+### Issue: "CORS errors"
+
+**Solution:** Flask-CORS is enabled by default. If issues persist:
+
+```python
+# In flask_api.py, customize CORS:
+from flask_cors import CORS
+CORS(app, resources={r"/api/*": {"origins": "*"}})
+```
+
+## Advantages Over Gradio API
+
+| Feature | Flask API | Gradio HTTP |
+|---------|-----------|-------------|
+| **Simplicity** | ✅ Simple JSON | ⚠️ Gradio-specific format |
+| **Control** | ✅ Full control | ⚠️ Limited by Gradio |
+| **Path issues** | ✅ None | ⚠️ Can have path problems |
+| **Dependencies** | ✅ Just Flask | ⚠️ Requires Gradio client |
+| **API design** | ✅ You design it | ⚠️ Gradio's design |
+| **Startup** | ✅ Fast | ⚠️ Slower (Gradio overhead) |
+
+## Production Deployment
+
+### Using PM2 (Process Manager)
+
+```bash
+# Install PM2
+npm install -g pm2
+
+# Start Flask API
+pm2 start flask_api.py --interpreter python3 --name flask-api
+
+# Start Express (if using)
+pm2 start server.js --name express
+
+# View status
+pm2 status
+```
+
+### Using systemd (Linux)
+
+Create `/etc/systemd/system/flask-api.service`:
+```ini
+[Unit]
+Description=Research AI Assistant Flask API
+After=network.target
+
+[Service]
+Type=simple
+User=your-user
+WorkingDirectory=/path/to/Research_AI_Assistant
+Environment="FLASK_PORT=5001"
+Environment="HF_TOKEN=your-token"
+ExecStart=/usr/bin/python3 flask_api.py
+Restart=always
+
+[Install]
+WantedBy=multi-user.target
+```
+
+Then:
+```bash
+sudo systemctl enable flask-api
+sudo systemctl start flask-api
+```
+
+## Security Considerations
+
+1. **CORS**: Currently allows all origins (`*`). In production, restrict to your frontend domain:
+   ```python
+   CORS(app, origins=["https://your-frontend.com"])
+   ```
+
+2. **Rate Limiting**: Consider adding rate limiting:
+   ```bash
+   pip install flask-limiter
+   ```
+
+3. **Authentication**: Add API keys or tokens for production:
+   ```python
+   @app.before_request
+   def check_api_key():
+       api_key = request.headers.get('X-API-Key')
+       if api_key != os.getenv('API_KEY'):
+           return jsonify({'error': 'Unauthorized'}), 401
+   ```
+
+## Summary
+
+✅ **Simple**: Just Flask, no complex wrappers  
+✅ **Flexible**: Your API design, your control  
+✅ **Compatible**: Works alongside Gradio  
+✅ **Standard**: Standard REST API pattern  
+✅ **Fast**: Direct Python imports, no overhead  
+
+Start Flask, integrate your frontend, done! 🚀
+

express_flask_integration.js

@@ -0,0 +1,145 @@
+/**
+ * Express.js Integration with Flask API
+ * 
+ * This module provides Express routes that proxy requests to the Flask API.
+ * Use this if you have an Express.js frontend that needs to communicate with
+ * the Python Flask API backend.
+ * 
+ * Usage in server.js:
+ *   const { setupFlaskRoutes, initializeFlask } = require('./express_flask_integration');
+ *   setupFlaskRoutes(app);
+ *   await initializeFlask();
+ */
+
+const axios = require('axios');
+
+// Flask API configuration
+const FLASK_API_URL = process.env.FLASK_API_URL || 'http://localhost:5001';
+const FLASK_HEALTH_ENDPOINT = `${FLASK_API_URL}/health`;
+const FLASK_CHAT_ENDPOINT = `${FLASK_API_URL}/api/chat`;
+
+/**
+ * Initialize Flask API connection
+ * Checks if Flask API is running and orchestrator is available
+ */
+async function initializeFlask() {
+    console.log('Checking Flask API connection...');
+    
+    try {
+        const response = await axios.get(FLASK_HEALTH_ENDPOINT, {
+            timeout: 5000
+        });
+        
+        if (response.data.status === 'ok') {
+            console.log('✓ Flask API connected');
+            
+            if (response.data.orchestrator_available) {
+                console.log('✓ AI Orchestrator available');
+            } else {
+                console.warn('⚠ AI Orchestrator not available - some features may be limited');
+            }
+            
+            return true;
+        } else {
+            console.error('✗ Flask API health check failed');
+            return false;
+        }
+    } catch (error) {
+        console.error('✗ Cannot connect to Flask API:', error.message);
+        console.error(`  Make sure Flask API is running on ${FLASK_API_URL}`);
+        console.error('  Start it with: python flask_api.py');
+        return false;
+    }
+}
+
+/**
+ * Setup Flask routes on Express app
+ * Adds /api/flask/* routes that proxy to Flask API
+ */
+function setupFlaskRoutes(app) {
+    // Health check route (proxies to Flask)
+    app.get('/api/flask/health', async (req, res) => {
+        try {
+            const response = await axios.get(FLASK_HEALTH_ENDPOINT);
+            res.json(response.data);
+        } catch (error) {
+            res.status(503).json({
+                error: 'Flask API unavailable',
+                message: error.message
+            });
+        }
+    });
+    
+    // Main chat endpoint (proxies to Flask)
+    app.post('/api/chat', async (req, res) => {
+        try {
+            const { message, history, session_id, user_id } = req.body;
+            
+            if (!message || !message.trim()) {
+                return res.status(400).json({
+                    error: 'Message is required'
+                });
+            }
+            
+            // Forward request to Flask API
+            const response = await axios.post(FLASK_CHAT_ENDPOINT, {
+                message: message.trim(),
+                history: history || [],
+                session_id: session_id || `express-session-${Date.now()}`,
+                user_id: user_id || 'Express_User'
+            }, {
+                timeout: 60000  // 60 second timeout for AI processing
+            });
+            
+            // Return Flask response
+            res.json(response.data);
+            
+        } catch (error) {
+            console.error('Error in Flask chat endpoint:', error);
+            
+            if (error.code === 'ECONNREFUSED') {
+                return res.status(503).json({
+                    error: 'Flask API unavailable',
+                    message: 'The AI backend is not running. Please start Flask API.',
+                    response: 'I apologize, but the AI service is temporarily unavailable. Please try again later.'
+                });
+            }
+            
+            if (error.response) {
+                // Flask API returned an error
+                return res.status(error.response.status).json(error.response.data);
+            }
+            
+            // Other errors
+            res.status(500).json({
+                error: 'Internal server error',
+                message: error.message,
+                response: 'I apologize, but I encountered an error processing your request. Please try again.'
+            });
+        }
+    });
+    
+    // Orchestrator status route
+    app.get('/api/flask/orchestrator/status', async (req, res) => {
+        try {
+            const response = await axios.get(`${FLASK_API_URL}/api/orchestrator/status`);
+            res.json(response.data);
+        } catch (error) {
+            res.status(503).json({
+                error: 'Flask API unavailable',
+                message: error.message
+            });
+        }
+    });
+    
+    console.log('✓ Flask routes configured');
+    console.log(`  POST /api/chat -> ${FLASK_CHAT_ENDPOINT}`);
+    console.log(`  GET  /api/flask/health -> ${FLASK_HEALTH_ENDPOINT}`);
+}
+
+module.exports = {
+    setupFlaskRoutes,
+    initializeFlask,
+    FLASK_API_URL
+};
+

flask_api.py

@@ -0,0 +1,182 @@
+"""
+Flask API for Research AI Assistant
+Provides REST API endpoints for external integrations (e.g., Express.js frontend)
+
+Run separately from Gradio on port 5001 (Gradio runs on 7860)
+Usage: python flask_api.py
+"""
+
+import os
+import sys
+import logging
+import asyncio
+from flask import Flask, request, jsonify
+from flask_cors import CORS
+from typing import Dict, Any, Optional
+
+# Configure logging
+logging.basicConfig(
+    level=logging.INFO,
+    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
+)
+logger = logging.getLogger(__name__)
+
+# Create Flask app
+app = Flask(__name__)
+CORS(app)  # Enable CORS for Express.js and other frontends
+
+# Import orchestrator and handlers from app.py
+# This imports without triggering Gradio launch
+_orchestrator = None
+_process_message_async = None
+
+def initialize_orchestrator():
+    """Initialize the orchestrator from app.py"""
+    global _orchestrator, _process_message_async
+    
+    try:
+        logger.info("=" * 60)
+        logger.info("STARTING FLASK API")
+        logger.info("=" * 60)
+        logger.info("INITIALIZING AI ORCHESTRATOR")
+        logger.info("=" * 60)
+        
+        # Import from app.py (this won't trigger app.py's launch code)
+        from app import orchestrator, process_message_async
+        
+        _orchestrator = orchestrator
+        _process_message_async = process_message_async
+        
+        if _orchestrator is not None:
+            logger.info("✓ AI ORCHESTRATOR READY")
+        else:
+            logger.warning("⚠ Orchestrator not available - some features may be limited")
+        
+        logger.info("=" * 60)
+        
+    except Exception as e:
+        logger.error(f"Failed to initialize orchestrator: {e}", exc_info=True)
+        _orchestrator = None
+        _process_message_async = None
+
+# Initialize on startup
+initialize_orchestrator()
+
+@app.route('/health', methods=['GET'])
+def health_check():
+    """Health check endpoint"""
+    return jsonify({
+        'status': 'ok',
+        'orchestrator_available': _orchestrator is not None,
+        'service': 'flask_api'
+    })
+
+@app.route('/api/chat', methods=['POST'])
+def chat():
+    """
+    Main chat endpoint
+    Receives JSON:
+    {
+        "message": "user message",
+        "history": [],  // optional, chat history
+        "session_id": "session-123",  // optional
+        "user_id": "user-123"  // optional, defaults to "Admin_J"
+    }
+    
+    Returns JSON:
+    {
+        "response": "assistant response",
+        "history": [...],  // updated chat history
+        "reasoning": {...},  // reasoning data
+        "performance": {...},  // performance metrics
+        "context": {...},  // context data
+        "session_id": "session-123",
+        "skills": "..."  // skills HTML
+    }
+    """
+    try:
+        data = request.get_json()
+        
+        if not data:
+            return jsonify({'error': 'No JSON data provided'}), 400
+        
+        message = data.get('message', '')
+        if not message or not message.strip():
+            return jsonify({'error': 'Message is required'}), 400
+        
+        # Extract optional parameters
+        history = data.get('history', [])
+        session_id = data.get('session_id', f"flask-session-{os.urandom(4).hex()}")
+        user_id = data.get('user_id', 'Admin_J')
+        
+        logger.info(f"Chat request: message={message[:50]}..., session_id={session_id}, user_id={user_id}")
+        
+        # Process message using the async handler from app.py
+        if _process_message_async is None:
+            return jsonify({
+                'error': 'Orchestrator not available',
+                'response': 'I apologize, but the AI system is not available. Please try again later.'
+            }), 503
+        
+        # Run async function in sync context
+        loop = asyncio.new_event_loop()
+        asyncio.set_event_loop(loop)
+        try:
+            result = loop.run_until_complete(
+                _process_message_async(
+                    message=message.strip(),
+                    history=history,
+                    session_id=session_id,
+                    user_id=user_id
+                )
+            )
+        finally:
+            loop.close()
+        
+        # result is a tuple: (updated_history, empty_string, reasoning_data, performance_data, context_data, session_id, skills_html)
+        updated_history, _, reasoning_data, performance_data, context_data, returned_session_id, skills_html = result
+        
+        # Extract assistant response from history (last message)
+        response = ""
+        if updated_history and len(updated_history) > 0:
+            last_entry = updated_history[-1]
+            if isinstance(last_entry, dict):
+                response = last_entry.get('content', '')
+            elif isinstance(last_entry, tuple) and len(last_entry) >= 2:
+                response = last_entry[1]  # (user, assistant) format
+        
+        return jsonify({
+            'response': response,
+            'history': updated_history,
+            'reasoning': reasoning_data or {},
+            'performance': performance_data or {},
+            'context': context_data or {},
+            'session_id': returned_session_id or session_id,
+            'skills': skills_html or ''
+        })
+        
+    except Exception as e:
+        logger.error(f"Error in chat endpoint: {e}", exc_info=True)
+        return jsonify({
+            'error': str(e),
+            'response': 'I apologize, but I encountered an error processing your request. Please try again.'
+        }), 500
+
+@app.route('/api/orchestrator/status', methods=['GET'])
+def orchestrator_status():
+    """Check orchestrator status"""
+    return jsonify({
+        'orchestrator_available': _orchestrator is not None,
+        'orchestrator_type': type(_orchestrator).__name__ if _orchestrator else None
+    })
+
+if __name__ == '__main__':
+    port = int(os.getenv('FLASK_PORT', 5001))
+    host = os.getenv('FLASK_HOST', '0.0.0.0')
+    
+    logger.info(f"Starting Flask API on port {port}")
+    logger.info(f"Health check: http://{host}:{port}/health")
+    logger.info(f"Chat endpoint: http://{host}:{port}/api/chat")
+    
+    app.run(host=host, port=port, debug=False)
+

requirements.txt

@@ -66,6 +66,10 @@ tzdata>=2023.3
 ujson>=5.8.0
 orjson>=3.9.0
 
+# Flask API for external integrations
+flask>=3.0.0
+flask-cors>=4.0.0
+
 # HF Spaces Specific Dependencies
 # Note: huggingface-cli is part of huggingface-hub (installed by SDK)
 gradio-client>=0.8.0