Merge branch 'master' into 'main'

commit README.md

See merge request ul-dsri/sandbox/sachin-sharma-in/ml-inference-service!2

README.md

@@ -1,93 +1,264 @@
-# ml-Inference-service
+# ML Inference Service (FastAPI)
 
+A production-ready **FastAPI** web service that serves **image classification** models.  
+This repo ships with a working example using **ResNet-18** (downloaded from Hugging Face) under `models/resnet-18/` and exposes a simple **REST** endpoint.
 
+---
 
-## Getting started
+## ✨ What you get
 
-To make it easy for you to get started with GitLab, here's a list of recommended next steps.
+- FastAPI application with clean layering (routes → controller → service)
+- Hot-loaded model on startup (single instance reused per request)
+- Hugging Face–compatible local model folder (`config.json`, weights, preprocessor, etc.)
+- Example endpoint: `POST /predict/resnet` that accepts a base64 image and returns:
+  - `prediction` (class label)
+  - `confidence` (softmax probability)
+  - `predicted_label` (class index)
+  - `model` (model id)
+  - `mediaType` (echoed)
 
-Already a pro? Just edit this README.md and make it your own. Want to make it easy? [Use the template at the bottom](#editing-this-readme)!
+---
 
-## Add your files
-
-- [ ] [Create](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#create-a-file) or [upload](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#upload-a-file) files
-- [ ] [Add files using the command line](https://docs.gitlab.com/topics/git/add_files/#add-files-to-a-git-repository) or push an existing Git repository with the following command:
+## 🧭 Project Layout
 
 ```
-cd existing_repo
-git remote add origin https://gitlab.com/ul-dsri/sandbox/sachin-sharma-in/ml-inference-service.git
-git branch -M main
-git push -uf origin main
+ml-inference-service/
+├─ main.py
+├─ app/
+│  ├─ __init__.py
+│  ├─ core/
+│  │  ├─ app.py               # App factory & router wiring
+│  │  ├─ config.py            # Settings (app name/version/debug)
+│  │  ├─ dependencies.py      # DI for model services
+│  │  ├─ lifespan.py          # Startup: load model & register service
+│  │  └─ logging.py           # Logger setup
+│  ├─ api/
+│  │  ├─ models.py            # Pydantic request/response
+│  │  ├─ controllers.py       # HTTP → service orchestration
+│  │  └─ routes/
+│  │     ├─ prediction.py     # `POST /predict/resnet`
+│  │     └─ resnet_service_manager.py (legacy, unused)
+│  └─ services/
+│     └─ inference.py         # ResNetInferenceService (load/predict)
+├─ models/
+│  └─ resnet-18/              # Sample HF-style model folder
+├─ scripts/
+│  └─ model_download.bash     # One-liner to snapshot HF weights locally
+├─ requirements.in / requirements.txt
+└─ test_main.http             # Example request you can run from IDEs
 ```
 
-## Integrate with your tools
-
-- [ ] [Set up project integrations](https://gitlab.com/ul-dsri/sandbox/sachin-sharma-in/ml-inference-service/-/settings/integrations)
-
-## Collaborate with your team
-
-- [ ] [Invite team members and collaborators](https://docs.gitlab.com/ee/user/project/members/)
-- [ ] [Create a new merge request](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html)
-- [ ] [Automatically close issues from merge requests](https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically)
-- [ ] [Enable merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/approvals/)
-- [ ] [Set auto-merge](https://docs.gitlab.com/user/project/merge_requests/auto_merge/)
+---
 
-## Test and Deploy
+## 🚀 Quickstart
 
-Use the built-in continuous integration in GitLab.
-
-- [ ] [Get started with GitLab CI/CD](https://docs.gitlab.com/ee/ci/quick_start/)
-- [ ] [Analyze your code for known vulnerabilities with Static Application Security Testing (SAST)](https://docs.gitlab.com/ee/user/application_security/sast/)
-- [ ] [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/ee/topics/autodevops/requirements.html)
-- [ ] [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/ee/user/clusters/agent/)
-- [ ] [Set up protected environments](https://docs.gitlab.com/ee/ci/environments/protected_environments.html)
+### 1) Install dependencies (Python 3.9+)
+```bash
+python -m venv .venv
+source .venv/bin/activate   # Windows: .venv\Scripts\activate
+pip install -r requirements.txt
+```
 
-***
+### 2) Download the sample model (ResNet‑18) locally
+```bash
+bash scripts/model_download.bash
+```
+This populates `models/resnet-18/` with Hugging Face artifacts (`config.json`, weights, `preprocessor_config.json`, etc.).
 
-# Editing this README
+### 3) Run the server
+```bash
+uvicorn main:app --reload
+```
+Server listens on `http://127.0.0.1:8000`.
 
-When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!). Thanks to [makeareadme.com](https://www.makeareadme.com/) for this template.
+### 4) Call the API
+- Use `test_main.http` from your IDE (VSCode/IntelliJ) **or** curl:
 
-## Suggestions for a good README
+```bash
+curl -X POST http://127.0.0.1:8000/predict/resnet   -H "Content-Type: application/json"   -d '{
+    "image": { "mediaType": "image/jpeg", "data": "<base64-encoded-bytes>" }
+  }'
+```
 
-Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.
+**Response (example):**
+```json
+{
+  "prediction": "tiger cat",
+  "confidence": 0.9971,
+  "predicted_label": 282,
+  "model": "microsoft/resnet-18",
+  "mediaType": "image/jpeg"
+}
+```
 
-## Name
-Choose a self-explaining name for your project.
+---
+
+## 🧩 Bring Your Own Model (BYOM)
+
+There are **two** ways to integrate your own model.
+
+### Option A — *Drop-in replacement (zero code changes)*
+
+If your model is a **Hugging Face image classification** model that works with
+`AutoImageProcessor` and `ResNetForImageClassification` **or** a compatible
+`*ForImageClassification` class from `transformers`, you can simply place the
+model folder alongside `resnet-18` and point the service at it.
+
+1. Put your HF-style folder under `models/<your-model-name>/` containing at least:
+   - `config.json`
+   - weights (e.g., `pytorch_model.bin` or `model.safetensors`)
+   - `preprocessor_config.json` / `image_processor` files
+
+2. **Choose one** of these approaches:
+   - **Simplest**: Replace the contents of `models/resnet-18/` with your model files *but keep the folder name*. The existing `/predict/resnet` endpoint will now serve your model.
+   - **Preferred**: Change the model id used at startup:
+     - Open `app/core/lifespan.py` and modify the service initialization:
+       ```python
+       resnet_service = ResNetInferenceService(
+           model_name="your-org/your-model",  # used for local folder name
+           use_local_model=True               # loads from models/your-model/
+       )
+       ```
+     - Ensure your local folder is `models/your-model/`.
+
+> How folder naming works: when `use_local_model=True`, the service derives the
+> local directory as `models/<last-segment-of-model_name>`. For
+> `"microsoft/resnet-18"` that becomes `models/resnet-18`. For
+> `"your-org/awesome-vit-base"`, it becomes `models/awesome-vit-base`.
+
+That’s it. No code changes elsewhere if your model is a standard image classifier.
+
+---
+
+### Option B — *New task/model type (minimal code: new service + route)*
+
+If you are **not** serving a Hugging Face image classifier (e.g., object detection,
+segmentation, text models), implement a small service class and a route mirroring
+the `ResNetInferenceService` flow.
+
+1. **Create your service** (copy and adapt `ResNetInferenceService`):
+   - File: `app/services/<your_model>_service.py`
+   - Responsibilities you must implement:
+     - `__init__(model_name: str, use_local_model: bool)` → set `self.model_path`
+     - `load_model()` → load weights & preprocessor
+     - `predict(image: PIL.Image.Image) -> Dict[str, Any]` → run inference and return a dict with:
+       ```python
+       {
+         "prediction": "<your label or structured result>",
+         "confidence": <float 0..1>,
+         "predicted_label": <int or meaningful code>,
+         "model": "<model id>"
+       }
+       ```
+       *Feel free to extend the payload; just update the API schema accordingly.*
+
+2. **Wire the dependency**:
+   - Register your service at startup in `app/core/lifespan.py` similar to ResNet:
+     ```python
+     from app.core.dependencies import set_resnet_service  # or create your own set/get
+     from app.services.your_model_service import YourModelService
+
+     svc = YourModelService(model_name="your-org/your-model", use_local_model=True)
+     svc.load_model()
+     set_resnet_service(svc)  # or create set_your_model_service(...)
+     ```
+   - Optionally create **new getters/setters** in `app/core/dependencies.py` if you serve multiple models in parallel (one getter per model).
+
+3. **Add a route**:
+   - Create `app/api/routes/your_model.py` analogous to `prediction.py`:
+     ```python
+     from fastapi import APIRouter, Depends
+     from app.api.controllers import PredictionController
+     from app.api.models import ImageRequest, PredictionResponse
+     from app.core.dependencies import get_resnet_service  # or your getter
+     from app.services.your_model_service import YourModelService
+
+     router = APIRouter()
+
+     @router.post("/predict/your-model", response_model=PredictionResponse)
+     async def predict_image(request: ImageRequest, service: YourModelService = Depends(get_resnet_service)):
+         controller = PredictionController(service)  # reuse the controller
+         return await controller.predict(request)
+     ```
+   - Register the router in `app/core/app.py`:
+     ```python
+     from app.api.routes import your_model as your_model_routes
+     app.include_router(your_model_routes.router)
+     ```
+
+4. **Adjust schemas if needed**:
+   - The default `PredictionResponse` in `app/api/models.py` is for single-label classification. For other tasks, either extend it or define a new response model and use it in your route’s `response_model=`.
+
+> **Tip**: Keep your controller thin and push all model-specific logic into your service class. The server glue (DI + routes) stays identical across models.
+
+---
+
+## 🧪 Validating your setup
+
+- **Startup logs** should include: `Initializing ResNet service with local model: models/<folder>` and `Model and processor loaded successfully`.
+- Hitting your endpoint should return a **200** with a JSON body like the example above.
+- If you see `Local model directory not found`, check your `models/<name>/` path and filenames.
+
+---
+
+## 🔌 Request & Response Shapes
+
+### Request
+```json
+{
+  "image": {
+    "mediaType": "image/jpeg",
+    "data": "<base64-encoded image bytes>"
+  }
+}
+```
 
-## Description
-Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.
+### Response
+```json
+{
+  "prediction": "string label",
+  "confidence": 0.0,
+  "predicted_label": 0,
+  "model": "your-org/your-model",
+  "mediaType": "image/jpeg"
+}
+```
 
-## Badges
-On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.
+---
 
-## Visuals
-Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.
+## ⚙️ Configuration
 
-## Installation
-Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.
+Basic settings live in `app/core/config.py`. Out of the box we keep it simple:
+- `app_name`, `app_version`, `debug`
 
-## Usage
-Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.
+If you want to make the **model** configurable without touching code, extend `Settings` with a `model_name` env var and consume it in `lifespan.py` when creating your service instance.
 
-## Support
-Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.
+Example:
+```python
+# app/core/config.py
+from pydantic_settings import BaseSettings
+from pydantic import Field
 
-## Roadmap
-If you have ideas for releases in the future, it is a good idea to list them in the README.
+class Settings(BaseSettings):
+    app_name: str = Field("ML Inference Service")
+    app_version: str = Field("0.1.0")
+    debug: bool = Field(False)
+    model_name: str = Field("microsoft/resnet-18", description="HF model id used at startup")
 
-## Contributing
-State if you are open to contributions and what your requirements are for accepting them.
+settings = Settings()
 
-For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.
+# app/core/lifespan.py
+from app.core.config import settings
+svc = ResNetInferenceService(model_name=settings.model_name, use_local_model=True)
+```
 
-You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.
+Then set `MODEL_NAME=your-org/your-model` in your environment (Pydantic will map `model_name` from `MODEL_NAME`).
 
-## Authors and acknowledgment
-Show your appreciation to those who have contributed to the project.
+---
 
-## License
-For open source projects, say how it is licensed.
+## 📦 Packaging & Deployment
 
-## Project status
-If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.
+- **Dev**: `uvicorn main:app --reload`
+- **Prod**: Use a process manager (e.g., `gunicorn -k uvicorn.workers.UvicornWorker`) and add health checks.
+- **Containerize**: Copy only `requirements.txt` and source, install wheels, and bake the `models/` folder into the image or mount it as a volume.
+- **CPU vs GPU**: This example uses CPU by default. If you have CUDA, install a CUDA-enabled PyTorch build and set device placement in your service.

app/core/config.py

@@ -4,7 +4,6 @@ Basic configuration management.
 Starting simple - just app settings. We'll expand as needed.
 """
 
-from typing import Optional
 from pydantic import Field
 from pydantic_settings import BaseSettings  # Changed import