#streaming

Explore how to build local multi-endpoint ML APIs with LitServe, showcasing batching, streaming, caching and multi-task inference with Hugging Face pipelines.

'A practical Colab walkthrough to install Ollama, pull lightweight models, stream token outputs via /api/chat, and build a Gradio chat interface for interactive multi-turn testing.'