Merge pull request #1654 from open-webui/dev

0.1.121

.github/ISSUE_TEMPLATE/bug_report.md

@@ -24,6 +24,9 @@ assignees: ''
 
 ## Environment
 
+- **Open WebUI Version:** [e.g., 0.1.120]
+- **Ollama (if applicable):** [e.g., 0.1.30, 0.1.32-rc1]
+
 - **Operating System:** [e.g., Windows 10, macOS Big Sur, Ubuntu 20.04]
 - **Browser (if applicable):** [e.g., Chrome 100.0, Firefox 98.0]
 

CHANGELOG.md

@@ -5,6 +5,35 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.121] - 2024-04-24
+
+### Fixed
+
+- **🔧 Translation Issues**: Addressed various translation discrepancies.
+- **🔒 LiteLLM Security Fix**: Updated LiteLLM version to resolve a security vulnerability.
+- **🖥️ HTML Tag Display**: Rectified the issue where the '< br >' tag wasn't displaying correctly.
+- **🔗 WebSocket Connection**: Resolved the failure of WebSocket connection under HTTPS security for ComfyUI server.
+- **📜 FileReader Optimization**: Implemented FileReader initialization per image in multi-file drag & drop to ensure reusability.
+- **🏷️ Tag Display**: Corrected tag display inconsistencies.
+- **📦 Archived Chat Styling**: Fixed styling issues in archived chat.
+- **🔖 Safari Copy Button Bug**: Addressed the bug where the copy button failed to copy links in Safari.
+
+## [0.1.120] - 2024-04-20
+
+### Added
+
+- **📦 Archive Chat Feature**: Easily archive chats with a new sidebar button, and access archived chats via the profile button > archived chats.
+- **🔊 Configurable Text-to-Speech Endpoint**: Customize your Text-to-Speech experience with configurable OpenAI endpoints.
+- **🛠️ Improved Error Handling**: Enhanced error message handling for connection failures.
+- **⌨️ Enhanced Shortcut**: When editing messages, use ctrl/cmd+enter to save and submit, and esc to close.
+- **🌐 Language Support**: Added support for Georgian and enhanced translations for Portuguese and Vietnamese.
+
+### Fixed
+
+- **🔧 Model Selector**: Resolved issue where default model selection was not saving.
+- **🔗 Share Link Copy Button**: Fixed bug where the copy button wasn't copying links in Safari.
+- **🎨 Light Theme Styling**: Addressed styling issue with the light theme.
+
 ## [0.1.119] - 2024-04-16
 
 ### Added

Dockerfile

@@ -8,8 +8,8 @@ ARG USE_CUDA_VER=cu121
 # any sentence transformer model; models to use can be found at https://huggingface.co/models?library=sentence-transformers
 # Leaderboard: https://huggingface.co/spaces/mteb/leaderboard 
 # for better performance and multilangauge support use "intfloat/multilingual-e5-large" (~2.5GB) or "intfloat/multilingual-e5-base" (~1.5GB)
-# IMPORTANT: If you change the default model (all-MiniLM-L6-v2) and vice versa, you aren't able to use RAG Chat with your previous documents loaded in the WebUI! You need to re-embed them.
+# IMPORTANT: If you change the default model (sentence-transformers/all-MiniLM-L6-v2) and vice versa, you aren't able to use RAG Chat with your previous documents loaded in the WebUI! You need to re-embed them.
-ARG USE_EMBEDDING_MODEL=all-MiniLM-L6-v2
+ARG USE_EMBEDDING_MODEL=sentence-transformers/all-MiniLM-L6-v2
 
 ######## WebUI frontend ########
 FROM --platform=$BUILDPLATFORM node:21-alpine3.19 as build
@@ -98,13 +98,13 @@ RUN pip3 install uv && \
         # If you use CUDA the whisper and embedding model will be downloaded on first use
         pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/$USE_CUDA_DOCKER_VER --no-cache-dir && \
         uv pip install --system -r requirements.txt --no-cache-dir && \
-        python -c "import os; from faster_whisper import WhisperModel; WhisperModel(os.environ['WHISPER_MODEL'], device='cpu', compute_type='int8', download_root=os.environ['WHISPER_MODEL_DIR'])" && \
+        python -c "import os; from sentence_transformers import SentenceTransformer; SentenceTransformer(os.environ['RAG_EMBEDDING_MODEL'], device='cpu')" && \
-        python -c "import os; from chromadb.utils import embedding_functions; sentence_transformer_ef = embedding_functions.SentenceTransformerEmbeddingFunction(model_name=os.environ['RAG_EMBEDDING_MODEL'], device='cpu')"; \
+        python -c "import os; from faster_whisper import WhisperModel; WhisperModel(os.environ['WHISPER_MODEL'], device='cpu', compute_type='int8', download_root=os.environ['WHISPER_MODEL_DIR'])"; \
     else \
         pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu --no-cache-dir && \
         uv pip install --system -r requirements.txt --no-cache-dir && \
-        python -c "import os; from faster_whisper import WhisperModel; WhisperModel(os.environ['WHISPER_MODEL'], device='cpu', compute_type='int8', download_root=os.environ['WHISPER_MODEL_DIR'])" && \
+        python -c "import os; from sentence_transformers import SentenceTransformer; SentenceTransformer(os.environ['RAG_EMBEDDING_MODEL'], device='cpu')" && \
-        python -c "import os; from chromadb.utils import embedding_functions; sentence_transformer_ef = embedding_functions.SentenceTransformerEmbeddingFunction(model_name=os.environ['RAG_EMBEDDING_MODEL'], device='cpu')"; \
+        python -c "import os; from faster_whisper import WhisperModel; WhisperModel(os.environ['WHISPER_MODEL'], device='cpu', compute_type='int8', download_root=os.environ['WHISPER_MODEL_DIR'])"; \
     fi
 
 

README.md

@@ -25,22 +25,28 @@ Open WebUI is an extensible, feature-rich, and user-friendly self-hosted WebUI d
 
 - 🚀 **Effortless Setup**: Install seamlessly using Docker or Kubernetes (kubectl, kustomize or helm) for a hassle-free experience.
 
+- 🌈 **Theme Customization**: Choose from a variety of themes to personalize your Open WebUI experience.
+
 - 💻 **Code Syntax Highlighting**: Enjoy enhanced code readability with our syntax highlighting feature.
 
 - ✒️🔢 **Full Markdown and LaTeX Support**: Elevate your LLM experience with comprehensive Markdown and LaTeX capabilities for enriched interaction.
 
 - 📚 **Local RAG Integration**: Dive into the future of chat interactions with the groundbreaking Retrieval Augmented Generation (RAG) support. This feature seamlessly integrates document interactions into your chat experience. You can load documents directly into the chat or add files to your document library, effortlessly accessing them using `#` command in the prompt. In its alpha phase, occasional issues may arise as we actively refine and enhance this feature to ensure optimal performance and reliability.
 
+- 🔍 **RAG Embedding Support**: Change the RAG embedding model directly in document settings, enhancing document processing. This feature supports Ollama and OpenAI models.
+
 - 🌐 **Web Browsing Capability**: Seamlessly integrate websites into your chat experience using the `#` command followed by the URL. This feature allows you to incorporate web content directly into your conversations, enhancing the richness and depth of your interactions.
 
 - 📜 **Prompt Preset Support**: Instantly access preset prompts using the `/` command in the chat input. Load predefined conversation starters effortlessly and expedite your interactions. Effortlessly import prompts through [Open WebUI Community](https://openwebui.com/) integration.
 
-- 👍👎 **RLHF Annotation**: Empower your messages by rating them with thumbs up and thumbs down, facilitating the creation of datasets for Reinforcement Learning from Human Feedback (RLHF). Utilize your messages to train or fine-tune models, all while ensuring the confidentiality of locally saved data.
+- 👍👎 **RLHF Annotation**: Empower your messages by rating them with thumbs up and thumbs down, followed by the option to provide textual feedback, facilitating the creation of datasets for Reinforcement Learning from Human Feedback (RLHF). Utilize your messages to train or fine-tune models, all while ensuring the confidentiality of locally saved data.
 
 - 🏷️ **Conversation Tagging**: Effortlessly categorize and locate specific chats for quick reference and streamlined data collection.
 
 - 📥🗑️ **Download/Delete Models**: Easily download or remove models directly from the web UI.
 
+- 🔄 **Update All Ollama Models**: Easily update locally installed models all at once with a convenient button, streamlining model management.
+
 - ⬆️ **GGUF File Model Creation**: Effortlessly create Ollama models by uploading GGUF files directly from the web UI. Streamlined process with options to upload from your machine or download GGUF files from Hugging Face.
 
 - 🤖 **Multiple Model Support**: Seamlessly switch between different chat models for diverse interactions.
@@ -53,28 +59,42 @@ Open WebUI is an extensible, feature-rich, and user-friendly self-hosted WebUI d
 
 - 💬 **Collaborative Chat**: Harness the collective intelligence of multiple models by seamlessly orchestrating group conversations. Use the `@` command to specify the model, enabling dynamic and diverse dialogues within your chat interface. Immerse yourself in the collective intelligence woven into your chat environment.
 
+- 🗨️ **Local Chat Sharing**: Generate and share chat links seamlessly between users, enhancing collaboration and communication.
+
 - 🔄 **Regeneration History Access**: Easily revisit and explore your entire regeneration history.
 
 - 📜 **Chat History**: Effortlessly access and manage your conversation history.
 
+- 📬 **Archive Chats**: Effortlessly store away completed conversations with LLMs for future reference, maintaining a tidy and clutter-free chat interface while allowing for easy retrieval and reference.
+
 - 📤📥 **Import/Export Chat History**: Seamlessly move your chat data in and out of the platform.
 
 - 🗣️ **Voice Input Support**: Engage with your model through voice interactions; enjoy the convenience of talking to your model directly. Additionally, explore the option for sending voice input automatically after 3 seconds of silence for a streamlined experience.
 
+- 🔊 **Configurable Text-to-Speech Endpoint**: Customize your Text-to-Speech experience with configurable OpenAI endpoints.
+
 - ⚙️ **Fine-Tuned Control with Advanced Parameters**: Gain a deeper level of control by adjusting parameters such as temperature and defining your system prompts to tailor the conversation to your specific preferences and needs.
 
-- 🎨🤖 **Image Generation Integration**: Seamlessly incorporate image generation capabilities using AUTOMATIC1111 API (local) and DALL-E, enriching your chat experience with dynamic visual content.
+- 🎨🤖 **Image Generation Integration**: Seamlessly incorporate image generation capabilities using options such as AUTOMATIC1111 API (local), ComfyUI (local), and DALL-E, enriching your chat experience with dynamic visual content.
 
 - 🤝 **OpenAI API Integration**: Effortlessly integrate OpenAI-compatible API for versatile conversations alongside Ollama models. Customize the API Base URL to link with **LMStudio, Mistral, OpenRouter, and more**.
 
 - ✨ **Multiple OpenAI-Compatible API Support**: Seamlessly integrate and customize various OpenAI-compatible APIs, enhancing the versatility of your chat interactions.
 
+- 🔑 **API Key Generation Support**: Generate secret keys to leverage Open WebUI with OpenAI libraries, simplifying integration and development.
+
 - 🔗 **External Ollama Server Connection**: Seamlessly link to an external Ollama server hosted on a different address by configuring the environment variable.
 
 - 🔀 **Multiple Ollama Instance Load Balancing**: Effortlessly distribute chat requests across multiple Ollama instances for enhanced performance and reliability.
 
 - 👥 **Multi-User Management**: Easily oversee and administer users via our intuitive admin panel, streamlining user management processes.
 
+- 🔗 **Webhook Integration**: Subscribe to new user sign-up events via webhook (compatible with Google Chat and Microsoft Teams), providing real-time notifications and automation capabilities.
+
+- 🛡️ **Model Whitelisting**: Admins can whitelist models for users with the 'user' role, enhancing security and access control.
+
+- 📧 **Trusted Email Authentication**: Authenticate using a trusted email header, adding an additional layer of security and authentication.
+
 - 🔐 **Role-Based Access Control (RBAC)**: Ensure secure access with restricted permissions; only authorized individuals can access your Ollama, and exclusive model creation/pulling rights are reserved for administrators.
 
 - 🔒 **Backend Reverse Proxy Support**: Bolster security through direct communication between Open WebUI backend and Ollama. This key feature eliminates the need to expose Ollama over LAN. Requests made to the '/ollama/api' route from the web UI are seamlessly redirected to Ollama from the backend, enhancing overall system security.
@@ -185,4 +205,4 @@ If you have any questions, suggestions, or need assistance, please open an issue
 
 ---
 
-Created by [Timothy J. Baek](https://github.com/tjbck) - Let's make Open Web UI even more amazing together! 💪
+Created by [Timothy J. Baek](https://github.com/tjbck) - Let's make Open WebUI even more amazing together! 💪

backend/apps/audio/main.py

@@ -10,8 +10,19 @@ from fastapi import (
     File,
     Form,
 )
+
+from fastapi.responses import StreamingResponse, JSONResponse, FileResponse
+
 from fastapi.middleware.cors import CORSMiddleware
 from faster_whisper import WhisperModel
+from pydantic import BaseModel
+
+
+import requests
+import hashlib
+from pathlib import Path
+import json
+
 
 from constants import ERROR_MESSAGES
 from utils.utils import (
@@ -30,6 +41,8 @@ from config import (
     WHISPER_MODEL_DIR,
     WHISPER_MODEL_AUTO_UPDATE,
     DEVICE_TYPE,
+    AUDIO_OPENAI_API_BASE_URL,
+    AUDIO_OPENAI_API_KEY,
 )
 
 log = logging.getLogger(__name__)
@@ -44,12 +57,104 @@ app.add_middleware(
     allow_headers=["*"],
 )
 
+
+app.state.OPENAI_API_BASE_URL = AUDIO_OPENAI_API_BASE_URL
+app.state.OPENAI_API_KEY = AUDIO_OPENAI_API_KEY
+
 # setting device type for whisper model
 whisper_device_type = DEVICE_TYPE if DEVICE_TYPE and DEVICE_TYPE == "cuda" else "cpu"
 log.info(f"whisper_device_type: {whisper_device_type}")
 
+SPEECH_CACHE_DIR = Path(CACHE_DIR).joinpath("./audio/speech/")
+SPEECH_CACHE_DIR.mkdir(parents=True, exist_ok=True)
 
-@app.post("/transcribe")
+
+class OpenAIConfigUpdateForm(BaseModel):
+    url: str
+    key: str
+
+
+@app.get("/config")
+async def get_openai_config(user=Depends(get_admin_user)):
+    return {
+        "OPENAI_API_BASE_URL": app.state.OPENAI_API_BASE_URL,
+        "OPENAI_API_KEY": app.state.OPENAI_API_KEY,
+    }
+
+
+@app.post("/config/update")
+async def update_openai_config(
+    form_data: OpenAIConfigUpdateForm, user=Depends(get_admin_user)
+):
+    if form_data.key == "":
+        raise HTTPException(status_code=400, detail=ERROR_MESSAGES.API_KEY_NOT_FOUND)
+
+    app.state.OPENAI_API_BASE_URL = form_data.url
+    app.state.OPENAI_API_KEY = form_data.key
+
+    return {
+        "status": True,
+        "OPENAI_API_BASE_URL": app.state.OPENAI_API_BASE_URL,
+        "OPENAI_API_KEY": app.state.OPENAI_API_KEY,
+    }
+
+
+@app.post("/speech")
+async def speech(request: Request, user=Depends(get_verified_user)):
+    body = await request.body()
+    name = hashlib.sha256(body).hexdigest()
+
+    file_path = SPEECH_CACHE_DIR.joinpath(f"{name}.mp3")
+    file_body_path = SPEECH_CACHE_DIR.joinpath(f"{name}.json")
+
+    # Check if the file already exists in the cache
+    if file_path.is_file():
+        return FileResponse(file_path)
+
+    headers = {}
+    headers["Authorization"] = f"Bearer {app.state.OPENAI_API_KEY}"
+    headers["Content-Type"] = "application/json"
+
+    r = None
+    try:
+        r = requests.post(
+            url=f"{app.state.OPENAI_API_BASE_URL}/audio/speech",
+            data=body,
+            headers=headers,
+            stream=True,
+        )
+
+        r.raise_for_status()
+
+        # Save the streaming content to a file
+        with open(file_path, "wb") as f:
+            for chunk in r.iter_content(chunk_size=8192):
+                f.write(chunk)
+
+        with open(file_body_path, "w") as f:
+            json.dump(json.loads(body.decode("utf-8")), f)
+
+        # Return the saved file
+        return FileResponse(file_path)
+
+    except Exception as e:
+        log.exception(e)
+        error_detail = "Open WebUI: Server Connection Error"
+        if r is not None:
+            try:
+                res = r.json()
+                if "error" in res:
+                    error_detail = f"External: {res['error']['message']}"
+            except:
+                error_detail = f"External: {e}"
+
+        raise HTTPException(
+            status_code=r.status_code if r != None else 500,
+            detail=error_detail,
+        )
+
+
+@app.post("/transcriptions")
 def transcribe(
     file: UploadFile = File(...),
     user=Depends(get_current_user),

backend/apps/images/main.py

@@ -35,6 +35,8 @@ from config import (
     ENABLE_IMAGE_GENERATION,
     AUTOMATIC1111_BASE_URL,
     COMFYUI_BASE_URL,
+    IMAGES_OPENAI_API_BASE_URL,
+    IMAGES_OPENAI_API_KEY,
 )
 
 
@@ -56,7 +58,9 @@ app.add_middleware(
 app.state.ENGINE = ""
 app.state.ENABLED = ENABLE_IMAGE_GENERATION
 
-app.state.OPENAI_API_KEY = ""
+app.state.OPENAI_API_BASE_URL = IMAGES_OPENAI_API_BASE_URL
+app.state.OPENAI_API_KEY = IMAGES_OPENAI_API_KEY
+
 app.state.MODEL = ""
 
 
@@ -131,27 +135,33 @@ async def update_engine_url(
     }
 
 
-class OpenAIKeyUpdateForm(BaseModel):
+class OpenAIConfigUpdateForm(BaseModel):
+    url: str
     key: str
 
 
-@app.get("/key")
+@app.get("/openai/config")
-async def get_openai_key(user=Depends(get_admin_user)):
+async def get_openai_config(user=Depends(get_admin_user)):
-    return {"OPENAI_API_KEY": app.state.OPENAI_API_KEY}
+    return {
+        "OPENAI_API_BASE_URL": app.state.OPENAI_API_BASE_URL,
+        "OPENAI_API_KEY": app.state.OPENAI_API_KEY,
+    }
 
 
-@app.post("/key/update")
+@app.post("/openai/config/update")
-async def update_openai_key(
+async def update_openai_config(
-    form_data: OpenAIKeyUpdateForm, user=Depends(get_admin_user)
+    form_data: OpenAIConfigUpdateForm, user=Depends(get_admin_user)
 ):
-
     if form_data.key == "":
         raise HTTPException(status_code=400, detail=ERROR_MESSAGES.API_KEY_NOT_FOUND)
 
+    app.state.OPENAI_API_BASE_URL = form_data.url
     app.state.OPENAI_API_KEY = form_data.key
+
     return {
-        "OPENAI_API_KEY": app.state.OPENAI_API_KEY,
         "status": True,
+        "OPENAI_API_BASE_URL": app.state.OPENAI_API_BASE_URL,
+        "OPENAI_API_KEY": app.state.OPENAI_API_KEY,
     }
 
 
@@ -360,7 +370,7 @@ def generate_image(
             }
 
             r = requests.post(
-                url=f"https://api.openai.com/v1/images/generations",
+                url=f"{app.state.OPENAI_API_BASE_URL}/images/generations",
                 json=data,
                 headers=headers,
             )

backend/apps/images/utils/comfyui.py

@@ -195,7 +195,7 @@ class ImageGenerationPayload(BaseModel):
 def comfyui_generate_image(
     model: str, payload: ImageGenerationPayload, client_id, base_url
 ):
-    host = base_url.replace("http://", "").replace("https://", "")
+    ws_url = base_url.replace("http://", "ws://").replace("https://", "wss://")
 
     comfyui_prompt = json.loads(COMFYUI_DEFAULT_PROMPT)
 
@@ -217,7 +217,7 @@ def comfyui_generate_image(
 
     try:
         ws = websocket.WebSocket()
-        ws.connect(f"ws://{host}/ws?clientId={client_id}")
+        ws.connect(f"{ws_url}/ws?clientId={client_id}")
         log.info("WebSocket connection established.")
     except Exception as e:
         log.exception(f"Failed to connect to WebSocket server: {e}")

backend/apps/litellm/main.py

@@ -1,17 +1,25 @@
+import sys
+
+from fastapi import FastAPI, Depends, HTTPException
+from fastapi.routing import APIRoute
+from fastapi.middleware.cors import CORSMiddleware
+
 import logging
-
-from litellm.proxy.proxy_server import ProxyConfig, initialize
-from litellm.proxy.proxy_server import app
-
 from fastapi import FastAPI, Request, Depends, status, Response
 from fastapi.responses import JSONResponse
 
 from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint
 from starlette.responses import StreamingResponse
 import json
+import time
+import requests
 
-from utils.utils import get_http_authorization_cred, get_current_user
+from pydantic import BaseModel, ConfigDict
+from typing import Optional, List
+
+from utils.utils import get_verified_user, get_current_user, get_admin_user
 from config import SRC_LOG_LEVELS, ENV
+from constants import MESSAGES
 
 log = logging.getLogger(__name__)
 log.setLevel(SRC_LOG_LEVELS["LITELLM"])
@@ -20,81 +28,324 @@ log.setLevel(SRC_LOG_LEVELS["LITELLM"])
 from config import (
     MODEL_FILTER_ENABLED,
     MODEL_FILTER_LIST,
+    DATA_DIR,
+    LITELLM_PROXY_PORT,
+    LITELLM_PROXY_HOST,
+)
+
+from litellm.utils import get_llm_provider
+
+import asyncio
+import subprocess
+import yaml
+
+app = FastAPI()
+
+origins = ["*"]
+
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=origins,
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
 )
 
 
-proxy_config = ProxyConfig()
+LITELLM_CONFIG_DIR = f"{DATA_DIR}/litellm/config.yaml"
+
+with open(LITELLM_CONFIG_DIR, "r") as file:
+    litellm_config = yaml.safe_load(file)
+
+app.state.CONFIG = litellm_config
+
+# Global variable to store the subprocess reference
+background_process = None
 
 
-async def config():
+async def run_background_process(command):
-    router, model_list, general_settings = await proxy_config.load_config(
+    global background_process
-        router=None, config_file_path="./data/litellm/config.yaml"
+    log.info("run_background_process")
-    )
 
-    await initialize(config="./data/litellm/config.yaml", telemetry=False)
+    try:
+        # Log the command to be executed
+        log.info(f"Executing command: {command}")
+        # Execute the command and create a subprocess
+        process = await asyncio.create_subprocess_exec(
+            *command, stdout=subprocess.PIPE, stderr=subprocess.PIPE
+        )
+        background_process = process
+        log.info("Subprocess started successfully.")
+
+        # Capture STDERR for debugging purposes
+        stderr_output = await process.stderr.read()
+        stderr_text = stderr_output.decode().strip()
+        if stderr_text:
+            log.info(f"Subprocess STDERR: {stderr_text}")
+
+        # log.info output line by line
+        async for line in process.stdout:
+            log.info(line.decode().strip())
+
+        # Wait for the process to finish
+        returncode = await process.wait()
+        log.info(f"Subprocess exited with return code {returncode}")
+    except Exception as e:
+        log.error(f"Failed to start subprocess: {e}")
+        raise  # Optionally re-raise the exception if you want it to propagate
 
 
-async def startup():
+async def start_litellm_background():
-    await config()
+    log.info("start_litellm_background")
+    # Command to run in the background
+    command = [
+        "litellm",
+        "--port",
+        str(LITELLM_PROXY_PORT),
+        "--host",
+        LITELLM_PROXY_HOST,
+        "--telemetry",
+        "False",
+        "--config",
+        LITELLM_CONFIG_DIR,
+    ]
+
+    await run_background_process(command)
+
+
+async def shutdown_litellm_background():
+    log.info("shutdown_litellm_background")
+    global background_process
+    if background_process:
+        background_process.terminate()
+        await background_process.wait()  # Ensure the process has terminated
+        log.info("Subprocess terminated")
+        background_process = None
 
 
 @app.on_event("startup")
-async def on_startup():
+async def startup_event():
-    await startup()
+    log.info("startup_event")
+    # TODO: Check config.yaml file and create one
+    asyncio.create_task(start_litellm_background())
 
 
 app.state.MODEL_FILTER_ENABLED = MODEL_FILTER_ENABLED
 app.state.MODEL_FILTER_LIST = MODEL_FILTER_LIST
 
 
-@app.middleware("http")
+@app.get("/")
-async def auth_middleware(request: Request, call_next):
+async def get_status():
-    auth_header = request.headers.get("Authorization", "")
+    return {"status": True}
-    request.state.user = None
+
+
+async def restart_litellm():
+    """
+    Endpoint to restart the litellm background service.
+    """
+    log.info("Requested restart of litellm service.")
+    try:
+        # Shut down the existing process if it is running
+        await shutdown_litellm_background()
+        log.info("litellm service shutdown complete.")
+
+        # Restart the background service
+
+        asyncio.create_task(start_litellm_background())
+        log.info("litellm service restart complete.")
+
+        return {
+            "status": "success",
+            "message": "litellm service restarted successfully.",
+        }
+    except Exception as e:
+        log.info(f"Error restarting litellm service: {e}")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail=str(e)
+        )
+
+
+@app.get("/restart")
+async def restart_litellm_handler(user=Depends(get_admin_user)):
+    return await restart_litellm()
+
+
+@app.get("/config")
+async def get_config(user=Depends(get_admin_user)):
+    return app.state.CONFIG
+
+
+class LiteLLMConfigForm(BaseModel):
+    general_settings: Optional[dict] = None
+    litellm_settings: Optional[dict] = None
+    model_list: Optional[List[dict]] = None
+    router_settings: Optional[dict] = None
+
+    model_config = ConfigDict(protected_namespaces=())
+
+
+@app.post("/config/update")
+async def update_config(form_data: LiteLLMConfigForm, user=Depends(get_admin_user)):
+    app.state.CONFIG = form_data.model_dump(exclude_none=True)
+
+    with open(LITELLM_CONFIG_DIR, "w") as file:
+        yaml.dump(app.state.CONFIG, file)
+
+    await restart_litellm()
+    return app.state.CONFIG
+
+
+@app.get("/models")
+@app.get("/v1/models")
+async def get_models(user=Depends(get_current_user)):
+    while not background_process:
+        await asyncio.sleep(0.1)
+
+    url = f"http://localhost:{LITELLM_PROXY_PORT}/v1"
+    r = None
+    try:
+        r = requests.request(method="GET", url=f"{url}/models")
+        r.raise_for_status()
+
+        data = r.json()
+
+        if app.state.MODEL_FILTER_ENABLED:
+            if user and user.role == "user":
+                data["data"] = list(
+                    filter(
+                        lambda model: model["id"] in app.state.MODEL_FILTER_LIST,
+                        data["data"],
+                    )
+                )
+
+        return data
+    except Exception as e:
+
+        log.exception(e)
+        error_detail = "Open WebUI: Server Connection Error"
+        if r is not None:
+            try:
+                res = r.json()
+                if "error" in res:
+                    error_detail = f"External: {res['error']}"
+            except:
+                error_detail = f"External: {e}"
+
+        return {
+            "data": [
+                {
+                    "id": model["model_name"],
+                    "object": "model",
+                    "created": int(time.time()),
+                    "owned_by": "openai",
+                }
+                for model in app.state.CONFIG["model_list"]
+            ],
+            "object": "list",
+        }
+
+
+@app.get("/model/info")
+async def get_model_list(user=Depends(get_admin_user)):
+    return {"data": app.state.CONFIG["model_list"]}
+
+
+class AddLiteLLMModelForm(BaseModel):
+    model_name: str
+    litellm_params: dict
+
+    model_config = ConfigDict(protected_namespaces=())
+
+
+@app.post("/model/new")
+async def add_model_to_config(
+    form_data: AddLiteLLMModelForm, user=Depends(get_admin_user)
+):
+    try:
+        get_llm_provider(model=form_data.model_name)
+        app.state.CONFIG["model_list"].append(form_data.model_dump())
+
+        with open(LITELLM_CONFIG_DIR, "w") as file:
+            yaml.dump(app.state.CONFIG, file)
+
+        await restart_litellm()
+
+        return {"message": MESSAGES.MODEL_ADDED(form_data.model_name)}
+    except Exception as e:
+        print(e)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail=str(e)
+        )
+
+
+class DeleteLiteLLMModelForm(BaseModel):
+    id: str
+
+
+@app.post("/model/delete")
+async def delete_model_from_config(
+    form_data: DeleteLiteLLMModelForm, user=Depends(get_admin_user)
+):
+    app.state.CONFIG["model_list"] = [
+        model
+        for model in app.state.CONFIG["model_list"]
+        if model["model_name"] != form_data.id
+    ]
+
+    with open(LITELLM_CONFIG_DIR, "w") as file:
+        yaml.dump(app.state.CONFIG, file)
+
+    await restart_litellm()
+
+    return {"message": MESSAGES.MODEL_DELETED(form_data.id)}
+
+
+@app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
+async def proxy(path: str, request: Request, user=Depends(get_verified_user)):
+    body = await request.body()
+
+    url = f"http://localhost:{LITELLM_PROXY_PORT}"
+
+    target_url = f"{url}/{path}"
+
+    headers = {}
+    # headers["Authorization"] = f"Bearer {key}"
+    headers["Content-Type"] = "application/json"
+
+    r = None
 
     try:
-        user = get_current_user(get_http_authorization_cred(auth_header))
+        r = requests.request(
-        log.debug(f"user: {user}")
+            method=request.method,
-        request.state.user = user
+            url=target_url,
+            data=body,
+            headers=headers,
+            stream=True,
+        )
+
+        r.raise_for_status()
+
+        # Check if response is SSE
+        if "text/event-stream" in r.headers.get("Content-Type", ""):
+            return StreamingResponse(
+                r.iter_content(chunk_size=8192),
+                status_code=r.status_code,
+                headers=dict(r.headers),
+            )
+        else:
+            response_data = r.json()
+            return response_data
     except Exception as e:
-        return JSONResponse(status_code=400, content={"detail": str(e)})
+        log.exception(e)
+        error_detail = "Open WebUI: Server Connection Error"
+        if r is not None:
+            try:
+                res = r.json()
+                if "error" in res:
+                    error_detail = f"External: {res['error']['message'] if 'message' in res['error'] else res['error']}"
+            except:
+                error_detail = f"External: {e}"
 
-    response = await call_next(request)
+        raise HTTPException(
-    return response
+            status_code=r.status_code if r else 500, detail=error_detail
-
+        )
-
-class ModifyModelsResponseMiddleware(BaseHTTPMiddleware):
-    async def dispatch(
-        self, request: Request, call_next: RequestResponseEndpoint
-    ) -> Response:
-
-        response = await call_next(request)
-        user = request.state.user
-
-        if "/models" in request.url.path:
-            if isinstance(response, StreamingResponse):
-                # Read the content of the streaming response
-                body = b""
-                async for chunk in response.body_iterator:
-                    body += chunk
-
-                data = json.loads(body.decode("utf-8"))
-
-                if app.state.MODEL_FILTER_ENABLED:
-                    if user and user.role == "user":
-                        data["data"] = list(
-                            filter(
-                                lambda model: model["id"]
-                                in app.state.MODEL_FILTER_LIST,
-                                data["data"],
-                            )
-                        )
-
-                # Modified Flag
-                data["modified"] = True
-                return JSONResponse(content=data)
-
-        return response
-
-
-app.add_middleware(ModifyModelsResponseMiddleware)

backend/apps/openai/main.py

@@ -80,6 +80,7 @@ async def get_openai_urls(user=Depends(get_admin_user)):
 
 @app.post("/urls/update")
 async def update_openai_urls(form_data: UrlsUpdateForm, user=Depends(get_admin_user)):
+    await get_all_models()
     app.state.OPENAI_API_BASE_URLS = form_data.urls
     return {"OPENAI_API_BASE_URLS": app.state.OPENAI_API_BASE_URLS}
 
@@ -341,7 +342,7 @@ async def proxy(path: str, request: Request, user=Depends(get_verified_user)):
             try:
                 res = r.json()
                 if "error" in res:
-                    error_detail = f"External: {res['error']}"
+                    error_detail = f"External: {res['error']['message'] if 'message' in res['error'] else res['error']}"
             except:
                 error_detail = f"External: {e}"
 

backend/apps/rag/main.py

@@ -13,7 +13,6 @@ import os, shutil, logging, re
 from pathlib import Path
 from typing import List
 
-from chromadb.utils import embedding_functions
 from chromadb.utils.batch_utils import create_batches
 
 from langchain_community.document_loaders import (
@@ -38,6 +37,7 @@ import mimetypes
 import uuid
 import json
 
+import sentence_transformers
 
 from apps.ollama.main import generate_ollama_embeddings, GenerateEmbeddingsForm
 
@@ -48,11 +48,8 @@ from apps.web.models.documents import (
 )
 
 from apps.rag.utils import (
-    query_doc,
     query_embeddings_doc,
-    query_collection,
     query_embeddings_collection,
-    get_embedding_model_path,
     generate_openai_embeddings,
 )
 
@@ -69,7 +66,9 @@ from config import (
     DOCS_DIR,
     RAG_EMBEDDING_ENGINE,
     RAG_EMBEDDING_MODEL,
-    RAG_EMBEDDING_MODEL_AUTO_UPDATE,
+    RAG_EMBEDDING_MODEL_TRUST_REMOTE_CODE,
+    RAG_OPENAI_API_BASE_URL,
+    RAG_OPENAI_API_KEY,
     DEVICE_TYPE,
     CHROMA_CLIENT,
     CHUNK_SIZE,
@@ -94,20 +93,17 @@ app.state.RAG_EMBEDDING_ENGINE = RAG_EMBEDDING_ENGINE
 app.state.RAG_EMBEDDING_MODEL = RAG_EMBEDDING_MODEL
 app.state.RAG_TEMPLATE = RAG_TEMPLATE
 
-app.state.RAG_OPENAI_API_BASE_URL = "https://api.openai.com"
+app.state.OPENAI_API_BASE_URL = RAG_OPENAI_API_BASE_URL
-app.state.RAG_OPENAI_API_KEY = ""
+app.state.OPENAI_API_KEY = RAG_OPENAI_API_KEY
 
 app.state.PDF_EXTRACT_IMAGES = False
 
-
+if app.state.RAG_EMBEDDING_ENGINE == "":
-app.state.sentence_transformer_ef = (
+    app.state.sentence_transformer_ef = sentence_transformers.SentenceTransformer(
-    embedding_functions.SentenceTransformerEmbeddingFunction(
+        app.state.RAG_EMBEDDING_MODEL,
-        model_name=get_embedding_model_path(
-            app.state.RAG_EMBEDDING_MODEL, RAG_EMBEDDING_MODEL_AUTO_UPDATE
-        ),
         device=DEVICE_TYPE,
+        trust_remote_code=RAG_EMBEDDING_MODEL_TRUST_REMOTE_CODE,
     )
-)
 
 
 origins = ["*"]
@@ -148,8 +144,8 @@ async def get_embedding_config(user=Depends(get_admin_user)):
         "embedding_engine": app.state.RAG_EMBEDDING_ENGINE,
         "embedding_model": app.state.RAG_EMBEDDING_MODEL,
         "openai_config": {
-            "url": app.state.RAG_OPENAI_API_BASE_URL,
+            "url": app.state.OPENAI_API_BASE_URL,
-            "key": app.state.RAG_OPENAI_API_KEY,
+            "key": app.state.OPENAI_API_KEY,
         },
     }
 
@@ -180,16 +176,13 @@ async def update_embedding_config(
             app.state.sentence_transformer_ef = None
 
             if form_data.openai_config != None:
-                app.state.RAG_OPENAI_API_BASE_URL = form_data.openai_config.url
+                app.state.OPENAI_API_BASE_URL = form_data.openai_config.url
-                app.state.RAG_OPENAI_API_KEY = form_data.openai_config.key
+                app.state.OPENAI_API_KEY = form_data.openai_config.key
         else:
-            sentence_transformer_ef = (
+            sentence_transformer_ef = sentence_transformers.SentenceTransformer(
-                embedding_functions.SentenceTransformerEmbeddingFunction(
+                app.state.RAG_EMBEDDING_MODEL,
-                    model_name=get_embedding_model_path(
+                device=DEVICE_TYPE,
-                        form_data.embedding_model, True
+                trust_remote_code=RAG_EMBEDDING_MODEL_TRUST_REMOTE_CODE,
-                    ),
-                    device=DEVICE_TYPE,
-                )
             )
             app.state.RAG_EMBEDDING_MODEL = form_data.embedding_model
             app.state.sentence_transformer_ef = sentence_transformer_ef
@@ -199,8 +192,8 @@ async def update_embedding_config(
             "embedding_engine": app.state.RAG_EMBEDDING_ENGINE,
             "embedding_model": app.state.RAG_EMBEDDING_MODEL,
             "openai_config": {
-                "url": app.state.RAG_OPENAI_API_BASE_URL,
+                "url": app.state.OPENAI_API_BASE_URL,
-                "key": app.state.RAG_OPENAI_API_KEY,
+                "key": app.state.OPENAI_API_KEY,
             },
         }
 
@@ -292,38 +285,34 @@ def query_doc_handler(
     form_data: QueryDocForm,
     user=Depends(get_current_user),
 ):
-
     try:
         if app.state.RAG_EMBEDDING_ENGINE == "":
-            return query_doc(
+            query_embeddings = app.state.sentence_transformer_ef.encode(
-                collection_name=form_data.collection_name,
+                form_data.query
-                query=form_data.query,
+            ).tolist()
-                k=form_data.k if form_data.k else app.state.TOP_K,
+        elif app.state.RAG_EMBEDDING_ENGINE == "ollama":
-                embedding_function=app.state.sentence_transformer_ef,
+            query_embeddings = generate_ollama_embeddings(
+                GenerateEmbeddingsForm(
+                    **{
+                        "model": app.state.RAG_EMBEDDING_MODEL,
+                        "prompt": form_data.query,
+                    }
+                )
+            )
+        elif app.state.RAG_EMBEDDING_ENGINE == "openai":
+            query_embeddings = generate_openai_embeddings(
+                model=app.state.RAG_EMBEDDING_MODEL,
+                text=form_data.query,
+                key=app.state.OPENAI_API_KEY,
+                url=app.state.OPENAI_API_BASE_URL,
             )
-        else:
-            if app.state.RAG_EMBEDDING_ENGINE == "ollama":
-                query_embeddings = generate_ollama_embeddings(
-                    GenerateEmbeddingsForm(
-                        **{
-                            "model": app.state.RAG_EMBEDDING_MODEL,
-                            "prompt": form_data.query,
-                        }
-                    )
-                )
-            elif app.state.RAG_EMBEDDING_ENGINE == "openai":
-                query_embeddings = generate_openai_embeddings(
-                    model=app.state.RAG_EMBEDDING_MODEL,
-                    text=form_data.query,
-                    key=app.state.RAG_OPENAI_API_KEY,
-                    url=app.state.RAG_OPENAI_API_BASE_URL,
-                )
 
-            return query_embeddings_doc(
+        return query_embeddings_doc(
-                collection_name=form_data.collection_name,
+            collection_name=form_data.collection_name,
-                query_embeddings=query_embeddings,
+            query=form_data.query,
-                k=form_data.k if form_data.k else app.state.TOP_K,
+            query_embeddings=query_embeddings,
-            )
+            k=form_data.k if form_data.k else app.state.TOP_K,
+        )
 
     except Exception as e:
         log.exception(e)
@@ -346,36 +335,31 @@ def query_collection_handler(
 ):
     try:
         if app.state.RAG_EMBEDDING_ENGINE == "":
-            return query_collection(
+            query_embeddings = app.state.sentence_transformer_ef.encode(
-                collection_names=form_data.collection_names,
+                form_data.query
-                query=form_data.query,
+            ).tolist()
-                k=form_data.k if form_data.k else app.state.TOP_K,
+        elif app.state.RAG_EMBEDDING_ENGINE == "ollama":
-                embedding_function=app.state.sentence_transformer_ef,
+            query_embeddings = generate_ollama_embeddings(
-            )
+                GenerateEmbeddingsForm(
-        else:
+                    **{
-
+                        "model": app.state.RAG_EMBEDDING_MODEL,
-            if app.state.RAG_EMBEDDING_ENGINE == "ollama":
+                        "prompt": form_data.query,
-                query_embeddings = generate_ollama_embeddings(
+                    }
-                    GenerateEmbeddingsForm(
-                        **{
-                            "model": app.state.RAG_EMBEDDING_MODEL,
-                            "prompt": form_data.query,
-                        }
-                    )
                 )
-            elif app.state.RAG_EMBEDDING_ENGINE == "openai":
-                query_embeddings = generate_openai_embeddings(
-                    model=app.state.RAG_EMBEDDING_MODEL,
-                    text=form_data.query,
-                    key=app.state.RAG_OPENAI_API_KEY,
-                    url=app.state.RAG_OPENAI_API_BASE_URL,
-                )
-
-            return query_embeddings_collection(
-                collection_names=form_data.collection_names,
-                query_embeddings=query_embeddings,
-                k=form_data.k if form_data.k else app.state.TOP_K,
             )
+        elif app.state.RAG_EMBEDDING_ENGINE == "openai":
+            query_embeddings = generate_openai_embeddings(
+                model=app.state.RAG_EMBEDDING_MODEL,
+                text=form_data.query,
+                key=app.state.OPENAI_API_KEY,
+                url=app.state.OPENAI_API_BASE_URL,
+            )
+
+        return query_embeddings_collection(
+            collection_names=form_data.collection_names,
+            query_embeddings=query_embeddings,
+            k=form_data.k if form_data.k else app.state.TOP_K,
+        )
 
     except Exception as e:
         log.exception(e)
@@ -443,6 +427,8 @@ def store_docs_in_vector_db(docs, collection_name, overwrite: bool = False) -> b
     log.info(f"store_docs_in_vector_db {docs} {collection_name}")
 
     texts = [doc.page_content for doc in docs]
+    texts = list(map(lambda x: x.replace("\n", " "), texts))
+
     metadatas = [doc.metadata for doc in docs]
 
     try:
@@ -452,52 +438,38 @@ def store_docs_in_vector_db(docs, collection_name, overwrite: bool = False) -> b
                     log.info(f"deleting existing collection {collection_name}")
                     CHROMA_CLIENT.delete_collection(name=collection_name)
 
+        collection = CHROMA_CLIENT.create_collection(name=collection_name)
+
         if app.state.RAG_EMBEDDING_ENGINE == "":
-
+            embeddings = app.state.sentence_transformer_ef.encode(texts).tolist()
-            collection = CHROMA_CLIENT.create_collection(
+        elif app.state.RAG_EMBEDDING_ENGINE == "ollama":
-                name=collection_name,
+            embeddings = [
-                embedding_function=app.state.sentence_transformer_ef,
+                generate_ollama_embeddings(
-            )
+                    GenerateEmbeddingsForm(
-
+                        **{"model": app.state.RAG_EMBEDDING_MODEL, "prompt": text}
-            for batch in create_batches(
-                api=CHROMA_CLIENT,
-                ids=[str(uuid.uuid1()) for _ in texts],
-                metadatas=metadatas,
-                documents=texts,
-            ):
-                collection.add(*batch)
-
-        else:
-            collection = CHROMA_CLIENT.create_collection(name=collection_name)
-
-            if app.state.RAG_EMBEDDING_ENGINE == "ollama":
-                embeddings = [
-                    generate_ollama_embeddings(
-                        GenerateEmbeddingsForm(
-                            **{"model": app.state.RAG_EMBEDDING_MODEL, "prompt": text}
-                        )
                     )
-                    for text in texts
+                )
-                ]
+                for text in texts
-            elif app.state.RAG_EMBEDDING_ENGINE == "openai":
+            ]
-                embeddings = [
+        elif app.state.RAG_EMBEDDING_ENGINE == "openai":
-                    generate_openai_embeddings(
+            embeddings = [
-                        model=app.state.RAG_EMBEDDING_MODEL,
+                generate_openai_embeddings(
-                        text=text,
+                    model=app.state.RAG_EMBEDDING_MODEL,
-                        key=app.state.RAG_OPENAI_API_KEY,
+                    text=text,
-                        url=app.state.RAG_OPENAI_API_BASE_URL,
+                    key=app.state.OPENAI_API_KEY,
-                    )
+                    url=app.state.OPENAI_API_BASE_URL,
-                    for text in texts
+                )
-                ]
+                for text in texts
+            ]
 
-            for batch in create_batches(
+        for batch in create_batches(
-                api=CHROMA_CLIENT,
+            api=CHROMA_CLIENT,
-                ids=[str(uuid.uuid1()) for _ in texts],
+            ids=[str(uuid.uuid1()) for _ in texts],
-                metadatas=metadatas,
+            metadatas=metadatas,
-                embeddings=embeddings,
+            embeddings=embeddings,
-                documents=texts,
+            documents=texts,
-            ):
+        ):
-                collection.add(*batch)
+            collection.add(*batch)
 
         return True
     except Exception as e:

backend/apps/rag/utils.py

@@ -1,13 +1,12 @@
-import os
-import re
 import logging
-from typing import List
 import requests
 
+from typing import List
 
-from huggingface_hub import snapshot_download
+from apps.ollama.main import (
-from apps.ollama.main import generate_ollama_embeddings, GenerateEmbeddingsForm
+    generate_ollama_embeddings,
-
+    GenerateEmbeddingsForm,
+)
 
 from config import SRC_LOG_LEVELS, CHROMA_CLIENT
 
@@ -16,29 +15,12 @@ log = logging.getLogger(__name__)
 log.setLevel(SRC_LOG_LEVELS["RAG"])
 
 
-def query_doc(collection_name: str, query: str, k: int, embedding_function):
+def query_embeddings_doc(collection_name: str, query: str, query_embeddings, k: int):
-    try:
-        # if you use docker use the model from the environment variable
-        collection = CHROMA_CLIENT.get_collection(
-            name=collection_name,
-            embedding_function=embedding_function,
-        )
-        result = collection.query(
-            query_texts=[query],
-            n_results=k,
-        )
-        return result
-    except Exception as e:
-        raise e
-
-
-def query_embeddings_doc(collection_name: str, query_embeddings, k: int):
     try:
         # if you use docker use the model from the environment variable
         log.info(f"query_embeddings_doc {query_embeddings}")
-        collection = CHROMA_CLIENT.get_collection(
+        collection = CHROMA_CLIENT.get_collection(name=collection_name)
-            name=collection_name,
+
-        )
         result = collection.query(
             query_embeddings=[query_embeddings],
             n_results=k,
@@ -95,43 +77,20 @@ def merge_and_sort_query_results(query_results, k):
     return merged_query_results
 
 
-def query_collection(
+def query_embeddings_collection(
-    collection_names: List[str], query: str, k: int, embedding_function
+    collection_names: List[str], query: str, query_embeddings, k: int
 ):
 
-    results = []
-
-    for collection_name in collection_names:
-        try:
-            # if you use docker use the model from the environment variable
-            collection = CHROMA_CLIENT.get_collection(
-                name=collection_name,
-                embedding_function=embedding_function,
-            )
-
-            result = collection.query(
-                query_texts=[query],
-                n_results=k,
-            )
-            results.append(result)
-        except:
-            pass
-
-    return merge_and_sort_query_results(results, k)
-
-
-def query_embeddings_collection(collection_names: List[str], query_embeddings, k: int):
-
     results = []
     log.info(f"query_embeddings_collection {query_embeddings}")
 
     for collection_name in collection_names:
         try:
-            collection = CHROMA_CLIENT.get_collection(name=collection_name)
+            result = query_embeddings_doc(
-
+                collection_name=collection_name,
-            result = collection.query(
+                query=query,
-                query_embeddings=[query_embeddings],
+                query_embeddings=query_embeddings,
-                n_results=k,
+                k=k,
             )
             results.append(result)
         except:
@@ -197,51 +156,38 @@ def rag_messages(
                 context = doc["content"]
             else:
                 if embedding_engine == "":
-                    if doc["type"] == "collection":
+                    query_embeddings = embedding_function.encode(query).tolist()
-                        context = query_collection(
+                elif embedding_engine == "ollama":
-                            collection_names=doc["collection_names"],
+                    query_embeddings = generate_ollama_embeddings(
-                            query=query,
+                        GenerateEmbeddingsForm(
-                            k=k,
+                            **{
-                            embedding_function=embedding_function,
+                                "model": embedding_model,
-                        )
+                                "prompt": query,
-                    else:
+                            }
-                        context = query_doc(
-                            collection_name=doc["collection_name"],
-                            query=query,
-                            k=k,
-                            embedding_function=embedding_function,
                         )
+                    )
+                elif embedding_engine == "openai":
+                    query_embeddings = generate_openai_embeddings(
+                        model=embedding_model,
+                        text=query,
+                        key=openai_key,
+                        url=openai_url,
+                    )
 
+                if doc["type"] == "collection":
+                    context = query_embeddings_collection(
+                        collection_names=doc["collection_names"],
+                        query=query,
+                        query_embeddings=query_embeddings,
+                        k=k,
+                    )
                 else:
-                    if embedding_engine == "ollama":
+                    context = query_embeddings_doc(
-                        query_embeddings = generate_ollama_embeddings(
+                        collection_name=doc["collection_name"],
-                            GenerateEmbeddingsForm(
+                        query=query,
-                                **{
+                        query_embeddings=query_embeddings,
-                                    "model": embedding_model,
+                        k=k,
-                                    "prompt": query,
+                    )
-                                }
-                            )
-                        )
-                    elif embedding_engine == "openai":
-                        query_embeddings = generate_openai_embeddings(
-                            model=embedding_model,
-                            text=query,
-                            key=openai_key,
-                            url=openai_url,
-                        )
-
-                    if doc["type"] == "collection":
-                        context = query_embeddings_collection(
-                            collection_names=doc["collection_names"],
-                            query_embeddings=query_embeddings,
-                            k=k,
-                        )
-                    else:
-                        context = query_embeddings_doc(
-                            collection_name=doc["collection_name"],
-                            query_embeddings=query_embeddings,
-                            k=k,
-                        )
 
         except Exception as e:
             log.exception(e)
@@ -283,52 +229,12 @@ def rag_messages(
     return messages
 
 
-def get_embedding_model_path(
-    embedding_model: str, update_embedding_model: bool = False
-):
-    # Construct huggingface_hub kwargs with local_files_only to return the snapshot path
-    cache_dir = os.getenv("SENTENCE_TRANSFORMERS_HOME")
-
-    local_files_only = not update_embedding_model
-
-    snapshot_kwargs = {
-        "cache_dir": cache_dir,
-        "local_files_only": local_files_only,
-    }
-
-    log.debug(f"embedding_model: {embedding_model}")
-    log.debug(f"snapshot_kwargs: {snapshot_kwargs}")
-
-    # Inspiration from upstream sentence_transformers
-    if (
-        os.path.exists(embedding_model)
-        or ("\\" in embedding_model or embedding_model.count("/") > 1)
-        and local_files_only
-    ):
-        # If fully qualified path exists, return input, else set repo_id
-        return embedding_model
-    elif "/" not in embedding_model:
-        # Set valid repo_id for model short-name
-        embedding_model = "sentence-transformers" + "/" + embedding_model
-
-    snapshot_kwargs["repo_id"] = embedding_model
-
-    # Attempt to query the huggingface_hub library to determine the local path and/or to update
-    try:
-        embedding_model_repo_path = snapshot_download(**snapshot_kwargs)
-        log.debug(f"embedding_model_repo_path: {embedding_model_repo_path}")
-        return embedding_model_repo_path
-    except Exception as e:
-        log.exception(f"Cannot determine embedding model snapshot path: {e}")
-        return embedding_model
-
-
 def generate_openai_embeddings(
-    model: str, text: str, key: str, url: str = "https://api.openai.com"
+    model: str, text: str, key: str, url: str = "https://api.openai.com/v1"
 ):
     try:
         r = requests.post(
-            f"{url}/v1/embeddings",
+            f"{url}/embeddings",
             headers={
                 "Content-Type": "application/json",
                 "Authorization": f"Bearer {key}",

backend/apps/web/internal/migrations/004_add_archived.py

@@ -0,0 +1,46 @@
+"""Peewee migrations -- 002_add_local_sharing.py.
+
+Some examples (model - class or model name)::
+
+    > Model = migrator.orm['table_name']            # Return model in current state by name
+    > Model = migrator.ModelClass                   # Return model in current state by name
+
+    > migrator.sql(sql)                             # Run custom SQL
+    > migrator.run(func, *args, **kwargs)           # Run python function with the given args
+    > migrator.create_model(Model)                  # Create a model (could be used as decorator)
+    > migrator.remove_model(model, cascade=True)    # Remove a model
+    > migrator.add_fields(model, **fields)          # Add fields to a model
+    > migrator.change_fields(model, **fields)       # Change fields
+    > migrator.remove_fields(model, *field_names, cascade=True)
+    > migrator.rename_field(model, old_field_name, new_field_name)
+    > migrator.rename_table(model, new_table_name)
+    > migrator.add_index(model, *col_names, unique=False)
+    > migrator.add_not_null(model, *field_names)
+    > migrator.add_default(model, field_name, default)
+    > migrator.add_constraint(model, name, sql)
+    > migrator.drop_index(model, *col_names)
+    > migrator.drop_not_null(model, *field_names)
+    > migrator.drop_constraints(model, *constraints)
+
+"""
+
+from contextlib import suppress
+
+import peewee as pw
+from peewee_migrate import Migrator
+
+
+with suppress(ImportError):
+    import playhouse.postgres_ext as pw_pext
+
+
+def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
+    """Write your migrations here."""
+
+    migrator.add_fields("chat", archived=pw.BooleanField(default=False))
+
+
+def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
+    """Write your rollback migrations here."""
+
+    migrator.remove_fields("chat", "archived")

backend/apps/web/internal/migrations/005_add_updated_at.py

@@ -0,0 +1,77 @@
+"""Peewee migrations -- 002_add_local_sharing.py.
+
+Some examples (model - class or model name)::
+
+    > Model = migrator.orm['table_name']            # Return model in current state by name
+    > Model = migrator.ModelClass                   # Return model in current state by name
+
+    > migrator.sql(sql)                             # Run custom SQL
+    > migrator.run(func, *args, **kwargs)           # Run python function with the given args
+    > migrator.create_model(Model)                  # Create a model (could be used as decorator)
+    > migrator.remove_model(model, cascade=True)    # Remove a model
+    > migrator.add_fields(model, **fields)          # Add fields to a model
+    > migrator.change_fields(model, **fields)       # Change fields
+    > migrator.remove_fields(model, *field_names, cascade=True)
+    > migrator.rename_field(model, old_field_name, new_field_name)
+    > migrator.rename_table(model, new_table_name)
+    > migrator.add_index(model, *col_names, unique=False)
+    > migrator.add_not_null(model, *field_names)
+    > migrator.add_default(model, field_name, default)
+    > migrator.add_constraint(model, name, sql)
+    > migrator.drop_index(model, *col_names)
+    > migrator.drop_not_null(model, *field_names)
+    > migrator.drop_constraints(model, *constraints)
+
+"""
+
+from contextlib import suppress
+
+import peewee as pw
+from peewee_migrate import Migrator
+
+
+with suppress(ImportError):
+    import playhouse.postgres_ext as pw_pext
+
+
+def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
+    """Write your migrations here."""
+
+    # Adding fields created_at and updated_at to the 'chat' table
+    migrator.add_fields(
+        "chat",
+        created_at=pw.DateTimeField(null=True),  # Allow null for transition
+        updated_at=pw.DateTimeField(null=True),  # Allow null for transition
+    )
+
+    # Populate the new fields from an existing 'timestamp' field
+    migrator.sql(
+        "UPDATE chat SET created_at = timestamp, updated_at = timestamp WHERE timestamp IS NOT NULL"
+    )
+
+    # Now that the data has been copied, remove the original 'timestamp' field
+    migrator.remove_fields("chat", "timestamp")
+
+    # Update the fields to be not null now that they are populated
+    migrator.change_fields(
+        "chat",
+        created_at=pw.DateTimeField(null=False),
+        updated_at=pw.DateTimeField(null=False),
+    )
+
+
+def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
+    """Write your rollback migrations here."""
+
+    # Recreate the timestamp field initially allowing null values for safe transition
+    migrator.add_fields("chat", timestamp=pw.DateTimeField(null=True))
+
+    # Copy the earliest created_at date back into the new timestamp field
+    # This assumes created_at was originally a copy of timestamp
+    migrator.sql("UPDATE chat SET timestamp = created_at")
+
+    # Remove the created_at and updated_at fields
+    migrator.remove_fields("chat", "created_at", "updated_at")
+
+    # Finally, alter the timestamp field to not allow nulls if that was the original setting
+    migrator.change_fields("chat", timestamp=pw.DateTimeField(null=False))

backend/apps/web/models/chats.py

@@ -19,8 +19,12 @@ class Chat(Model):
     user_id = CharField()
     title = CharField()
     chat = TextField()  # Save Chat JSON as Text
-    timestamp = DateField()
+
+    created_at = DateTimeField()
+    updated_at = DateTimeField()
+
     share_id = CharField(null=True, unique=True)
+    archived = BooleanField(default=False)
 
     class Meta:
         database = DB
@@ -31,8 +35,12 @@ class ChatModel(BaseModel):
     user_id: str
     title: str
     chat: str
-    timestamp: int  # timestamp in epoch
+
+    created_at: int  # timestamp in epoch
+    updated_at: int  # timestamp in epoch
+
     share_id: Optional[str] = None
+    archived: bool = False
 
 
 ####################
@@ -53,13 +61,17 @@ class ChatResponse(BaseModel):
     user_id: str
     title: str
     chat: dict
-    timestamp: int  # timestamp in epoch
+    updated_at: int  # timestamp in epoch
+    created_at: int  # timestamp in epoch
     share_id: Optional[str] = None  # id of the chat to be shared
+    archived: bool
 
 
 class ChatTitleIdResponse(BaseModel):
     id: str
     title: str
+    updated_at: int
+    created_at: int
 
 
 class ChatTable:
@@ -77,7 +89,8 @@ class ChatTable:
                     form_data.chat["title"] if "title" in form_data.chat else "New Chat"
                 ),
                 "chat": json.dumps(form_data.chat),
-                "timestamp": int(time.time()),
+                "created_at": int(time.time()),
+                "updated_at": int(time.time()),
             }
         )
 
@@ -89,7 +102,7 @@ class ChatTable:
             query = Chat.update(
                 chat=json.dumps(chat),
                 title=chat["title"] if "title" in chat else "New Chat",
-                timestamp=int(time.time()),
+                updated_at=int(time.time()),
             ).where(Chat.id == id)
             query.execute()
 
@@ -111,7 +124,8 @@ class ChatTable:
                 "user_id": f"shared-{chat_id}",
                 "title": chat.title,
                 "chat": chat.chat,
-                "timestamp": int(time.time()),
+                "created_at": chat.created_at,
+                "updated_at": int(time.time()),
             }
         )
         shared_result = Chat.create(**shared_chat.model_dump())
@@ -163,14 +177,42 @@ class ChatTable:
         except:
             return None
 
+    def toggle_chat_archive_by_id(self, id: str) -> Optional[ChatModel]:
+        try:
+            chat = self.get_chat_by_id(id)
+            query = Chat.update(
+                archived=(not chat.archived),
+            ).where(Chat.id == id)
+
+            query.execute()
+
+            chat = Chat.get(Chat.id == id)
+            return ChatModel(**model_to_dict(chat))
+        except:
+            return None
+
+    def get_archived_chat_lists_by_user_id(
+        self, user_id: str, skip: int = 0, limit: int = 50
+    ) -> List[ChatModel]:
+        return [
+            ChatModel(**model_to_dict(chat))
+            for chat in Chat.select()
+            .where(Chat.archived == True)
+            .where(Chat.user_id == user_id)
+            .order_by(Chat.updated_at.desc())
+            # .limit(limit)
+            # .offset(skip)
+        ]
+
     def get_chat_lists_by_user_id(
         self, user_id: str, skip: int = 0, limit: int = 50
     ) -> List[ChatModel]:
         return [
             ChatModel(**model_to_dict(chat))
             for chat in Chat.select()
+            .where(Chat.archived == False)
             .where(Chat.user_id == user_id)
-            .order_by(Chat.timestamp.desc())
+            .order_by(Chat.updated_at.desc())
             # .limit(limit)
             # .offset(skip)
         ]
@@ -181,14 +223,15 @@ class ChatTable:
         return [
             ChatModel(**model_to_dict(chat))
             for chat in Chat.select()
+            .where(Chat.archived == False)
             .where(Chat.id.in_(chat_ids))
-            .order_by(Chat.timestamp.desc())
+            .order_by(Chat.updated_at.desc())
         ]
 
     def get_all_chats(self) -> List[ChatModel]:
         return [
             ChatModel(**model_to_dict(chat))
-            for chat in Chat.select().order_by(Chat.timestamp.desc())
+            for chat in Chat.select().order_by(Chat.updated_at.desc())
         ]
 
     def get_all_chats_by_user_id(self, user_id: str) -> List[ChatModel]:
@@ -196,7 +239,7 @@ class ChatTable:
             ChatModel(**model_to_dict(chat))
             for chat in Chat.select()
             .where(Chat.user_id == user_id)
-            .order_by(Chat.timestamp.desc())
+            .order_by(Chat.updated_at.desc())
         ]
 
     def get_chat_by_id(self, id: str) -> Optional[ChatModel]:

backend/apps/web/models/tags.py

@@ -136,7 +136,9 @@ class TagTable:
 
         return [
             TagModel(**model_to_dict(tag))
-            for tag in Tag.select().where(Tag.name.in_(tag_names))
+            for tag in Tag.select()
+            .where(Tag.user_id == user_id)
+            .where(Tag.name.in_(tag_names))
         ]
 
     def get_tags_by_chat_id_and_user_id(
@@ -151,7 +153,9 @@ class TagTable:
 
         return [
             TagModel(**model_to_dict(tag))
-            for tag in Tag.select().where(Tag.name.in_(tag_names))
+            for tag in Tag.select()
+            .where(Tag.user_id == user_id)
+            .where(Tag.name.in_(tag_names))
         ]
 
     def get_chat_ids_by_tag_name_and_user_id(

backend/apps/web/routers/chats.py

@@ -28,7 +28,7 @@ from apps.web.models.tags import (
 
 from constants import ERROR_MESSAGES
 
-from config import SRC_LOG_LEVELS
+from config import SRC_LOG_LEVELS, ENABLE_ADMIN_EXPORT
 
 log = logging.getLogger(__name__)
 log.setLevel(SRC_LOG_LEVELS["MODELS"])
@@ -47,6 +47,18 @@ async def get_user_chats(
     return Chats.get_chat_lists_by_user_id(user.id, skip, limit)
 
 
+############################
+# GetArchivedChats
+############################
+
+
+@router.get("/archived", response_model=List[ChatTitleIdResponse])
+async def get_archived_user_chats(
+    user=Depends(get_current_user), skip: int = 0, limit: int = 50
+):
+    return Chats.get_archived_chat_lists_by_user_id(user.id, skip, limit)
+
+
 ############################
 # GetAllChats
 ############################
@@ -67,6 +79,11 @@ async def get_all_user_chats(user=Depends(get_current_user)):
 
 @router.get("/all/db", response_model=List[ChatResponse])
 async def get_all_user_chats_in_db(user=Depends(get_admin_user)):
+    if not ENABLE_ADMIN_EXPORT:
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail=ERROR_MESSAGES.ACCESS_PROHIBITED,
+        )
     return [
         ChatResponse(**{**chat.model_dump(), "chat": json.loads(chat.chat)})
         for chat in Chats.get_all_chats()
@@ -189,6 +206,23 @@ async def delete_chat_by_id(request: Request, id: str, user=Depends(get_current_
     return result
 
 
+############################
+# ArchiveChat
+############################
+
+
+@router.get("/{id}/archive", response_model=Optional[ChatResponse])
+async def archive_chat_by_id(id: str, user=Depends(get_current_user)):
+    chat = Chats.get_chat_by_id_and_user_id(id, user.id)
+    if chat:
+        chat = Chats.toggle_chat_archive_by_id(id)
+        return ChatResponse(**{**chat.model_dump(), "chat": json.loads(chat.chat)})
+    else:
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED, detail=ERROR_MESSAGES.DEFAULT()
+        )
+
+
 ############################
 # ShareChatById
 ############################

backend/apps/web/routers/utils.py

@@ -91,7 +91,11 @@ async def download_chat_as_pdf(
 
 @router.get("/db/download")
 async def download_db(user=Depends(get_admin_user)):
-
+    if not ENABLE_ADMIN_EXPORT:
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail=ERROR_MESSAGES.ACCESS_PROHIBITED,
+        )
     return FileResponse(
         f"{DATA_DIR}/webui.db",
         media_type="application/octet-stream",

backend/config.py

@@ -321,6 +321,18 @@ OPENAI_API_BASE_URLS = [
     for url in OPENAI_API_BASE_URLS.split(";")
 ]
 
+OPENAI_API_KEY = ""
+
+try:
+    OPENAI_API_KEY = OPENAI_API_KEYS[
+        OPENAI_API_BASE_URLS.index("https://api.openai.com/v1")
+    ]
+except:
+    pass
+
+OPENAI_API_BASE_URL = "https://api.openai.com/v1"
+
+
 ####################################
 # WEBUI
 ####################################
@@ -370,6 +382,8 @@ MODEL_FILTER_LIST = [model.strip() for model in MODEL_FILTER_LIST.split(";")]
 
 WEBHOOK_URL = os.environ.get("WEBHOOK_URL", "")
 
+ENABLE_ADMIN_EXPORT = os.environ.get("ENABLE_ADMIN_EXPORT", "True").lower() == "true"
+
 ####################################
 # WEBUI_VERSION
 ####################################
@@ -404,18 +418,19 @@ if WEBUI_AUTH and WEBUI_SECRET_KEY == "":
 ####################################
 
 CHROMA_DATA_PATH = f"{DATA_DIR}/vector_db"
-# this uses the model defined in the Dockerfile ENV variable. If you dont use docker or docker based deployments such as k8s, the default embedding model will be used (all-MiniLM-L6-v2)
+# this uses the model defined in the Dockerfile ENV variable. If you dont use docker or docker based deployments such as k8s, the default embedding model will be used (sentence-transformers/all-MiniLM-L6-v2)
 
 RAG_EMBEDDING_ENGINE = os.environ.get("RAG_EMBEDDING_ENGINE", "")
 
-RAG_EMBEDDING_MODEL = os.environ.get("RAG_EMBEDDING_MODEL", "all-MiniLM-L6-v2")
+RAG_EMBEDDING_MODEL = os.environ.get(
+    "RAG_EMBEDDING_MODEL", "sentence-transformers/all-MiniLM-L6-v2"
+)
 log.info(f"Embedding model set: {RAG_EMBEDDING_MODEL}"),
 
-RAG_EMBEDDING_MODEL_AUTO_UPDATE = (
+RAG_EMBEDDING_MODEL_TRUST_REMOTE_CODE = (
-    os.environ.get("RAG_EMBEDDING_MODEL_AUTO_UPDATE", "").lower() == "true"
+    os.environ.get("RAG_EMBEDDING_MODEL_TRUST_REMOTE_CODE", "").lower() == "true"
 )
 
-
 # device type embedding models - "cpu" (default), "cuda" (nvidia gpu required) or "mps" (apple silicon) - choosing this right can lead to better performance
 USE_CUDA = os.environ.get("USE_CUDA_DOCKER", "false")
 
@@ -447,6 +462,9 @@ And answer according to the language of the user's question.
 Given the context information, answer the query.
 Query: [query]"""
 
+RAG_OPENAI_API_BASE_URL = os.getenv("RAG_OPENAI_API_BASE_URL", OPENAI_API_BASE_URL)
+RAG_OPENAI_API_KEY = os.getenv("RAG_OPENAI_API_KEY", OPENAI_API_KEY)
+
 ####################################
 # Transcribe
 ####################################
@@ -467,3 +485,26 @@ ENABLE_IMAGE_GENERATION = (
 )
 AUTOMATIC1111_BASE_URL = os.getenv("AUTOMATIC1111_BASE_URL", "")
 COMFYUI_BASE_URL = os.getenv("COMFYUI_BASE_URL", "")
+
+
+IMAGES_OPENAI_API_BASE_URL = os.getenv(
+    "IMAGES_OPENAI_API_BASE_URL", OPENAI_API_BASE_URL
+)
+IMAGES_OPENAI_API_KEY = os.getenv("IMAGES_OPENAI_API_KEY", OPENAI_API_KEY)
+
+
+####################################
+# Audio
+####################################
+
+AUDIO_OPENAI_API_BASE_URL = os.getenv("AUDIO_OPENAI_API_BASE_URL", OPENAI_API_BASE_URL)
+AUDIO_OPENAI_API_KEY = os.getenv("AUDIO_OPENAI_API_KEY", OPENAI_API_KEY)
+
+####################################
+# LiteLLM
+####################################
+
+LITELLM_PROXY_PORT = int(os.getenv("LITELLM_PROXY_PORT", "14365"))
+if LITELLM_PROXY_PORT < 0 or LITELLM_PROXY_PORT > 65535:
+    raise ValueError("Invalid port number for LITELLM_PROXY_PORT")
+LITELLM_PROXY_HOST = os.getenv("LITELLM_PROXY_HOST", "127.0.0.1")

backend/constants.py

@@ -3,6 +3,10 @@ from enum import Enum
 
 class MESSAGES(str, Enum):
     DEFAULT = lambda msg="": f"{msg if msg else ''}"
+    MODEL_ADDED = lambda model="": f"The model '{model}' has been added successfully."
+    MODEL_DELETED = (
+        lambda model="": f"The model '{model}' has been deleted successfully."
+    )
 
 
 class WEBHOOK_MESSAGES(str, Enum):

backend/main.py

@@ -20,12 +20,17 @@ from starlette.middleware.base import BaseHTTPMiddleware
 from apps.ollama.main import app as ollama_app
 from apps.openai.main import app as openai_app
 
-from apps.litellm.main import app as litellm_app, startup as litellm_app_startup
+from apps.litellm.main import (
+    app as litellm_app,
+    start_litellm_background,
+    shutdown_litellm_background,
+)
 from apps.audio.main import app as audio_app
 from apps.images.main import app as images_app
 from apps.rag.main import app as rag_app
 from apps.web.main import app as webui_app
 
+import asyncio
 from pydantic import BaseModel
 from typing import List
 
@@ -47,6 +52,7 @@ from config import (
     GLOBAL_LOG_LEVEL,
     SRC_LOG_LEVELS,
     WEBHOOK_URL,
+    ENABLE_ADMIN_EXPORT,
 )
 from constants import ERROR_MESSAGES
 
@@ -117,8 +123,8 @@ class RAGMiddleware(BaseHTTPMiddleware):
                     rag_app.state.RAG_EMBEDDING_ENGINE,
                     rag_app.state.RAG_EMBEDDING_MODEL,
                     rag_app.state.sentence_transformer_ef,
-                    rag_app.state.RAG_OPENAI_API_KEY,
+                    rag_app.state.OPENAI_API_KEY,
-                    rag_app.state.RAG_OPENAI_API_BASE_URL,
+                    rag_app.state.OPENAI_API_BASE_URL,
                 )
                 del data["docs"]
 
@@ -170,7 +176,7 @@ async def check_url(request: Request, call_next):
 
 @app.on_event("startup")
 async def on_startup():
-    await litellm_app_startup()
+    asyncio.create_task(start_litellm_background())
 
 
 app.mount("/api/v1", webui_app)
@@ -202,6 +208,7 @@ async def get_app_config():
         "default_models": webui_app.state.DEFAULT_MODELS,
         "default_prompt_suggestions": webui_app.state.DEFAULT_PROMPT_SUGGESTIONS,
         "trusted_header_auth": bool(webui_app.state.AUTH_TRUSTED_EMAIL_HEADER),
+        "admin_export_enabled": ENABLE_ADMIN_EXPORT,
     }
 
 
@@ -315,3 +322,8 @@ app.mount(
     SPAStaticFiles(directory=FRONTEND_BUILD_DIR, html=True),
     name="spa-static-files",
 )
+
+
+@app.on_event("shutdown")
+async def shutdown_event():
+    await shutdown_litellm_background()

backend/requirements.txt

@@ -17,7 +17,9 @@ peewee
 peewee-migrate
 bcrypt
 
-litellm==1.30.7
+litellm==1.35.17
+litellm[proxy]==1.35.17
+
 boto3
 
 argon2-cffi
@@ -25,6 +27,7 @@ apscheduler
 google-generativeai
 
 langchain
+langchain-chroma
 langchain-community
 fake_useragent
 chromadb
@@ -43,6 +46,7 @@ opencv-python-headless
 rapidocr-onnxruntime
 
 fpdf2
+rank_bm25
 
 faster-whisper
 

package-lock.json

@@ -1,12 +1,12 @@
 {
 	"name": "open-webui",
-	"version": "0.1.119",
+	"version": "0.1.121",
 	"lockfileVersion": 3,
 	"requires": true,
 	"packages": {
 		"": {
 			"name": "open-webui",
-			"version": "0.1.119",
+			"version": "0.1.121",
 			"dependencies": {
 				"@sveltejs/adapter-node": "^1.3.1",
 				"async": "^3.2.5",

package.json

@@ -1,6 +1,6 @@
 {
 	"name": "open-webui",
-	"version": "0.1.119",
+	"version": "0.1.121",
 	"private": true,
 	"scripts": {
 		"dev": "vite dev --host",

src/lib/apis/audio/index.ts

@@ -1,11 +1,73 @@
 import { AUDIO_API_BASE_URL } from '$lib/constants';
 
+export const getAudioConfig = async (token: string) => {
+	let error = null;
+
+	const res = await fetch(`${AUDIO_API_BASE_URL}/config`, {
+		method: 'GET',
+		headers: {
+			'Content-Type': 'application/json',
+			Authorization: `Bearer ${token}`
+		}
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.catch((err) => {
+			console.log(err);
+			error = err.detail;
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
+type OpenAIConfigForm = {
+	url: string;
+	key: string;
+};
+
+export const updateAudioConfig = async (token: string, payload: OpenAIConfigForm) => {
+	let error = null;
+
+	const res = await fetch(`${AUDIO_API_BASE_URL}/config/update`, {
+		method: 'POST',
+		headers: {
+			'Content-Type': 'application/json',
+			Authorization: `Bearer ${token}`
+		},
+		body: JSON.stringify({
+			...payload
+		})
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.catch((err) => {
+			console.log(err);
+			error = err.detail;
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
 export const transcribeAudio = async (token: string, file: File) => {
 	const data = new FormData();
 	data.append('file', file);
 
 	let error = null;
-	const res = await fetch(`${AUDIO_API_BASE_URL}/transcribe`, {
+	const res = await fetch(`${AUDIO_API_BASE_URL}/transcriptions`, {
 		method: 'POST',
 		headers: {
 			Accept: 'application/json',
@@ -29,3 +91,40 @@ export const transcribeAudio = async (token: string, file: File) => {
 
 	return res;
 };
+
+export const synthesizeOpenAISpeech = async (
+	token: string = '',
+	speaker: string = 'alloy',
+	text: string = ''
+) => {
+	let error = null;
+
+	const res = await fetch(`${AUDIO_API_BASE_URL}/speech`, {
+		method: 'POST',
+		headers: {
+			Authorization: `Bearer ${token}`,
+			'Content-Type': 'application/json'
+		},
+		body: JSON.stringify({
+			model: 'tts-1',
+			input: text,
+			voice: speaker
+		})
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res;
+		})
+		.catch((err) => {
+			error = err.detail;
+			console.log(err);
+
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};

src/lib/apis/chats/index.ts

@@ -62,6 +62,37 @@ export const getChatList = async (token: string = '') => {
 	return res;
 };
 
+export const getArchivedChatList = async (token: string = '') => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/chats/archived`, {
+		method: 'GET',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			...(token && { authorization: `Bearer ${token}` })
+		}
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.then((json) => {
+			return json;
+		})
+		.catch((err) => {
+			error = err;
+			console.log(err);
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
 export const getAllChats = async (token: string) => {
 	let error = null;
 
@@ -282,6 +313,38 @@ export const shareChatById = async (token: string, id: string) => {
 	return res;
 };
 
+export const archiveChatById = async (token: string, id: string) => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/chats/${id}/archive`, {
+		method: 'GET',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			...(token && { authorization: `Bearer ${token}` })
+		}
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.then((json) => {
+			return json;
+		})
+		.catch((err) => {
+			error = err;
+
+			console.log(err);
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
 export const deleteSharedChatById = async (token: string, id: string) => {
 	let error = null;
 

src/lib/apis/images/index.ts

@@ -72,10 +72,10 @@ export const updateImageGenerationConfig = async (
 	return res;
 };
 
-export const getOpenAIKey = async (token: string = '') => {
+export const getOpenAIConfig = async (token: string = '') => {
 	let error = null;
 
-	const res = await fetch(`${IMAGES_API_BASE_URL}/key`, {
+	const res = await fetch(`${IMAGES_API_BASE_URL}/openai/config`, {
 		method: 'GET',
 		headers: {
 			Accept: 'application/json',
@@ -101,13 +101,13 @@ export const getOpenAIKey = async (token: string = '') => {
 		throw error;
 	}
 
-	return res.OPENAI_API_KEY;
+	return res;
 };
 
-export const updateOpenAIKey = async (token: string = '', key: string) => {
+export const updateOpenAIConfig = async (token: string = '', url: string, key: string) => {
 	let error = null;
 
-	const res = await fetch(`${IMAGES_API_BASE_URL}/key/update`, {
+	const res = await fetch(`${IMAGES_API_BASE_URL}/openai/config/update`, {
 		method: 'POST',
 		headers: {
 			Accept: 'application/json',
@@ -115,6 +115,7 @@ export const updateOpenAIKey = async (token: string = '', key: string) => {
 			...(token && { authorization: `Bearer ${token}` })
 		},
 		body: JSON.stringify({
+			url: url,
 			key: key
 		})
 	})
@@ -136,7 +137,7 @@ export const updateOpenAIKey = async (token: string = '', key: string) => {
 		throw error;
 	}
 
-	return res.OPENAI_API_KEY;
+	return res;
 };
 
 export const getImageGenerationEngineUrls = async (token: string = '') => {

src/lib/apis/streaming/index.ts

@@ -0,0 +1,70 @@
+type TextStreamUpdate = {
+	done: boolean;
+	value: string;
+};
+
+// createOpenAITextStream takes a ReadableStreamDefaultReader from an SSE response,
+// and returns an async generator that emits delta updates with large deltas chunked into random sized chunks
+export async function createOpenAITextStream(
+	messageStream: ReadableStreamDefaultReader,
+	splitLargeDeltas: boolean
+): Promise<AsyncGenerator<TextStreamUpdate>> {
+	let iterator = openAIStreamToIterator(messageStream);
+	if (splitLargeDeltas) {
+		iterator = streamLargeDeltasAsRandomChunks(iterator);
+	}
+	return iterator;
+}
+
+async function* openAIStreamToIterator(
+	reader: ReadableStreamDefaultReader
+): AsyncGenerator<TextStreamUpdate> {
+	while (true) {
+		const { value, done } = await reader.read();
+		if (done) {
+			yield { done: true, value: '' };
+			break;
+		}
+		const lines = value.split('\n');
+		for (const line of lines) {
+			if (line !== '') {
+				console.log(line);
+				if (line === 'data: [DONE]') {
+					yield { done: true, value: '' };
+				} else {
+					const data = JSON.parse(line.replace(/^data: /, ''));
+					console.log(data);
+
+					yield { done: false, value: data.choices[0].delta.content ?? '' };
+				}
+			}
+		}
+	}
+}
+
+// streamLargeDeltasAsRandomChunks will chunk large deltas (length > 5) into random sized chunks between 1-3 characters
+// This is to simulate a more fluid streaming, even though some providers may send large chunks of text at once
+async function* streamLargeDeltasAsRandomChunks(
+	iterator: AsyncGenerator<TextStreamUpdate>
+): AsyncGenerator<TextStreamUpdate> {
+	for await (const textStreamUpdate of iterator) {
+		if (textStreamUpdate.done) {
+			yield textStreamUpdate;
+			return;
+		}
+		let content = textStreamUpdate.value;
+		if (content.length < 5) {
+			yield { done: false, value: content };
+			continue;
+		}
+		while (content != '') {
+			const chunkSize = Math.min(Math.floor(Math.random() * 3) + 1, content.length);
+			const chunk = content.slice(0, chunkSize);
+			yield { done: false, value: chunk };
+			await sleep(5);
+			content = content.slice(chunkSize);
+		}
+	}
+}
+
+const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

src/lib/components/admin/Settings/Database.svelte

@@ -1,6 +1,7 @@
 <script lang="ts">
 	import { downloadDatabase } from '$lib/apis/utils';
 	import { onMount, getContext } from 'svelte';
+	import { config } from '$lib/stores';
 
 	const i18n = getContext('i18n');
 
@@ -24,32 +25,34 @@
 			<div class="  flex w-full justify-between">
 				<!-- <div class=" self-center text-xs font-medium">{$i18n.t('Allow Chat Deletion')}</div> -->
 
-				<button
+				{#if $config?.admin_export_enabled ?? true}
-					class=" flex rounded-md py-1.5 px-3 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
+					<button
-					type="button"
+						class=" flex rounded-md py-1.5 px-3 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
-					on:click={() => {
+						type="button"
-						// exportAllUserChats();
+						on:click={() => {
+							// exportAllUserChats();
 
-						downloadDatabase(localStorage.token);
+							downloadDatabase(localStorage.token);
-					}}
+						}}
-				>
+					>
-					<div class=" self-center mr-3">
+						<div class=" self-center mr-3">
-						<svg
+							<svg
-							xmlns="http://www.w3.org/2000/svg"
+								xmlns="http://www.w3.org/2000/svg"
-							viewBox="0 0 16 16"
+								viewBox="0 0 16 16"
-							fill="currentColor"
+								fill="currentColor"
-							class="w-4 h-4"
+								class="w-4 h-4"
-						>
+							>
-							<path d="M2 3a1 1 0 0 1 1-1h10a1 1 0 0 1 1 1v1a1 1 0 0 1-1 1H3a1 1 0 0 1-1-1V3Z" />
+								<path d="M2 3a1 1 0 0 1 1-1h10a1 1 0 0 1 1 1v1a1 1 0 0 1-1 1H3a1 1 0 0 1-1-1V3Z" />
-							<path
+								<path
-								fill-rule="evenodd"
+									fill-rule="evenodd"
-								d="M13 6H3v6a2 2 0 0 0 2 2h6a2 2 0 0 0 2-2V6ZM8.75 7.75a.75.75 0 0 0-1.5 0v2.69L6.03 9.22a.75.75 0 0 0-1.06 1.06l2.5 2.5a.75.75 0 0 0 1.06 0l2.5-2.5a.75.75 0 1 0-1.06-1.06l-1.22 1.22V7.75Z"
+									d="M13 6H3v6a2 2 0 0 0 2 2h6a2 2 0 0 0 2-2V6ZM8.75 7.75a.75.75 0 0 0-1.5 0v2.69L6.03 9.22a.75.75 0 0 0-1.06 1.06l2.5 2.5a.75.75 0 0 0 1.06 0l2.5-2.5a.75.75 0 1 0-1.06-1.06l-1.22 1.22V7.75Z"
-								clip-rule="evenodd"
+									clip-rule="evenodd"
-							/>
+								/>
-						</svg>
+							</svg>
-					</div>
+						</div>
-					<div class=" self-center text-sm font-medium">{$i18n.t('Download Database')}</div>
+						<div class=" self-center text-sm font-medium">{$i18n.t('Download Database')}</div>
-				</button>
+					</button>
+				{/if}
 			</div>
 		</div>
 	</div>

src/lib/components/chat/MessageInput.svelte

@@ -316,39 +316,38 @@
 			console.log(e);
 
 			if (e.dataTransfer?.files) {
-				let reader = new FileReader();
+				const inputFiles = Array.from(e.dataTransfer?.files);
-
-				reader.onload = (event) => {
-					files = [
-						...files,
-						{
-							type: 'image',
-							url: `${event.target.result}`
-						}
-					];
-				};
-
-				const inputFiles = e.dataTransfer?.files;
 
 				if (inputFiles && inputFiles.length > 0) {
-					const file = inputFiles[0];
+					inputFiles.forEach((file) => {
-					console.log(file, file.name.split('.').at(-1));
+						console.log(file, file.name.split('.').at(-1));
-					if (['image/gif', 'image/jpeg', 'image/png'].includes(file['type'])) {
+						if (['image/gif', 'image/jpeg', 'image/png'].includes(file['type'])) {
-						reader.readAsDataURL(file);
+							let reader = new FileReader();
-					} else if (
+							reader.onload = (event) => {
-						SUPPORTED_FILE_TYPE.includes(file['type']) ||
+								files = [
-						SUPPORTED_FILE_EXTENSIONS.includes(file.name.split('.').at(-1))
+									...files,
-					) {
+									{
-						uploadDoc(file);
+										type: 'image',
-					} else {
+										url: `${event.target.result}`
-						toast.error(
+									}
-							$i18n.t(
+								];
-								`Unknown File Type '{{file_type}}', but accepting and treating as plain text`,
+							};
-								{ file_type: file['type'] }
+							reader.readAsDataURL(file);
-							)
+						} else if (
-						);
+							SUPPORTED_FILE_TYPE.includes(file['type']) ||
-						uploadDoc(file);
+							SUPPORTED_FILE_EXTENSIONS.includes(file.name.split('.').at(-1))
-					}
+						) {
+							uploadDoc(file);
+						} else {
+							toast.error(
+								$i18n.t(
+									`Unknown File Type '{{file_type}}', but accepting and treating as plain text`,
+									{ file_type: file['type'] }
+								)
+							);
+							uploadDoc(file);
+						}
+					});
 				} else {
 					toast.error($i18n.t(`File not found.`));
 				}
@@ -467,40 +466,42 @@
 					bind:files={inputFiles}
 					type="file"
 					hidden
+					multiple
 					on:change={async () => {
-						let reader = new FileReader();
-						reader.onload = (event) => {
-							files = [
-								...files,
-								{
-									type: 'image',
-									url: `${event.target.result}`
-								}
-							];
-							inputFiles = null;
-							filesInputElement.value = '';
-						};
-
 						if (inputFiles && inputFiles.length > 0) {
-							const file = inputFiles[0];
+							const _inputFiles = Array.from(inputFiles);
-							if (['image/gif', 'image/jpeg', 'image/png'].includes(file['type'])) {
+							_inputFiles.forEach((file) => {
-								reader.readAsDataURL(file);
+								if (['image/gif', 'image/jpeg', 'image/png'].includes(file['type'])) {
-							} else if (
+									let reader = new FileReader();
-								SUPPORTED_FILE_TYPE.includes(file['type']) ||
+									reader.onload = (event) => {
-								SUPPORTED_FILE_EXTENSIONS.includes(file.name.split('.').at(-1))
+										files = [
-							) {
+											...files,
-								uploadDoc(file);
+											{
-								filesInputElement.value = '';
+												type: 'image',
-							} else {
+												url: `${event.target.result}`
-								toast.error(
+											}
-									$i18n.t(
+										];
-										`Unknown File Type '{{file_type}}', but accepting and treating as plain text`,
+										inputFiles = null;
-										{ file_type: file['type'] }
+										filesInputElement.value = '';
-									)
+									};
-								);
+									reader.readAsDataURL(file);
-								uploadDoc(file);
+								} else if (
-								filesInputElement.value = '';
+									SUPPORTED_FILE_TYPE.includes(file['type']) ||
-							}
+									SUPPORTED_FILE_EXTENSIONS.includes(file.name.split('.').at(-1))
+								) {
+									uploadDoc(file);
+									filesInputElement.value = '';
+								} else {
+									toast.error(
+										$i18n.t(
+											`Unknown File Type '{{file_type}}', but accepting and treating as plain text`,
+											{ file_type: file['type'] }
+										)
+									);
+									uploadDoc(file);
+									filesInputElement.value = '';
+								}
+							});
 						} else {
 							toast.error($i18n.t(`File not found.`));
 						}

src/lib/components/chat/Messages.svelte

@@ -12,6 +12,7 @@
 	import Placeholder from './Messages/Placeholder.svelte';
 	import Spinner from '../common/Spinner.svelte';
 	import { imageGenerations } from '$lib/apis/images';
+	import { copyToClipboard } from '$lib/utils';
 
 	const i18n = getContext('i18n');
 
@@ -42,40 +43,11 @@
 		element.scrollTop = element.scrollHeight;
 	};
 
-	const copyToClipboard = (text) => {
+	const copyToClipboardWithToast = async (text) => {
-		if (!navigator.clipboard) {
+		const res = await copyToClipboard(text);
-			var textArea = document.createElement('textarea');
+		if (res) {
-			textArea.value = text;
+			toast.success($i18n.t('Copying to clipboard was successful!'));
-
-			// Avoid scrolling to bottom
-			textArea.style.top = '0';
-			textArea.style.left = '0';
-			textArea.style.position = 'fixed';
-
-			document.body.appendChild(textArea);
-			textArea.focus();
-			textArea.select();
-
-			try {
-				var successful = document.execCommand('copy');
-				var msg = successful ? 'successful' : 'unsuccessful';
-				console.log('Fallback: Copying text command was ' + msg);
-			} catch (err) {
-				console.error('Fallback: Oops, unable to copy', err);
-			}
-
-			document.body.removeChild(textArea);
-			return;
 		}
-		navigator.clipboard.writeText(text).then(
-			function () {
-				console.log('Async: Copying to clipboard was successful!');
-				toast.success($i18n.t('Copying to clipboard was successful!'));
-			},
-			function (err) {
-				console.error('Async: Could not copy text: ', err);
-			}
-		);
 	};
 
 	const confirmEditMessage = async (messageId, content) => {
@@ -330,7 +302,7 @@
 								{confirmEditMessage}
 								{showPreviousMessage}
 								{showNextMessage}
-								{copyToClipboard}
+								copyToClipboard={copyToClipboardWithToast}
 							/>
 						{:else}
 							<ResponseMessage
@@ -344,7 +316,7 @@
 								{showPreviousMessage}
 								{showNextMessage}
 								{rateMessage}
-								{copyToClipboard}
+								copyToClipboard={copyToClipboardWithToast}
 								{continueGeneration}
 								{regenerateResponse}
 								on:save={async (e) => {

src/lib/components/chat/Messages/RateComment.svelte

@@ -1,31 +1,39 @@
 <script lang="ts">
 	import { toast } from 'svelte-sonner';
 
-	import { createEventDispatcher, onMount } from 'svelte';
+	import { createEventDispatcher, onMount, getContext } from 'svelte';
+
+	const i18n = getContext('i18n');
 
 	const dispatch = createEventDispatcher();
 
+	export let messageId = null;
 	export let show = false;
 	export let message;
 
-	const LIKE_REASONS = [
+	let LIKE_REASONS = [];
-		`Accurate information`,
+	let DISLIKE_REASONS = [];
-		`Followed instructions perfectly`,
-		`Showcased creativity`,
-		`Positive attitude`,
-		`Attention to detail`,
-		`Thorough explanation`,
-		`Other`
-	];
 
-	const DISLIKE_REASONS = [
+	function loadReasons() {
-		`Don't like the style`,
+		LIKE_REASONS = [
-		`Not factually correct`,
+			$i18n.t('Accurate information'),
-		`Didn't fully follow instructions`,
+			$i18n.t('Followed instructions perfectly'),
-		`Refused when it shouldn't have`,
+			$i18n.t('Showcased creativity'),
-		`Being Lazy`,
+			$i18n.t('Positive attitude'),
-		`Other`
+			$i18n.t('Attention to detail'),
-	];
+			$i18n.t('Thorough explanation'),
+			$i18n.t('Other')
+		];
+
+		DISLIKE_REASONS = [
+			$i18n.t("Don't like the style"),
+			$i18n.t('Not factually correct'),
+			$i18n.t("Didn't fully follow instructions"),
+			$i18n.t("Refused when it shouldn't have"),
+			$i18n.t('Being lazy'),
+			$i18n.t('Other')
+		];
+	}
 
 	let reasons = [];
 	let selectedReason = null;
@@ -40,6 +48,7 @@
 	onMount(() => {
 		selectedReason = message.annotation.reason;
 		comment = message.annotation.comment;
+		loadReasons();
 	});
 
 	const submitHandler = () => {
@@ -50,14 +59,17 @@
 
 		dispatch('submit');
 
-		toast.success('Thanks for your feedback!');
+		toast.success($i18n.t('Thanks for your feedback!'));
 		show = false;
 	};
 </script>
 
-<div class=" my-2.5 rounded-xl px-4 py-3 border dark:border-gray-850">
+<div
+	class=" my-2.5 rounded-xl px-4 py-3 border dark:border-gray-850"
+	id="message-feedback-{messageId}"
+>
 	<div class="flex justify-between items-center">
-		<div class=" text-sm">Tell us more:</div>
+		<div class=" text-sm">{$i18n.t('Tell us more:')}</div>
 
 		<button
 			on:click={() => {
@@ -81,9 +93,9 @@
 		<div class="flex flex-wrap gap-2 text-sm mt-2.5">
 			{#each reasons as reason}
 				<button
-					class="px-3.5 py-1 border dark:border-gray-850 dark:hover:bg-gray-850 {selectedReason ===
+					class="px-3.5 py-1 border dark:border-gray-850 hover:bg-gray-100 dark:hover:bg-gray-850 {selectedReason ===
 					reason
-						? 'dark:bg-gray-800'
+						? 'bg-gray-200 dark:bg-gray-800'
 						: ''} transition rounded-lg"
 					on:click={() => {
 						selectedReason = reason;
@@ -99,7 +111,7 @@
 		<textarea
 			bind:value={comment}
 			class="w-full text-sm px-1 py-2 bg-transparent outline-none resize-none rounded-xl"
-			placeholder="Feel free to add specific details"
+			placeholder={$i18n.t('Feel free to add specific details')}
 			rows="2"
 		/>
 	</div>

src/lib/components/chat/Messages/ResponseMessage.svelte

@@ -15,7 +15,7 @@
 	const dispatch = createEventDispatcher();
 
 	import { config, settings } from '$lib/stores';
-	import { synthesizeOpenAISpeech } from '$lib/apis/openai';
+	import { synthesizeOpenAISpeech } from '$lib/apis/audio';
 	import { imageGenerations } from '$lib/apis/images';
 	import {
 		approximateToHumanReadable,
@@ -176,10 +176,12 @@
 
 	const toggleSpeakMessage = async () => {
 		if (speaking) {
-			speechSynthesis.cancel();
+			try {
+				speechSynthesis.cancel();
 
-			sentencesAudio[speakingIdx].pause();
+				sentencesAudio[speakingIdx].pause();
-			sentencesAudio[speakingIdx].currentTime = 0;
+				sentencesAudio[speakingIdx].currentTime = 0;
+			} catch {}
 
 			speaking = null;
 			speakingIdx = null;
@@ -221,6 +223,10 @@
 						sentence
 					).catch((error) => {
 						toast.error(error);
+
+						speaking = null;
+						loadingSpeech = false;
+
 						return null;
 					});
 
@@ -230,7 +236,6 @@
 						const audio = new Audio(blobUrl);
 						sentencesAudio[idx] = audio;
 						loadingSpeech = false;
-
 						lastPlayedAudioPromise = lastPlayedAudioPromise.then(() => playAudio(idx));
 					}
 				}
@@ -551,6 +556,12 @@
 													on:click={() => {
 														rateMessage(message.id, 1);
 														showRateComment = true;
+
+														window.setTimeout(() => {
+															document
+																.getElementById(`message-feedback-${message.id}`)
+																?.scrollIntoView();
+														}, 0);
 													}}
 												>
 													<svg
@@ -580,6 +591,11 @@
 													on:click={() => {
 														rateMessage(message.id, -1);
 														showRateComment = true;
+														window.setTimeout(() => {
+															document
+																.getElementById(`message-feedback-${message.id}`)
+																?.scrollIntoView();
+														}, 0);
 													}}
 												>
 													<svg
@@ -839,6 +855,7 @@
 
 								{#if showRateComment}
 									<RateComment
+										messageId={message.id}
 										bind:show={showRateComment}
 										bind:message
 										on:submit={() => {

src/lib/components/chat/Messages/UserMessage.svelte

@@ -176,10 +176,23 @@
 							e.target.style.height = '';
 							e.target.style.height = `${e.target.scrollHeight}px`;
 						}}
+						on:keydown={(e) => {
+							if (e.key === 'Escape') {
+								document.getElementById('close-edit-message-button')?.click();
+							}
+
+							const isCmdOrCtrlPressed = e.metaKey || e.ctrlKey;
+							const isEnterPressed = e.key === 'Enter';
+
+							if (isCmdOrCtrlPressed && isEnterPressed) {
+								document.getElementById('save-edit-message-button')?.click();
+							}
+						}}
 					/>
 
 					<div class=" mt-2 mb-1 flex justify-center space-x-2 text-sm font-medium">
 						<button
+							id="save-edit-message-button"
 							class="px-4 py-2 bg-emerald-600 hover:bg-emerald-700 text-gray-100 transition rounded-lg"
 							on:click={() => {
 								editMessageConfirmHandler();
@@ -189,6 +202,7 @@
 						</button>
 
 						<button
+							id="close-edit-message-button"
 							class=" px-4 py-2 hover:bg-gray-100 dark:bg-gray-800 dark:hover:bg-gray-700 text-gray-700 dark:text-gray-100 transition outline outline-1 outline-gray-200 dark:outline-gray-600 rounded-lg"
 							on:click={() => {
 								cancelEditMessage();

src/lib/components/chat/Settings/Audio.svelte

@@ -1,4 +1,6 @@
 <script lang="ts">
+	import { getAudioConfig, updateAudioConfig } from '$lib/apis/audio';
+	import { user } from '$lib/stores';
 	import { createEventDispatcher, onMount, getContext } from 'svelte';
 	import { toast } from 'svelte-sonner';
 	const dispatch = createEventDispatcher();
@@ -9,6 +11,9 @@
 
 	// Audio
 
+	let OpenAIUrl = '';
+	let OpenAIKey = '';
+
 	let STTEngines = ['', 'openai'];
 	let STTEngine = '';
 
@@ -69,6 +74,20 @@
 		saveSettings({ speechAutoSend: speechAutoSend });
 	};
 
+	const updateConfigHandler = async () => {
+		if (TTSEngine === 'openai') {
+			const res = await updateAudioConfig(localStorage.token, {
+				url: OpenAIUrl,
+				key: OpenAIKey
+			});
+
+			if (res) {
+				OpenAIUrl = res.OPENAI_API_BASE_URL;
+				OpenAIKey = res.OPENAI_API_KEY;
+			}
+		}
+	};
+
 	onMount(async () => {
 		let settings = JSON.parse(localStorage.getItem('settings') ?? '{}');
 
@@ -85,12 +104,24 @@
 		} else {
 			getWebAPIVoices();
 		}
+
+		if ($user.role === 'admin') {
+			const res = await getAudioConfig(localStorage.token);
+
+			if (res) {
+				OpenAIUrl = res.OPENAI_API_BASE_URL;
+				OpenAIKey = res.OPENAI_API_KEY;
+			}
+		}
 	});
 </script>
 
 <form
 	class="flex flex-col h-full justify-between space-y-3 text-sm"
-	on:submit|preventDefault={() => {
+	on:submit|preventDefault={async () => {
+		if ($user.role === 'admin') {
+			await updateConfigHandler();
+		}
 		saveSettings({
 			audio: {
 				STTEngine: STTEngine !== '' ? STTEngine : undefined,
@@ -101,7 +132,7 @@
 		dispatch('save');
 	}}
 >
-	<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-80">
+	<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-[22rem]">
 		<div>
 			<div class=" mb-1 text-sm font-medium">{$i18n.t('STT Settings')}</div>
 
@@ -196,6 +227,26 @@
 				</div>
 			</div>
 
+			{#if $user.role === 'admin'}
+				{#if TTSEngine === 'openai'}
+					<div class="mt-1 flex gap-2 mb-1">
+						<input
+							class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+							placeholder={$i18n.t('API Base URL')}
+							bind:value={OpenAIUrl}
+							required
+						/>
+
+						<input
+							class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+							placeholder={$i18n.t('API Key')}
+							bind:value={OpenAIKey}
+							required
+						/>
+					</div>
+				{/if}
+			{/if}
+
 			<div class=" py-0.5 flex w-full justify-between">
 				<div class=" self-center text-xs font-medium">{$i18n.t('Auto-playback response')}</div>
 
@@ -223,7 +274,7 @@
 				<div class="flex w-full">
 					<div class="flex-1">
 						<select
-							class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-800 outline-none"
+							class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
 							bind:value={speaker}
 							placeholder="Select a voice"
 						>
@@ -241,16 +292,18 @@
 				<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Set Voice')}</div>
 				<div class="flex w-full">
 					<div class="flex-1">
-						<select
+						<input
-							class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-800 outline-none"
+							list="voice-list"
+							class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
 							bind:value={speaker}
 							placeholder="Select a voice"
-						>
+						/>
+
+						<datalist id="voice-list">
 							{#each voices as voice}
-								<option value={voice.name} class="bg-gray-100 dark:bg-gray-700">{voice.name}</option
+								<option value={voice.name} />
-								>
 							{/each}
-						</select>
+						</datalist>
 					</div>
 				</div>
 			</div>

src/lib/components/chat/Settings/Chats.svelte

@@ -301,7 +301,7 @@
 			</button>
 		{/if}
 
-		{#if $user?.role === 'admin'}
+		{#if $user?.role === 'admin' && ($config?.admin_export_enabled ?? true)}
 			<hr class=" dark:border-gray-700" />
 
 			<button

src/lib/components/chat/Settings/Images.svelte

@@ -15,8 +15,8 @@
 		updateImageSize,
 		getImageSteps,
 		updateImageSteps,
-		getOpenAIKey,
+		getOpenAIConfig,
-		updateOpenAIKey
+		updateOpenAIConfig
 	} from '$lib/apis/images';
 	import { getBackendConfig } from '$lib/apis';
 	const dispatch = createEventDispatcher();
@@ -33,6 +33,7 @@
 	let AUTOMATIC1111_BASE_URL = '';
 	let COMFYUI_BASE_URL = '';
 
+	let OPENAI_API_BASE_URL = '';
 	let OPENAI_API_KEY = '';
 
 	let selectedModel = '';
@@ -131,7 +132,10 @@
 			AUTOMATIC1111_BASE_URL = URLS.AUTOMATIC1111_BASE_URL;
 			COMFYUI_BASE_URL = URLS.COMFYUI_BASE_URL;
 
-			OPENAI_API_KEY = await getOpenAIKey(localStorage.token);
+			const config = await getOpenAIConfig(localStorage.token);
+
+			OPENAI_API_KEY = config.OPENAI_API_KEY;
+			OPENAI_API_BASE_URL = config.OPENAI_API_BASE_URL;
 
 			imageSize = await getImageSize(localStorage.token);
 			steps = await getImageSteps(localStorage.token);
@@ -149,7 +153,7 @@
 		loading = true;
 
 		if (imageGenerationEngine === 'openai') {
-			await updateOpenAIKey(localStorage.token, OPENAI_API_KEY);
+			await updateOpenAIConfig(localStorage.token, OPENAI_API_BASE_URL, OPENAI_API_KEY);
 		}
 
 		await updateDefaultImageGenerationModel(localStorage.token, selectedModel);
@@ -300,13 +304,22 @@
 				</button>
 			</div>
 		{:else if imageGenerationEngine === 'openai'}
-			<div class=" mb-2.5 text-sm font-medium">{$i18n.t('OpenAI API Key')}</div>
+			<div>
-			<div class="flex w-full">
+				<div class=" mb-1.5 text-sm font-medium">{$i18n.t('OpenAI API Config')}</div>
-				<div class="flex-1 mr-2">
+
+				<div class="flex gap-2 mb-1">
 					<input
 						class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
-						placeholder={$i18n.t('Enter API Key')}
+						placeholder={$i18n.t('API Base URL')}
+						bind:value={OPENAI_API_BASE_URL}
+						required
+					/>
+
+					<input
+						class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+						placeholder={$i18n.t('API Key')}
 						bind:value={OPENAI_API_KEY}
+						required
 					/>
 				</div>
 			</div>
@@ -319,19 +332,39 @@
 				<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Set Default Model')}</div>
 				<div class="flex w-full">
 					<div class="flex-1 mr-2">
-						<select
+						{#if imageGenerationEngine === 'openai' && !OPENAI_API_BASE_URL.includes('https://api.openai.com')}
-							class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+							<div class="flex w-full">
-							bind:value={selectedModel}
+								<div class="flex-1">
-							placeholder={$i18n.t('Select a model')}
+									<input
-							required
+										list="model-list"
-						>
+										class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
-							{#if !selectedModel}
+										bind:value={selectedModel}
-								<option value="" disabled selected>{$i18n.t('Select a model')}</option>
+										placeholder="Select a model"
-							{/if}
+									/>
-							{#each models ?? [] as model}
+
-								<option value={model.id} class="bg-gray-100 dark:bg-gray-700">{model.name}</option>
+									<datalist id="model-list">
-							{/each}
+										{#each models ?? [] as model}
-						</select>
+											<option value={model.id}>{model.name}</option>
+										{/each}
+									</datalist>
+								</div>
+							</div>
+						{:else}
+							<select
+								class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+								bind:value={selectedModel}
+								placeholder={$i18n.t('Select a model')}
+								required
+							>
+								{#if !selectedModel}
+									<option value="" disabled selected>{$i18n.t('Select a model')}</option>
+								{/if}
+								{#each models ?? [] as model}
+									<option value={model.id} class="bg-gray-100 dark:bg-gray-700">{model.name}</option
+									>
+								{/each}
+							</select>
+						{/if}
 					</div>
 				</div>
 			</div>

src/lib/components/chat/Settings/Interface.svelte

@@ -17,11 +17,17 @@
 	let titleAutoGenerateModelExternal = '';
 	let fullScreenMode = false;
 	let titleGenerationPrompt = '';
+	let splitLargeChunks = false;
 
 	// Interface
 	let promptSuggestions = [];
 	let showUsername = false;
 
+	const toggleSplitLargeChunks = async () => {
+		splitLargeChunks = !splitLargeChunks;
+		saveSettings({ splitLargeChunks: splitLargeChunks });
+	};
+
 	const toggleFullScreenMode = async () => {
 		fullScreenMode = !fullScreenMode;
 		saveSettings({ fullScreenMode: fullScreenMode });
@@ -197,6 +203,28 @@
 					</button>
 				</div>
 			</div>
+
+			<div>
+				<div class=" py-0.5 flex w-full justify-between">
+					<div class=" self-center text-xs font-medium">
+						{$i18n.t('Fluidly stream large external response chunks')}
+					</div>
+
+					<button
+						class="p-1 px-3 text-xs flex rounded transition"
+						on:click={() => {
+							toggleSplitLargeChunks();
+						}}
+						type="button"
+					>
+						{#if splitLargeChunks === true}
+							<span class="ml-2 self-center">{$i18n.t('On')}</span>
+						{:else}
+							<span class="ml-2 self-center">{$i18n.t('Off')}</span>
+						{/if}
+					</button>
+				</div>
+			</div>
 		</div>
 
 		<hr class=" dark:border-gray-700" />
@@ -288,20 +316,20 @@
 								<div class="flex border-b dark:border-gray-600 w-full">
 									<input
 										class="px-3 py-1.5 text-xs w-full bg-transparent outline-none border-r dark:border-gray-600"
-										placeholder="Title (e.g. Tell me a fun fact)"
+										placeholder={$i18n.t('Title (e.g. Tell me a fun fact)')}
 										bind:value={prompt.title[0]}
 									/>
 
 									<input
 										class="px-3 py-1.5 text-xs w-full bg-transparent outline-none border-r dark:border-gray-600"
-										placeholder="Subtitle (e.g. about the Roman Empire)"
+										placeholder={$i18n.t('Subtitle (e.g. about the Roman Empire)')}
 										bind:value={prompt.title[1]}
 									/>
 								</div>
 
 								<input
 									class="px-3 py-1.5 text-xs w-full bg-transparent outline-none border-r dark:border-gray-600"
-									placeholder="Prompt (e.g. Tell me a fun fact about the Roman Empire)"
+									placeholder={$i18n.t('Prompt (e.g. Tell me a fun fact about the Roman Empire)')}
 									bind:value={prompt.content}
 								/>
 							</div>

src/lib/components/chat/Settings/Models.svelte

@@ -13,7 +13,7 @@
 		uploadModel
 	} from '$lib/apis/ollama';
 	import { WEBUI_API_BASE_URL, WEBUI_BASE_URL } from '$lib/constants';
-	import { WEBUI_NAME, models, user } from '$lib/stores';
+	import { WEBUI_NAME, models, MODEL_DOWNLOAD_POOL, user } from '$lib/stores';
 	import { splitStream } from '$lib/utils';
 	import { onMount, getContext } from 'svelte';
 	import { addLiteLLMModel, deleteLiteLLMModel, getLiteLLMModelInfo } from '$lib/apis/litellm';
@@ -35,7 +35,7 @@
 	let liteLLMRPM = '';
 	let liteLLMMaxTokens = '';
 
-	let deleteLiteLLMModelId = '';
+	let deleteLiteLLMModelName = '';
 
 	$: liteLLMModelName = liteLLMModel;
 
@@ -50,12 +50,6 @@
 	let showExperimentalOllama = false;
 	let ollamaVersion = '';
 	const MAX_PARALLEL_DOWNLOADS = 3;
-	const modelDownloadQueue = queue(
-		(task: { modelName: string }, cb) =>
-			pullModelHandlerProcessor({ modelName: task.modelName, callback: cb }),
-		MAX_PARALLEL_DOWNLOADS
-	);
-	let modelDownloadStatus: Record<string, any> = {};
 
 	let modelTransferring = false;
 	let modelTag = '';
@@ -140,7 +134,8 @@
 
 	const pullModelHandler = async () => {
 		const sanitizedModelTag = modelTag.trim().replace(/^ollama\s+(run|pull)\s+/, '');
-		if (modelDownloadStatus[sanitizedModelTag]) {
+		console.log($MODEL_DOWNLOAD_POOL);
+		if ($MODEL_DOWNLOAD_POOL[sanitizedModelTag]) {
 			toast.error(
 				$i18n.t(`Model '{{modelTag}}' is already in queue for downloading.`, {
 					modelTag: sanitizedModelTag
@@ -148,40 +143,117 @@
 			);
 			return;
 		}
-		if (Object.keys(modelDownloadStatus).length === 3) {
+		if (Object.keys($MODEL_DOWNLOAD_POOL).length === MAX_PARALLEL_DOWNLOADS) {
 			toast.error(
 				$i18n.t('Maximum of 3 models can be downloaded simultaneously. Please try again later.')
 			);
 			return;
 		}
 
-		modelTransferring = true;
+		const res = await pullModel(localStorage.token, sanitizedModelTag, '0').catch((error) => {
+			toast.error(error);
+			return null;
+		});
 
-		modelDownloadQueue.push(
+		if (res) {
-			{ modelName: sanitizedModelTag },
+			const reader = res.body
-			async (data: { modelName: string; success: boolean; error?: Error }) => {
+				.pipeThrough(new TextDecoderStream())
-				const { modelName } = data;
+				.pipeThrough(splitStream('\n'))
-				// Remove the downloaded model
+				.getReader();
-				delete modelDownloadStatus[modelName];
 
-				modelDownloadStatus = { ...modelDownloadStatus };
+			while (true) {
+				try {
+					const { value, done } = await reader.read();
+					if (done) break;
 
-				if (!data.success) {
+					let lines = value.split('\n');
-					toast.error(data.error);
-				} else {
-					toast.success(
-						$i18n.t(`Model '{{modelName}}' has been successfully downloaded.`, { modelName })
-					);
 
-					const notification = new Notification($WEBUI_NAME, {
+					for (const line of lines) {
-						body: $i18n.t(`Model '{{modelName}}' has been successfully downloaded.`, { modelName }),
+						if (line !== '') {
-						icon: `${WEBUI_BASE_URL}/static/favicon.png`
+							let data = JSON.parse(line);
-					});
+							console.log(data);
+							if (data.error) {
+								throw data.error;
+							}
+							if (data.detail) {
+								throw data.detail;
+							}
 
-					models.set(await getModels());
+							if (data.id) {
+								MODEL_DOWNLOAD_POOL.set({
+									...$MODEL_DOWNLOAD_POOL,
+									[sanitizedModelTag]: {
+										...$MODEL_DOWNLOAD_POOL[sanitizedModelTag],
+										requestId: data.id,
+										reader,
+										done: false
+									}
+								});
+								console.log(data);
+							}
+
+							if (data.status) {
+								if (data.digest) {
+									let downloadProgress = 0;
+									if (data.completed) {
+										downloadProgress = Math.round((data.completed / data.total) * 1000) / 10;
+									} else {
+										downloadProgress = 100;
+									}
+
+									MODEL_DOWNLOAD_POOL.set({
+										...$MODEL_DOWNLOAD_POOL,
+										[sanitizedModelTag]: {
+											...$MODEL_DOWNLOAD_POOL[sanitizedModelTag],
+											pullProgress: downloadProgress,
+											digest: data.digest
+										}
+									});
+								} else {
+									toast.success(data.status);
+
+									MODEL_DOWNLOAD_POOL.set({
+										...$MODEL_DOWNLOAD_POOL,
+										[sanitizedModelTag]: {
+											...$MODEL_DOWNLOAD_POOL[sanitizedModelTag],
+											done: data.status === 'success'
+										}
+									});
+								}
+							}
+						}
+					}
+				} catch (error) {
+					console.log(error);
+					if (typeof error !== 'string') {
+						error = error.message;
+					}
+
+					toast.error(error);
+					// opts.callback({ success: false, error, modelName: opts.modelName });
 				}
 			}
-		);
+
+			console.log($MODEL_DOWNLOAD_POOL[sanitizedModelTag]);
+
+			if ($MODEL_DOWNLOAD_POOL[sanitizedModelTag].done) {
+				toast.success(
+					$i18n.t(`Model '{{modelName}}' has been successfully downloaded.`, {
+						modelName: sanitizedModelTag
+					})
+				);
+
+				models.set(await getModels(localStorage.token));
+			} else {
+				toast.error('Download canceled');
+			}
+
+			delete $MODEL_DOWNLOAD_POOL[sanitizedModelTag];
+
+			MODEL_DOWNLOAD_POOL.set({
+				...$MODEL_DOWNLOAD_POOL
+			});
+		}
 
 		modelTag = '';
 		modelTransferring = false;
@@ -352,88 +424,18 @@
 		models.set(await getModels());
 	};
 
-	const pullModelHandlerProcessor = async (opts: { modelName: string; callback: Function }) => {
+	const cancelModelPullHandler = async (model: string) => {
-		const res = await pullModel(localStorage.token, opts.modelName, selectedOllamaUrlIdx).catch(
+		const { reader, requestId } = $MODEL_DOWNLOAD_POOL[model];
-			(error) => {
+		if (reader) {
-				opts.callback({ success: false, error, modelName: opts.modelName });
+			await reader.cancel();
-				return null;
-			}
-		);
 
-		if (res) {
+			await cancelOllamaRequest(localStorage.token, requestId);
-			const reader = res.body
+			delete $MODEL_DOWNLOAD_POOL[model];
-				.pipeThrough(new TextDecoderStream())
+			MODEL_DOWNLOAD_POOL.set({
-				.pipeThrough(splitStream('\n'))
+				...$MODEL_DOWNLOAD_POOL
-				.getReader();
+			});
-
+			await deleteModel(localStorage.token, model);
-			while (true) {
+			toast.success(`${model} download has been canceled`);
-				try {
-					const { value, done } = await reader.read();
-					if (done) break;
-
-					let lines = value.split('\n');
-
-					for (const line of lines) {
-						if (line !== '') {
-							let data = JSON.parse(line);
-							console.log(data);
-							if (data.error) {
-								throw data.error;
-							}
-							if (data.detail) {
-								throw data.detail;
-							}
-
-							if (data.id) {
-								modelDownloadStatus[opts.modelName] = {
-									...modelDownloadStatus[opts.modelName],
-									requestId: data.id,
-									reader,
-									done: false
-								};
-								console.log(data);
-							}
-
-							if (data.status) {
-								if (data.digest) {
-									let downloadProgress = 0;
-									if (data.completed) {
-										downloadProgress = Math.round((data.completed / data.total) * 1000) / 10;
-									} else {
-										downloadProgress = 100;
-									}
-									modelDownloadStatus[opts.modelName] = {
-										...modelDownloadStatus[opts.modelName],
-										pullProgress: downloadProgress,
-										digest: data.digest
-									};
-								} else {
-									toast.success(data.status);
-
-									modelDownloadStatus[opts.modelName] = {
-										...modelDownloadStatus[opts.modelName],
-										done: data.status === 'success'
-									};
-								}
-							}
-						}
-					}
-				} catch (error) {
-					console.log(error);
-					if (typeof error !== 'string') {
-						error = error.message;
-					}
-					opts.callback({ success: false, error, modelName: opts.modelName });
-				}
-			}
-
-			console.log(modelDownloadStatus[opts.modelName]);
-
-			if (modelDownloadStatus[opts.modelName].done) {
-				opts.callback({ success: true, modelName: opts.modelName });
-			} else {
-				opts.callback({ success: false, error: 'Download canceled', modelName: opts.modelName });
-			}
 		}
 	};
 
@@ -472,7 +474,7 @@
 	};
 
 	const deleteLiteLLMModelHandler = async () => {
-		const res = await deleteLiteLLMModel(localStorage.token, deleteLiteLLMModelId).catch(
+		const res = await deleteLiteLLMModel(localStorage.token, deleteLiteLLMModelName).catch(
 			(error) => {
 				toast.error(error);
 				return null;
@@ -485,7 +487,7 @@
 			}
 		}
 
-		deleteLiteLLMModelId = '';
+		deleteLiteLLMModelName = '';
 		liteLLMModelInfo = await getLiteLLMModelInfo(localStorage.token);
 		models.set(await getModels());
 	};
@@ -503,18 +505,6 @@
 		ollamaVersion = await getOllamaVersion(localStorage.token).catch((error) => false);
 		liteLLMModelInfo = await getLiteLLMModelInfo(localStorage.token);
 	});
-
-	const cancelModelPullHandler = async (model: string) => {
-		const { reader, requestId } = modelDownloadStatus[model];
-		if (reader) {
-			await reader.cancel();
-
-			await cancelOllamaRequest(localStorage.token, requestId);
-			delete modelDownloadStatus[model];
-			await deleteModel(localStorage.token, model);
-			toast.success(`${model} download has been canceled`);
-		}
-	};
 </script>
 
 <div class="flex flex-col h-full justify-between text-sm">
@@ -643,9 +633,9 @@
 							>
 						</div>
 
-						{#if Object.keys(modelDownloadStatus).length > 0}
+						{#if Object.keys($MODEL_DOWNLOAD_POOL).length > 0}
-							{#each Object.keys(modelDownloadStatus) as model}
+							{#each Object.keys($MODEL_DOWNLOAD_POOL) as model}
-								{#if 'pullProgress' in modelDownloadStatus[model]}
+								{#if 'pullProgress' in $MODEL_DOWNLOAD_POOL[model]}
 									<div class="flex flex-col">
 										<div class="font-medium mb-1">{model}</div>
 										<div class="">
@@ -655,10 +645,10 @@
 														class="dark:bg-gray-600 bg-gray-500 text-xs font-medium text-gray-100 text-center p-0.5 leading-none rounded-full"
 														style="width: {Math.max(
 															15,
-															modelDownloadStatus[model].pullProgress ?? 0
+															$MODEL_DOWNLOAD_POOL[model].pullProgress ?? 0
 														)}%"
 													>
-														{modelDownloadStatus[model].pullProgress ?? 0}%
+														{$MODEL_DOWNLOAD_POOL[model].pullProgress ?? 0}%
 													</div>
 												</div>
 
@@ -689,9 +679,9 @@
 													</button>
 												</Tooltip>
 											</div>
-											{#if 'digest' in modelDownloadStatus[model]}
+											{#if 'digest' in $MODEL_DOWNLOAD_POOL[model]}
 												<div class="mt-1 text-xs dark:text-gray-500" style="font-size: 0.5rem;">
-													{modelDownloadStatus[model].digest}
+													{$MODEL_DOWNLOAD_POOL[model].digest}
 												</div>
 											{/if}
 										</div>
@@ -1099,14 +1089,14 @@
 								<div class="flex-1 mr-2">
 									<select
 										class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
-										bind:value={deleteLiteLLMModelId}
+										bind:value={deleteLiteLLMModelName}
 										placeholder={$i18n.t('Select a model')}
 									>
-										{#if !deleteLiteLLMModelId}
+										{#if !deleteLiteLLMModelName}
 											<option value="" disabled selected>{$i18n.t('Select a model')}</option>
 										{/if}
 										{#each liteLLMModelInfo as model}
-											<option value={model.model_info.id} class="bg-gray-100 dark:bg-gray-700"
+											<option value={model.model_name} class="bg-gray-100 dark:bg-gray-700"
 												>{model.model_name}</option
 											>
 										{/each}

src/lib/components/chat/SettingsModal.svelte

@@ -3,9 +3,7 @@
 	import { toast } from 'svelte-sonner';
 	import { models, settings, user } from '$lib/stores';
 
-	import { getOllamaModels } from '$lib/apis/ollama';
+	import { getModels as _getModels } from '$lib/utils';
-	import { getOpenAIModels } from '$lib/apis/openai';
-	import { getLiteLLMModels } from '$lib/apis/litellm';
 
 	import Modal from '../common/Modal.svelte';
 	import Account from './Settings/Account.svelte';
@@ -29,33 +27,11 @@
 		localStorage.setItem('settings', JSON.stringify($settings));
 	};
 
-	let selectedTab = 'general';
-
 	const getModels = async () => {
-		let models = await Promise.all([
+		return await _getModels(localStorage.token);
-			await getOllamaModels(localStorage.token).catch((error) => {
-				console.log(error);
-				return null;
-			}),
-			await getOpenAIModels(localStorage.token).catch((error) => {
-				console.log(error);
-				return null;
-			}),
-			await getLiteLLMModels(localStorage.token).catch((error) => {
-				console.log(error);
-				return null;
-			})
-		]);
-
-		models = models
-			.filter((models) => models)
-			.reduce((a, e, i, arr) => a.concat(e, ...(i < arr.length - 1 ? [{ name: 'hr' }] : [])), []);
-
-		// models.push(...(ollamaModels ? [{ name: 'hr' }, ...ollamaModels] : []));
-		// models.push(...(openAIModels ? [{ name: 'hr' }, ...openAIModels] : []));
-		// models.push(...(liteLLMModels ? [{ name: 'hr' }, ...liteLLMModels] : []));
-		return models;
 	};
+
+	let selectedTab = 'general';
 </script>
 
 <Modal bind:show>

src/lib/components/chat/ShareChatModal.svelte

@@ -3,24 +3,27 @@
 
 	import { toast } from 'svelte-sonner';
 	import { deleteSharedChatById, getChatById, shareChatById } from '$lib/apis/chats';
-	import { chatId, modelfiles } from '$lib/stores';
+	import { modelfiles } from '$lib/stores';
 	import { copyToClipboard } from '$lib/utils';
 
 	import Modal from '../common/Modal.svelte';
 	import Link from '../icons/Link.svelte';
 
+	export let chatId;
+
 	let chat = null;
+	let shareUrl = null;
 	const i18n = getContext('i18n');
 
 	const shareLocalChat = async () => {
 		const _chat = chat;
 
-		const sharedChat = await shareChatById(localStorage.token, $chatId);
+		const sharedChat = await shareChatById(localStorage.token, chatId);
-		const chatShareUrl = `${window.location.origin}/s/${sharedChat.id}`;
+		shareUrl = `${window.location.origin}/s/${sharedChat.id}`;
+		console.log(shareUrl);
+		chat = await getChatById(localStorage.token, chatId);
 
-		toast.success($i18n.t('Copied shared chat URL to clipboard!'));
+		return shareUrl;
-		copyToClipboard(chatShareUrl);
-		chat = await getChatById(localStorage.token, $chatId);
 	};
 
 	const shareChat = async () => {
@@ -56,8 +59,8 @@
 
 	$: if (show) {
 		(async () => {
-			if ($chatId) {
+			if (chatId) {
-				chat = await getChatById(localStorage.token, $chatId);
+				chat = await getChatById(localStorage.token, chatId);
 			} else {
 				chat = null;
 				console.log(chat);
@@ -101,10 +104,10 @@
 						<button
 							class="underline"
 							on:click={async () => {
-								const res = await deleteSharedChatById(localStorage.token, $chatId);
+								const res = await deleteSharedChatById(localStorage.token, chatId);
 
 								if (res) {
-									chat = await getChatById(localStorage.token, $chatId);
+									chat = await getChatById(localStorage.token, chatId);
 								}
 							}}>delete this link</button
 						> and create a new shared link.
@@ -131,8 +134,37 @@
 							<button
 								class=" self-center flex items-center gap-1 px-3.5 py-2 rounded-xl text-sm font-medium bg-emerald-600 hover:bg-emerald-500 text-white"
 								type="button"
-								on:click={() => {
+								on:click={async () => {
-									shareLocalChat();
+									const isSafari = /^((?!chrome|android).)*safari/i.test(navigator.userAgent);
+
+									if (isSafari) {
+										// Oh, Safari, you're so special, let's give you some extra love and attention
+										console.log('isSafari');
+
+										const getUrlPromise = async () => {
+											const url = await shareLocalChat();
+											return new Blob([url], { type: 'text/plain' });
+										};
+
+										navigator.clipboard
+											.write([
+												new ClipboardItem({
+													'text/plain': getUrlPromise()
+												})
+											])
+											.then(() => {
+												console.log('Async: Copying to clipboard was successful!');
+												return true;
+											})
+											.catch((error) => {
+												console.error('Async: Could not copy text: ', error);
+												return false;
+											});
+									} else {
+										copyToClipboard(await shareLocalChat());
+									}
+
+									toast.success($i18n.t('Copied shared chat URL to clipboard!'));
 									show = false;
 								}}
 							>

src/lib/components/common/Modal.svelte

@@ -15,8 +15,10 @@
 			return 'w-[16rem]';
 		} else if (size === 'sm') {
 			return 'w-[30rem]';
-		} else {
+		} else if (size === 'md') {
 			return 'w-[44rem]';
+		} else {
+			return 'w-[48rem]';
 		}
 	};
 
@@ -47,7 +49,7 @@
 	<!-- svelte-ignore a11y-no-static-element-interactions -->
 	<div
 		bind:this={modalElement}
-		class=" fixed top-0 right-0 left-0 bottom-0 bg-black/60 w-full min-h-screen h-screen flex justify-center z-50 overflow-hidden overscroll-contain"
+		class=" fixed top-0 right-0 left-0 bottom-0 bg-black/60 w-full min-h-screen h-screen flex justify-center z-[9999] overflow-hidden overscroll-contain"
 		in:fade={{ duration: 10 }}
 		on:click={() => {
 			show = false;

src/lib/components/documents/Settings/General.svelte

@@ -29,8 +29,8 @@
 	let embeddingEngine = '';
 	let embeddingModel = '';
 
-	let openAIKey = '';
+	let OpenAIKey = '';
-	let openAIUrl = '';
+	let OpenAIUrl = '';
 
 	let chunkSize = 0;
 	let chunkOverlap = 0;
@@ -79,7 +79,7 @@
 			return;
 		}
 
-		if ((embeddingEngine === 'openai' && openAIKey === '') || openAIUrl === '') {
+		if ((embeddingEngine === 'openai' && OpenAIKey === '') || OpenAIUrl === '') {
 			toast.error($i18n.t('OpenAI URL/Key required.'));
 			return;
 		}
@@ -93,8 +93,8 @@
 			...(embeddingEngine === 'openai'
 				? {
 						openai_config: {
-							key: openAIKey,
+							key: OpenAIKey,
-							url: openAIUrl
+							url: OpenAIUrl
 						}
 				  }
 				: {})
@@ -133,8 +133,8 @@
 			embeddingEngine = embeddingConfig.embedding_engine;
 			embeddingModel = embeddingConfig.embedding_model;
 
-			openAIKey = embeddingConfig.openai_config.key;
+			OpenAIKey = embeddingConfig.openai_config.key;
-			openAIUrl = embeddingConfig.openai_config.url;
+			OpenAIUrl = embeddingConfig.openai_config.url;
 		}
 	};
 
@@ -180,7 +180,7 @@
 							}
 						}}
 					>
-						<option value="">{$i18n.t('Default (SentenceTransformer)')}</option>
+						<option value="">{$i18n.t('Default (SentenceTransformers)')}</option>
 						<option value="ollama">{$i18n.t('Ollama')}</option>
 						<option value="openai">{$i18n.t('OpenAI')}</option>
 					</select>
@@ -192,14 +192,14 @@
 					<input
 						class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
 						placeholder={$i18n.t('API Base URL')}
-						bind:value={openAIUrl}
+						bind:value={OpenAIUrl}
 						required
 					/>
 
 					<input
 						class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
 						placeholder={$i18n.t('API Key')}
-						bind:value={openAIKey}
+						bind:value={OpenAIKey}
 						required
 					/>
 				</div>

src/lib/components/icons/ArchiveBox.svelte

@@ -0,0 +1,19 @@
+<script lang="ts">
+	export let className = 'size-3.5';
+	export let strokeWidth = '2.5';
+</script>
+
+<svg
+	xmlns="http://www.w3.org/2000/svg"
+	fill="none"
+	viewBox="0 0 24 24"
+	stroke-width={strokeWidth}
+	stroke="currentColor"
+	class={className}
+>
+	<path
+		stroke-linecap="round"
+		stroke-linejoin="round"
+		d="m20.25 7.5-.625 10.632a2.25 2.25 0 0 1-2.247 2.118H6.622a2.25 2.25 0 0 1-2.247-2.118L3.75 7.5M10 11.25h4M3.375 7.5h17.25c.621 0 1.125-.504 1.125-1.125v-1.5c0-.621-.504-1.125-1.125-1.125H3.375c-.621 0-1.125.504-1.125 1.125v1.5c0 .621.504 1.125 1.125 1.125Z"
+	/>
+</svg>

src/lib/components/icons/Share.svelte

@@ -0,0 +1,11 @@
+<script lang="ts">
+	export let className = 'w-4 h-4';
+</script>
+
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor" class={className}>
+	<path
+		fill-rule="evenodd"
+		d="M15.75 4.5a3 3 0 1 1 .825 2.066l-8.421 4.679a3.002 3.002 0 0 1 0 1.51l8.421 4.679a3 3 0 1 1-.729 1.31l-8.421-4.678a3 3 0 1 1 0-4.132l8.421-4.679a3 3 0 0 1-.096-.755Z"
+		clip-rule="evenodd"
+	/>
+</svg>

src/lib/components/layout/Navbar.svelte

@@ -25,7 +25,7 @@
 	let showDownloadChatModal = false;
 </script>
 
-<ShareChatModal bind:show={showShareChatModal} />
+<ShareChatModal bind:show={showShareChatModal} chatId={$chatId} />
 <nav id="nav" class=" sticky py-2.5 top-0 flex flex-row justify-center z-30">
 	<div
 		class=" flex {$settings?.fullScreenMode ?? null ? 'max-w-full' : 'max-w-3xl'} 

src/lib/components/layout/Sidebar.svelte

@@ -17,13 +17,17 @@
 		getChatById,
 		getChatListByTagName,
 		updateChatById,
-		getAllChatTags
+		getAllChatTags,
+		archiveChatById
 	} from '$lib/apis/chats';
 	import { toast } from 'svelte-sonner';
 	import { fade, slide } from 'svelte/transition';
 	import { WEBUI_BASE_URL } from '$lib/constants';
 	import Tooltip from '../common/Tooltip.svelte';
 	import ChatMenu from './Sidebar/ChatMenu.svelte';
+	import ShareChatModal from '../chat/ShareChatModal.svelte';
+	import ArchiveBox from '../icons/ArchiveBox.svelte';
+	import ArchivedChatsModal from './Sidebar/ArchivedChatsModal.svelte';
 
 	let show = false;
 	let navElement;
@@ -31,12 +35,16 @@
 	let title: string = 'UI';
 	let search = '';
 
+	let shareChatId = null;
+
 	let selectedChatId = null;
 
 	let chatDeleteId = null;
 	let chatTitleEditId = null;
 	let chatTitle = '';
 
+	let showArchivedChatsModal = false;
+	let showShareChatModal = false;
 	let showDropdown = false;
 	let isEditing = false;
 
@@ -134,8 +142,21 @@
 		localStorage.setItem('settings', JSON.stringify($settings));
 		location.href = '/';
 	};
+
+	const archiveChatHandler = async (id) => {
+		await archiveChatById(localStorage.token, id);
+		await chats.set(await getChatList(localStorage.token));
+	};
 </script>
 
+<ShareChatModal bind:show={showShareChatModal} chatId={shareChatId} />
+<ArchivedChatsModal
+	bind:show={showArchivedChatsModal}
+	on:change={async () => {
+		await chats.set(await getChatList(localStorage.token));
+	}}
+/>
+
 <div
 	bind:this={navElement}
 	class="h-screen max-h-[100dvh] min-h-screen {show
@@ -544,9 +565,13 @@
 									</button>
 								</div>
 							{:else}
-								<div class="flex self-center space-x-1.5 z-10">
+								<div class="flex self-center space-x-1 z-10">
 									<ChatMenu
 										chatId={chat.id}
+										shareHandler={() => {
+											shareChatId = selectedChatId;
+											showShareChatModal = true;
+										}}
 										renameHandler={() => {
 											chatTitle = chat.title;
 											chatTitleEditId = chat.id;
@@ -577,6 +602,18 @@
 											</svg>
 										</button>
 									</ChatMenu>
+
+									<Tooltip content="Archive">
+										<button
+											aria-label="Archive"
+											class=" self-center dark:hover:text-white transition"
+											on:click={() => {
+												archiveChatHandler(chat.id);
+											}}
+										>
+											<ArchiveBox />
+										</button>
+									</Tooltip>
 								</div>
 							{/if}
 						</div>
@@ -609,13 +646,13 @@
 					{#if showDropdown}
 						<div
 							id="dropdownDots"
-							class="absolute z-40 bottom-[70px] 4.5rem rounded-xl shadow w-[240px] bg-white dark:bg-gray-900"
+							class="absolute z-40 bottom-[70px] rounded-lg shadow w-[240px] bg-white dark:bg-gray-900"
 							transition:fade|slide={{ duration: 100 }}
 						>
-							<div class="py-2 w-full">
+							<div class="p-1 py-2 w-full">
 								{#if $user.role === 'admin'}
 									<button
-										class="flex py-2.5 px-3.5 w-full hover:bg-gray-100 dark:hover:bg-gray-800 transition"
+										class="flex rounded-md py-2.5 px-3.5 w-full hover:bg-gray-100 dark:hover:bg-gray-800 transition"
 										on:click={() => {
 											goto('/admin');
 											showDropdown = false;
@@ -641,7 +678,7 @@
 									</button>
 
 									<button
-										class="flex py-2.5 px-3.5 w-full hover:bg-gray-100 dark:hover:bg-gray-800 transition"
+										class="flex rounded-md py-2.5 px-3.5 w-full hover:bg-gray-100 dark:hover:bg-gray-800 transition"
 										on:click={() => {
 											goto('/playground');
 											showDropdown = false;
@@ -668,7 +705,20 @@
 								{/if}
 
 								<button
-									class="flex py-2.5 px-3.5 w-full hover:bg-gray-100 dark:hover:bg-gray-800 transition"
+									class="flex rounded-md py-2.5 px-3.5 w-full hover:bg-gray-100 dark:hover:bg-gray-800 transition"
+									on:click={() => {
+										showArchivedChatsModal = true;
+										showDropdown = false;
+									}}
+								>
+									<div class=" self-center mr-3">
+										<ArchiveBox className="size-5" strokeWidth="1.5" />
+									</div>
+									<div class=" self-center font-medium">{$i18n.t('Archived Chats')}</div>
+								</button>
+
+								<button
+									class="flex rounded-md py-2.5 px-3.5 w-full hover:bg-gray-100 dark:hover:bg-gray-800 transition"
 									on:click={async () => {
 										await showSettings.set(true);
 										showDropdown = false;
@@ -699,11 +749,11 @@
 								</button>
 							</div>
 
-							<hr class=" dark:border-gray-700 m-0 p-0" />
+							<hr class=" dark:border-gray-800 m-0 p-0" />
 
-							<div class="py-2 w-full">
+							<div class="p-1 py-2 w-full">
 								<button
-									class="flex py-2.5 px-3.5 w-full hover:bg-gray-100 dark:hover:bg-gray-800 transition"
+									class="flex rounded-md py-2.5 px-3.5 w-full hover:bg-gray-100 dark:hover:bg-gray-800 transition"
 									on:click={() => {
 										localStorage.removeItem('token');
 										location.href = '/auth';

src/lib/components/layout/Sidebar/ArchivedChatsModal.svelte

@@ -0,0 +1,170 @@
+<script lang="ts">
+	import { toast } from 'svelte-sonner';
+	import dayjs from 'dayjs';
+	import { getContext, createEventDispatcher } from 'svelte';
+
+	const dispatch = createEventDispatcher();
+
+	import Modal from '$lib/components/common/Modal.svelte';
+	import { archiveChatById, deleteChatById, getArchivedChatList } from '$lib/apis/chats';
+	import Tooltip from '$lib/components/common/Tooltip.svelte';
+
+	const i18n = getContext('i18n');
+
+	export let show = false;
+
+	let chats = [];
+
+	const unarchiveChatHandler = async (chatId) => {
+		const res = await archiveChatById(localStorage.token, chatId).catch((error) => {
+			toast.error(error);
+		});
+
+		chats = await getArchivedChatList(localStorage.token);
+
+		dispatch('change');
+	};
+
+	const deleteChatHandler = async (chatId) => {
+		const res = await deleteChatById(localStorage.token, chatId).catch((error) => {
+			toast.error(error);
+		});
+
+		chats = await getArchivedChatList(localStorage.token);
+	};
+
+	$: if (show) {
+		(async () => {
+			chats = await getArchivedChatList(localStorage.token);
+		})();
+	}
+</script>
+
+<Modal size="lg" bind:show>
+	<div>
+		<div class=" flex justify-between dark:text-gray-300 px-5 py-4">
+			<div class=" text-lg font-medium self-center">{$i18n.t('Archived Chats')}</div>
+			<button
+				class="self-center"
+				on:click={() => {
+					show = false;
+				}}
+			>
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 20 20"
+					fill="currentColor"
+					class="w-5 h-5"
+				>
+					<path
+						d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
+					/>
+				</svg>
+			</button>
+		</div>
+		<hr class=" dark:border-gray-850" />
+
+		<div class="flex flex-col md:flex-row w-full px-5 py-4 md:space-x-4 dark:text-gray-200">
+			<div class=" flex flex-col w-full sm:flex-row sm:justify-center sm:space-x-6">
+				{#if chats.length > 0}
+					<div class="text-left text-sm w-full mb-4 max-h-[22rem] overflow-y-scroll">
+						<div class="relative overflow-x-auto">
+							<table class="w-full text-sm text-left text-gray-500 dark:text-gray-400 table-auto">
+								<thead
+									class="text-xs text-gray-700 uppercase bg-transparent dark:text-gray-200 border-b-2 border-gray-800"
+								>
+									<tr>
+										<th scope="col" class="px-3 py-2"> {$i18n.t('Name')} </th>
+										<th scope="col" class="px-3 py-2 hidden md:flex"> {$i18n.t('Created At')} </th>
+										<th scope="col" class="px-3 py-2 text-right" />
+									</tr>
+								</thead>
+								<tbody>
+									{#each chats as chat, idx}
+										<tr
+											class="bg-white {idx !== chats.length - 1 &&
+												'border-b'} dark:bg-gray-900 dark:border-gray-850 text-xs"
+										>
+											<td class="px-3 py-1 w-2/3">
+												<a href="/c/{chat.id}" target="_blank">
+													<div class=" underline line-clamp-1">
+														{chat.title}
+													</div>
+												</a>
+											</td>
+
+											<td class=" px-3 py-1 hidden md:flex h-[2.5rem]">
+												<div class="my-auto">
+													{dayjs(chat.created_at * 1000).format($i18n.t('MMMM DD, YYYY HH:mm'))}
+												</div>
+											</td>
+
+											<td class="px-3 py-1 text-right">
+												<div class="flex justify-end w-full">
+													<Tooltip content="Unarchive Chat">
+														<button
+															class="self-center w-fit text-sm px-2 py-2 hover:bg-black/5 dark:hover:bg-white/5 rounded-xl"
+															on:click={async () => {
+																unarchiveChatHandler(chat.id);
+															}}
+														>
+															<svg
+																xmlns="http://www.w3.org/2000/svg"
+																fill="none"
+																viewBox="0 0 24 24"
+																stroke-width="1.5"
+																stroke="currentColor"
+																class="size-4"
+															>
+																<path
+																	stroke-linecap="round"
+																	stroke-linejoin="round"
+																	d="M9 8.25H7.5a2.25 2.25 0 0 0-2.25 2.25v9a2.25 2.25 0 0 0 2.25 2.25h9a2.25 2.25 0 0 0 2.25-2.25v-9a2.25 2.25 0 0 0-2.25-2.25H15m0-3-3-3m0 0-3 3m3-3V15"
+																/>
+															</svg>
+														</button>
+													</Tooltip>
+
+													<Tooltip content="Delete Chat">
+														<button
+															class="self-center w-fit text-sm px-2 py-2 hover:bg-black/5 dark:hover:bg-white/5 rounded-xl"
+															on:click={async () => {
+																deleteChatHandler(chat.id);
+															}}
+														>
+															<svg
+																xmlns="http://www.w3.org/2000/svg"
+																fill="none"
+																viewBox="0 0 24 24"
+																stroke-width="1.5"
+																stroke="currentColor"
+																class="w-4 h-4"
+															>
+																<path
+																	stroke-linecap="round"
+																	stroke-linejoin="round"
+																	d="m14.74 9-.346 9m-4.788 0L9.26 9m9.968-3.21c.342.052.682.107 1.022.166m-1.022-.165L18.16 19.673a2.25 2.25 0 0 1-2.244 2.077H8.084a2.25 2.25 0 0 1-2.244-2.077L4.772 5.79m14.456 0a48.108 48.108 0 0 0-3.478-.397m-12 .562c.34-.059.68-.114 1.022-.165m0 0a48.11 48.11 0 0 1 3.478-.397m7.5 0v-.916c0-1.18-.91-2.164-2.09-2.201a51.964 51.964 0 0 0-3.32 0c-1.18.037-2.09 1.022-2.09 2.201v.916m7.5 0a48.667 48.667 0 0 0-7.5 0"
+																/>
+															</svg>
+														</button>
+													</Tooltip>
+												</div>
+											</td>
+										</tr>
+									{/each}
+								</tbody>
+							</table>
+						</div>
+						<!-- {#each chats as chat}
+							<div>
+								{JSON.stringify(chat)}
+							</div>
+						{/each} -->
+					</div>
+				{:else}
+					<div class="text-left text-sm w-full mb-8">You have no archived conversations.</div>
+				{/if}
+			</div>
+		</div>
+	</div>
+</Modal>

src/lib/components/layout/Sidebar/ChatMenu.svelte

@@ -7,7 +7,9 @@
 	import Pencil from '$lib/components/icons/Pencil.svelte';
 	import Tooltip from '$lib/components/common/Tooltip.svelte';
 	import Tags from '$lib/components/chat/Tags.svelte';
+	import Share from '$lib/components/icons/Share.svelte';
 
+	export let shareHandler: Function;
 	export let renameHandler: Function;
 	export let deleteHandler: Function;
 	export let onClose: Function;
@@ -31,12 +33,22 @@
 
 	<div slot="content">
 		<DropdownMenu.Content
-			class="w-full max-w-[150px] rounded-lg px-1 py-1.5 border border-gray-300/30 dark:border-gray-700/50 z-50 bg-white dark:bg-gray-900 dark:text-white shadow"
+			class="w-full max-w-[180px] rounded-lg px-1 py-1.5 border border-gray-300/30 dark:border-gray-700/50 z-50 bg-white dark:bg-gray-900 dark:text-white shadow"
 			sideOffset={-2}
 			side="bottom"
 			align="start"
 			transition={flyAndScale}
 		>
+			<DropdownMenu.Item
+				class="flex gap-2 items-center px-3 py-2 text-sm  font-medium cursor-pointer dark:hover:bg-gray-850 rounded-md"
+				on:click={() => {
+					shareHandler();
+				}}
+			>
+				<Share />
+				<div class="flex items-center">Share</div>
+			</DropdownMenu.Item>
+
 			<DropdownMenu.Item
 				class="flex gap-2 items-center px-3 py-2 text-sm  font-medium cursor-pointer dark:hover:bg-gray-850 rounded-md"
 				on:click={() => {

src/lib/i18n/locales/bg-BG/translation.json

@@ -1,5 +1,5 @@
 {
-	"'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.": "'с', 'м', 'ч', 'д', 'с' или '-1' за неограничен срок.",
+	"'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.": "'s', 'm', 'h', 'd', 'w' или '-1' за неограничен срок.",
 	"(Beta)": "(Бета)",
 	"(e.g. `sh webui.sh --api`)": "(например `sh webui.sh --api`)",
 	"(latest)": "(последна)",

src/lib/i18n/locales/en-US/translation.json

@@ -152,6 +152,7 @@
 	"File Mode": "",
 	"File not found.": "",
 	"Fingerprint spoofing detected: Unable to use initials as avatar. Defaulting to default profile image.": "",
+	"Fluidly stream large external response chunks": "",
 	"Focus chat input": "",
 	"Format your variables using square brackets like this:": "",
 	"From (Base Model)": "",

src/lib/i18n/locales/ja-JP/translation.json

@@ -1,5 +1,5 @@
 {
-	"'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.": "'秒', '分', '時間', '日', '週' または '-1' で無期限。",
+	"'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.": "'s', 'm', 'h', 'd', 'w' または '-1' で無期限。",
 	"(Beta)": "(ベータ版)",
 	"(e.g. sh webui.sh --api)": "(例: sh webui.sh --api)",
 	"(latest)": "(最新)",

src/lib/i18n/locales/ka-GE/translation.json

@@ -0,0 +1,372 @@
+{
+	"'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.": "'s', 'm', 'h', 'd', 'w' ან '-1' ვადის გასვლისთვის.",
+	"(Beta)": "(ბეტა)",
+	"(e.g. `sh webui.sh --api`)": "(მაგ. `sh webui.sh --api`)",
+	"(latest)": "(უახლესი)",
+	"{{modelName}} is thinking...": "{{modelName}} ფიქრობს...",
+	"{{webUIName}} Backend Required": "{{webUIName}} საჭიროა ბექენდი",
+	"a user": "მომხმარებელი",
+	"About": "შესახებ",
+	"Account": "ანგარიში",
+	"Action": "ქმედება",
+	"Add a model": "მოდელის დამატება",
+	"Add a model tag name": "მოდელის ტეგის სახელის დამატება",
+	"Add a short description about what this modelfile does": "დაამატე მოკლე აღწერა იმის შესახებ, თუ რას აკეთებს ეს მოდელური ფაილი",
+	"Add a short title for this prompt": "დაამატე მოკლე სათაური ამ მოთხოვნისთვის",
+	"Add a tag": "დაამატე ტეგი",
+	"Add Docs": "დოკუმენტის დამატება",
+	"Add Files": "ფაილების დამატება",
+	"Add message": "შეტყობინების დამატება",
+	"add tags": "ტეგების დამატება",
+	"Adjusting these settings will apply changes universally to all users.": "ამ პარამეტრების რეგულირება ცვლილებებს უნივერსალურად გამოიყენებს ყველა მომხმარებლისთვის",
+	"admin": "ადმინისტრატორი",
+	"Admin Panel": "ადმინ პანელი",
+	"Admin Settings": "ადმინისტრატორის ხელსაწყოები",
+	"Advanced Parameters": "დამატებითი პარამეტრები",
+	"all": "ყველა",
+	"All Users": "ყველა მომხმარებელი",
+	"Allow": "ნების დართვა",
+	"Allow Chat Deletion": "მიმოწერის წაშლის დაშვება",
+	"alphanumeric characters and hyphens": "ალფანუმერული სიმბოლოები და დეფისები",
+	"Already have an account?": "უკვე გაქვს ანგარიში?",
+	"an assistant": "ასისტენტი",
+	"and": "და",
+	"API Base URL": "API საბაზისო URL",
+	"API Key": "API გასაღები",
+	"API RPM": "API RPM",
+	"are allowed - Activate this command by typing": "დაშვებულია - ბრძანების გასააქტიურებლად აკრიფეთ:",
+	"Are you sure?": "დარწმუნებული ხარ?",
+	"Audio": "ხმოვანი",
+	"Auto-playback response": "ავტომატური დაკვრის პასუხი",
+	"Auto-send input after 3 sec.": "შეყვანის ავტომატური გაგზავნა 3 წამის შემდეგ ",
+	"AUTOMATIC1111 Base URL": "AUTOMATIC1111 საბაზისო მისამართი",
+	"AUTOMATIC1111 Base URL is required.": "AUTOMATIC1111 საბაზისო მისამართი აუცილებელია",
+	"available!": "ხელმისაწვდომია!",
+	"Back": "უკან",
+	"Builder Mode": "მოდელის შექმნა",
+	"Cancel": "გაუქმება",
+	"Categories": "კატეგორიები",
+	"Change Password": "პაროლის შეცვლა",
+	"Chat": "მიმოწერა",
+	"Chat History": "მიმოწერის ისტორია",
+	"Chat History is off for this browser.": "მიმოწერის ისტორია ამ ბრაუზერისთვის გათიშულია",
+	"Chats": "მიმოწერები",
+	"Check Again": "თავიდან შემოწმება",
+	"Check for updates": "განახლებების ძიება",
+	"Checking for updates...": "მიმდინარეობს განახლებების ძიება...",
+	"Choose a model before saving...": "აირჩიეთ მოდელი შენახვამდე...",
+	"Chunk Overlap": "გადახურვა ფრაგმენტულია",
+	"Chunk Params": "გადახურვის პარამეტრები",
+	"Chunk Size": "გადახურვის ზომა",
+	"Click here for help.": "დახმარებისთვის, დააკლიკე აქ",
+	"Click here to check other modelfiles.": "სხვა მოდელური ფაილების სანახავად, დააკლიკე აქ",
+	"Click here to select": "ასარჩევად, დააკლიკე აქ",
+	"Click here to select documents.": "დოკუმენტების ასარჩევად, დააკლიკე აქ",
+	"click here.": "დააკლიკე აქ",
+	"Click on the user role button to change a user's role.": "დააკლიკეთ მომხმარებლის როლის ღილაკს რომ შეცვალოთ მომხმარების როლი",
+	"Close": "დახურვა",
+	"Collection": "ნაკრები",
+	"Command": "ბრძანება",
+	"Confirm Password": "პაროლის დამოწმება",
+	"Connections": "კავშირები",
+	"Content": "კონტენტი",
+	"Context Length": "კონტექსტის სიგრძე",
+	"Conversation Mode": "საუბრი რეჟიმი",
+	"Copy last code block": "ბოლო ბლოკის კოპირება",
+	"Copy last response": "ბოლო პასუხის კოპირება",
+	"Copying to clipboard was successful!": "კლავიატურაზე კოპირება წარმატებით დასრულდა",
+	"Create a concise, 3-5 word phrase as a header for the following query, strictly adhering to the 3-5 word limit and avoiding the use of the word 'title':": "შექმენით მოკლე, 3-5 სიტყვიანი ფრაზა, როგორც სათაური თქვენი შემდეგი შეკითხვისთვის, მკაცრად დაიცავით 3-5 სიტყვის ლიმიტი და მოერიდეთ გამოიყენოთ სიტყვა „სათაური“.",
+	"Create a modelfile": "მოდელური ფაილის შექმნა",
+	"Create Account": "ანგარიშის შექმნა",
+	"Created at": "შექმნილია",
+	"Created by": "ავტორი",
+	"Current Model": "მიმდინარე მოდელი",
+	"Current Password": "მიმდინარე პაროლი",
+	"Custom": "საკუთარი",
+	"Customize Ollama models for a specific purpose": "Ollama მოდელების დამუშავება სპეციფიური დანიშნულებისთვის",
+	"Dark": "მუქი",
+	"Database": "მონაცემთა ბაზა",
+	"DD/MM/YYYY HH:mm": "DD/MM/YYYY HH:mm",
+	"Default": "დეფოლტი",
+	"Default (Automatic1111)": "დეფოლტ (Automatic1111)",
+	"Default (Web API)": "დეფოლტ (Web API)",
+	"Default model updated": "დეფოლტ მოდელი განახლებულია",
+	"Default Prompt Suggestions": "",
+	"Default User Role": "მომხმარებლის დეფოლტ როლი",
+	"delete": "წაშლა",
+	"Delete a model": "მოდელის წაშლა",
+	"Delete chat": "შეტყობინების წაშლა",
+	"Delete Chats": "შეტყობინებების წაშლა",
+	"Deleted {{deleteModelTag}}": "{{deleteModelTag}} წაშლილია",
+	"Deleted {tagName}": "{tagName} წაშლილია",
+	"Description": "აღწერა",
+	"Notifications": "შეტყობინება",
+	"Disabled": "გაუქმებულია",
+	"Discover a modelfile": "აღმოაჩინეთ მოდელური ფაილი",
+	"Discover a prompt": "აღმოაჩინეთ მოთხოვნა",
+	"Discover, download, and explore custom prompts": "აღმოაჩინეთ, ჩამოტვირთეთ და შეისწავლეთ მორგებული მოთხოვნები",
+	"Discover, download, and explore model presets": "აღმოაჩინეთ, ჩამოტვირთეთ და შეისწავლეთ მოდელის წინასწარ პარამეტრები",
+	"Display the username instead of You in the Chat": "ჩატში აჩვენე მომხმარებლის სახელი თქვენს ნაცვლად",
+	"Document": "დოკუმენტი",
+	"Document Settings": "დოკუმენტის პარამეტრები",
+	"Documents": "დოკუმენტები",
+	"does not make any external connections, and your data stays securely on your locally hosted server.": "არ ამყარებს გარე კავშირებს და თქვენი მონაცემები უსაფრთხოდ რჩება თქვენს ადგილობრივ სერვერზე.",
+	"Don't Allow": "არ დაუშვა",
+	"Don't have an account?": "არ გაქვს ანგარიში?",
+	"Download as a File": "გადმოწერე როგორც ფაილი",
+	"Download Database": "გადმოწერე მონაცემთა ბაზა",
+	"Drop any files here to add to the conversation": "გადაიტანეთ ფაილები აქ, რათა დაამატოთ ისინი მიმოწერაში",
+	"e.g. '30s','10m'. Valid time units are 's', 'm', 'h'.": "მაგალითად, '30წ', '10მ'. მოქმედი დროის ერთეულები: 'წ', 'წთ', 'სთ'.",
+	"Edit Doc": "დოკუმენტის ედიტირება",
+	"Edit User": "მომხმარებლის ედიტირება",
+	"Email": "ელ-ფოსტა",
+	"Embedding model: {{embedding_model}}": "ჩაშენების მოდელი: {{embedding_model}}",
+	"Enable Chat History": "მიმოწერის ისტორიის ჩართვა",
+	"Enable New Sign Ups": "ახალი რეგისტრაციების ჩართვა",
+	"Enabled": "ჩართულია",
+	"Enter {{role}} message here": "შეიყვანე {{role}} შეტყობინება აქ",
+	"Enter API Key": "შეიყვანე API Key",
+	"Enter Chunk Overlap": "შეიყვანეთ ნაწილის გადახურვა",
+	"Enter Chunk Size": "შეიყვანე ბლოკის ზომა",
+	"Enter Image Size (e.g. 512x512)": "შეიყვანეთ სურათის ზომა (მაგ. 512x512)",
+	"Enter LiteLLM API Base URL (litellm_params.api_base)": "შეიყვანეთ LiteLLM API ბაზის მისამართი (litellm_params.api_base)",
+	"Enter LiteLLM API Key (litellm_params.api_key)": "შეიყვანეთ LiteLLM API გასაღები (litellm_params.api_key)",
+	"Enter LiteLLM API RPM (litellm_params.rpm)": "შეიყვანეთ LiteLLM API RPM (litellm_params.rpm)",
+	"Enter LiteLLM Model (litellm_params.model)": "შეიყვანეთ LiteLLM მოდელი (litellm_params.model)",
+	"Enter Max Tokens (litellm_params.max_tokens)": "შეიყვანეთ მაქსიმალური ტოკენები (litellm_params.max_tokens)",
+	"Enter model tag (e.g. {{modelTag}})": "შეიყვანეთ მოდელის ტეგი (მაგ. {{modelTag}})",
+	"Enter Number of Steps (e.g. 50)": "შეიყვანეთ ნაბიჯების რაოდენობა (მაგ. 50)",
+	"Enter stop sequence": "შეიყვანეთ ტოპ თანმიმდევრობა",
+	"Enter Top K": "შეიყვანეთ Top K",
+	"Enter URL (e.g. http://127.0.0.1:7860/)": "შეიყვანეთ მისამართი (მაგალითად http://127.0.0.1:7860/)",
+	"Enter Your Email": "შეიყვანეთ თქვენი ელ-ფოსტა",
+	"Enter Your Full Name": "შეიყვანეთ თქვენი სრული სახელი",
+	"Enter Your Password": "შეიყვანეთ თქვენი პაროლი",
+	"Experimental": "ექსპერიმენტალური",
+	"Export All Chats (All Users)": "",
+	"Export Chats": "მიმოწერის ექსპორტირება",
+	"Export Documents Mapping": "დოკუმენტების კავშირის ექსპორტი",
+	"Export Modelfiles": "მოდელური ფაილების ექსპორტი",
+	"Export Prompts": "მოთხოვნების ექსპორტი",
+	"Failed to read clipboard contents": "ბუფერში შიგთავსის წაკითხვა ვერ მოხერხდა",
+	"File Mode": "ფაილური რეჟიმი",
+	"File not found.": "ფაილი ვერ მოიძებნა",
+	"Fingerprint spoofing detected: Unable to use initials as avatar. Defaulting to default profile image.": "აღმოჩენილია თითის ანაბეჭდის გაყალბება: ინიციალების გამოყენება ავატარად შეუძლებელია. დეფოლტ პროფილის დეფოლტ სურათი.",
+	"Focus chat input": "ჩეთის შეყვანის ფოკუსი",
+	"Format your variables using square brackets like this:": "დააფორმატეთ თქვენი ცვლადები კვადრატული ფრჩხილების გამოყენებით:",
+	"From (Base Model)": "(საბაზო მოდელი) დან",
+	"Full Screen Mode": "Სრული ეკრანის რეჟიმი",
+	"General": "ზოგადი",
+	"General Settings": "ზოგადი პარამეტრები",
+	"Hello, {{name}}": "გამარჯობა, {{name}}",
+	"Hide": "დამალვა",
+	"Hide Additional Params": "დამატებითი პარამეტრების დამალვა",
+	"How can I help you today?": "როგორ შემიძლია დაგეხმარო დღეს?",
+	"Image Generation (Experimental)": "სურათების გენერაცია (ექსპერიმენტული)",
+	"Image Generation Engine": "სურათის გენერაციის ძრავა",
+	"Image Settings": "სურათის პარამეტრები",
+	"Images": "სურათები",
+	"Import Chats": "მიმოწერების იმპორტი",
+	"Import Documents Mapping": "დოკუმენტების კავშირის იმპორტი",
+	"Import Modelfiles": "მოდელური ფაილების იმპორტი",
+	"Import Prompts": "მოთხოვნების იმპორტი",
+	"Include `--api` flag when running stable-diffusion-webui": "ჩართეთ `--api` დროშა stable-diffusion-webui-ის გაშვებისას",
+	"Interface": "ინტერფეისი",
+	"join our Discord for help.": "შეუერთდით ჩვენს Discord-ს დახმარებისთვის",
+	"JSON": "JSON",
+	"JWT Expiration": "JWT-ის ვადა",
+	"JWT Token": "JWT ტოკენი",
+	"Keep Alive": "აქტიურად დატოვება",
+	"Keyboard shortcuts": "კლავიატურის მალსახმობები",
+	"Language": "ენა",
+	"Light": "მსუბუქი",
+	"Listening...": "გისმენ...",
+	"LLMs can make mistakes. Verify important information.": "შესაძლოა LLM-ებმა შეცდომები დაუშვან. გადაამოწმეთ მნიშვნელოვანი ინფორმაცია.",
+	"Made by OpenWebUI Community": "დამზადებულია OpenWebUI საზოგადოების მიერ",
+	"Make sure to enclose them with": "დარწმუნდით, რომ დაურთეთ ისინი",
+	"Manage LiteLLM Models": "LiteLLM მოდელების მართვა",
+	"Manage Models": "მოდელების მართვა",
+	"Manage Ollama Models": "Ollama მოდელების მართვა",
+	"Max Tokens": "მაქსიმალური ტოკენები",
+	"Maximum of 3 models can be downloaded simultaneously. Please try again later.": "მაქსიმუმ 3 მოდელის ჩამოტვირთვა შესაძლებელია ერთდროულად. Გთხოვთ სცადოთ მოგვიანებით.",
+	"Mirostat": "მიროსტატი",
+	"Mirostat Eta": "მიროსტატი ეტა",
+	"Mirostat Tau": "მიროსტატი ტაუ",
+	"MMMM DD, YYYY": "თვე დღე, წელი",
+	"Model '{{modelName}}' has been successfully downloaded.": "მოდელი „{{modelName}}“ წარმატებით ჩამოიტვირთა.",
+	"Model '{{modelTag}}' is already in queue for downloading.": "მოდელი „{{modelTag}}“ უკვე ჩამოტვირთვის რიგშია.",
+	"Model {{embedding_model}} update complete!": "მოდელის {{embedding_model}} განახლება დასრულდა!",
+	"Model {{embedding_model}} update failed or not required!": "მოდელის {{embedding_model}} განახლება ვერ მოხერხდა ან არ არის საჭირო!",
+	"Model {{modelId}} not found": "მოდელი {{modelId}} ვერ მოიძებნა",
+	"Model {{modelName}} already exists.": "მოდელი {{modelName}} უკვე არსებობს.",
+	"Model filesystem path detected. Model shortname is required for update, cannot continue.": "აღმოჩენილია მოდელის ფაილური სისტემის გზა. განახლებისთვის საჭიროა მოდელის მოკლე სახელი, გაგრძელება შეუძლებელია.",
+	"Model Name": "მოდელის სახელი",
+	"Model not selected": "მოდელი არ არის არჩეული",
+	"Model Tag Name": "მოდელის ტეგის სახელი",
+	"Model Whitelisting": "მოდელის თეთრ სიაში შეყვანა",
+	"Model(s) Whitelisted": "მოდელ(ებ)ი თეთრ სიაშია",
+	"Modelfile": "მოდელური ფაილი",
+	"Modelfile Advanced Settings": "მოდელური ფაილის პარამეტრები",
+	"Modelfile Content": "მოდელური ფაილის კონტენტი",
+	"Modelfiles": "მოდელური ფაილები",
+	"Models": "მოდელები",
+	"My Documents": "ჩემი დოკუმენტები",
+	"My Modelfiles": "ჩემი მოდელური ფაილები",
+	"My Prompts": "ჩემი მოთხოვნები",
+	"Name": "სახელი",
+	"Name Tag": "სახელის ტეგი",
+	"Name your modelfile": "თქვენი მოდელური ფაილის სახელი",
+	"New Chat": "ახალი მიმოწერა",
+	"New Password": "ახალი პაროლი",
+	"Not sure what to add?": "არ იცი რა დაამატო?",
+	"Not sure what to write? Switch to": "არ იცი რა დაწერო? გადართვა:",
+	"Off": "გამორთვა",
+	"Okay, Let's Go!": "კარგი, წავედით!",
+	"Ollama Base URL": "Ollama ბაზისური მისამართი",
+	"Ollama Version": "Ollama ვერსია",
+	"On": "ჩართვა",
+	"Only": "მხოლოდ",
+	"Only alphanumeric characters and hyphens are allowed in the command string.": "ბრძანების სტრიქონში დაშვებულია მხოლოდ ალფანუმერული სიმბოლოები და დეფისები.",
+	"Oops! Hold tight! Your files are still in the processing oven. We're cooking them up to perfection. Please be patient and we'll let you know once they're ready.": "უპს! გამაგრდი! თქვენი ფაილები ჯერ კიდევ დამუშავების ღუმელშია. ჩვენ მათ სრულყოფილებამდე ვამზადებთ. გთხოვთ მოითმინოთ და ჩვენ შეგატყობინებთ, როგორც კი ისინი მზად იქნებიან.",
+	"Oops! Looks like the URL is invalid. Please double-check and try again.": "უი! როგორც ჩანს, მისამართი არასწორია. გთხოვთ, გადაამოწმოთ და ისევ სცადოთ.",
+	"Oops! You're using an unsupported method (frontend only). Please serve the WebUI from the backend.": "უპს! თქვენ იყენებთ მხარდაუჭერელ მეთოდს (მხოლოდ frontend). გთხოვთ, მოემსახუროთ WebUI-ს ბექენდიდან",
+	"Open": "ღია",
+	"Open AI": "ღია AI",
+	"Open AI (Dall-E)": "Open AI (Dall-E)",
+	"Open new chat": "ახალი მიმოწერის გახსნა",
+	"OpenAI API": "OpenAI API",
+	"OpenAI API Key": "OpenAI API გასაღები",
+	"OpenAI API Key is required.": "OpenAI API გასაღები აუცილებელია",
+	"or": "ან",
+	"Parameters": "პარამეტრები",
+	"Password": "პაროლი",
+	"PDF Extract Images (OCR)": "PDF იდან ამოღებული სურათები (OCR)",
+	"pending": "ლოდინის რეჟიმშია",
+	"Permission denied when accessing microphone: {{error}}": "ნებართვა უარყოფილია მიკროფონზე წვდომისას: {{error}}",
+	"Playground": "სათამაშო მოედანი",
+	"Profile": "პროფილი",
+	"Prompt Content": "მოთხოვნის შინაარსი",
+	"Prompt suggestions": "მოთხოვნის რჩევები",
+	"Prompts": "მოთხოვნები",
+	"Pull a model from Ollama.com": "Ollama.com იდან მოდელის გადაწერა ",
+	"Pull Progress": "პროგრესის გადაწერა",
+	"Query Params": "პარამეტრების ძიება",
+	"RAG Template": "RAG შაბლონი",
+	"Raw Format": "საწყისი ფორმატი",
+	"Record voice": "ხმის ჩაწერა",
+	"Redirecting you to OpenWebUI Community": "გადამისამართდებით OpenWebUI საზოგადოებაში",
+	"Release Notes": "Გამოშვების შენიშვნები",
+	"Repeat Last N": "გაიმეორეთ ბოლო N",
+	"Repeat Penalty": "გაიმეორეთ პენალტი",
+	"Request Mode": "მოთხოვნის რეჟიმი",
+	"Reset Vector Storage": "ვექტორული მეხსიერების გადატვირთვა",
+	"Response AutoCopy to Clipboard": "პასუხის ავტომატური კოპირება ბუფერში",
+	"Role": "როლი",
+	"Rosé Pine": "ვარდისფერი ფიჭვის ხე",
+	"Rosé Pine Dawn": "ვარდისფერი ფიჭვის გარიჟრაჟი",
+	"Save": "შენახვა",
+	"Save & Create": "დამახსოვრება და შექმნა",
+	"Save & Submit": "დამახსოვრება და გაგზავნა",
+	"Save & Update": "დამახსოვრება და განახლება",
+	"Saving chat logs directly to your browser's storage is no longer supported. Please take a moment to download and delete your chat logs by clicking the button below. Don't worry, you can easily re-import your chat logs to the backend through": "ჩეთის ისტორიის შენახვა პირდაპირ თქვენი ბრაუზერის საცავში აღარ არის მხარდაჭერილი. გთხოვთ, დაუთმოთ და წაშალოთ თქვენი ჩატის ჟურნალები ქვემოთ მოცემულ ღილაკზე დაწკაპუნებით. არ ინერვიულოთ, თქვენ შეგიძლიათ მარტივად ხელახლა შემოიტანოთ თქვენი ჩეთის ისტორია ბექენდში",
+	"Scan": "სკანირება",
+	"Scan complete!": "სკანირება დასრულდა!",
+	"Scan for documents from {{path}}": "დოკუმენტების სკანირება {{ path}}-დან",
+	"Search": "ძიება",
+	"Search Documents": "დოკუმენტების ძიება",
+	"Search Prompts": "მოთხოვნების ძიება",
+	"See readme.md for instructions": "იხილეთ readme.md ინსტრუქციებისთვის",
+	"See what's new": "სიახლეების ნახვა",
+	"Seed": "სიდი",
+	"Select a mode": "რეჟიმის არჩევა",
+	"Select a model": "მოდელის არჩევა",
+	"Select an Ollama instance": "",
+	"Send a Message": "შეტყობინების გაგზავნა",
+	"Send message": "შეტყობინების გაგზავნა",
+	"Server connection verified": "სერვერთან კავშირი დადასტურებულია",
+	"Set as default": "დეფოლტად დაყენება",
+	"Set Default Model": "დეფოლტ მოდელის დაყენება",
+	"Set Image Size": "სურათის ზომის დაყენება",
+	"Set Steps": "ნაბიჯების დაყენება",
+	"Set Title Auto-Generation Model": "სათაურის ავტომატური გენერაციის მოდელის დაყენება",
+	"Set Voice": "ხმის დაყენება",
+	"Settings": "ხელსაწყოები",
+	"Settings saved successfully!": "პარამეტრები წარმატებით განახლდა!",
+	"Share to OpenWebUI Community": "გააზიარე OpenWebUI საზოგადოებაში ",
+	"short-summary": "მოკლე შინაარსი",
+	"Show": "ჩვენება",
+	"Show Additional Params": "დამატებითი პარამეტრების ჩვენება",
+	"Show shortcuts": "მალსახმობების ჩვენება",
+	"sidebar": "საიდბარი",
+	"Sign in": "ავტორიზაცია",
+	"Sign Out": "გასვლა",
+	"Sign up": "რეგისტრაცია",
+	"Speech recognition error: {{error}}": "მეტყველების ამოცნობის შეცდომა: {{error}}",
+	"Speech-to-Text Engine": "ხმოვან-ტექსტური ძრავი",
+	"SpeechRecognition API is not supported in this browser.": "მეტყველების ამოცნობის API არ არის მხარდაჭერილი ამ ბრაუზერში.",
+	"Stop Sequence": "შეჩერების თანმიმდევრობა",
+	"STT Settings": "მეტყველების ამოცნობის პარამეტრები",
+	"Submit": "გაგზავნა",
+	"Success": "წარმატება",
+	"Successfully updated.": "წარმატებით განახლდა",
+	"Sync All": "სინქრონიზაცია",
+	"System": "სისტემა",
+	"System Prompt": "სისტემური მოთხოვნა",
+	"Tags": "ტეგები",
+	"Temperature": "ტემპერატურა",
+	"Template": "შაბლონი",
+	"Text Completion": "ტექსტის დასრულება",
+	"Text-to-Speech Engine": "ტექსტურ-ხმოვანი ძრავი",
+	"Tfs Z": "Tfs Z",
+	"Theme": "თემა",
+	"This ensures that your valuable conversations are securely saved to your backend database. Thank you!": "ეს უზრუნველყოფს, რომ თქვენი ძვირფასი საუბრები უსაფრთხოდ შეინახება თქვენს backend მონაცემთა ბაზაში. Გმადლობთ!",
+	"This setting does not sync across browsers or devices.": "ეს პარამეტრი არ სინქრონიზდება ბრაუზერებსა და მოწყობილობებში",
+	"Tip: Update multiple variable slots consecutively by pressing the tab key in the chat input after each replacement.": "რჩევა: განაახლეთ რამდენიმე ცვლადი სლოტი თანმიმდევრულად, ყოველი ჩანაცვლების შემდეგ ჩატის ღილაკზე დაჭერით.",
+	"Title": "სათაური",
+	"Title Auto-Generation": "სათაურის ავტო-გენერაცია",
+	"Title Generation Prompt": "სათაურის გენერაციის მოთხოვნა ",
+	"to": "ში",
+	"To access the available model names for downloading,": "ჩამოტვირთვისთვის ხელმისაწვდომი მოდელების სახელებზე წვდომისთვის",
+	"To access the GGUF models available for downloading,": "ჩასატვირთად ხელმისაწვდომი GGUF მოდელებზე წვდომისთვის",
+	"to chat input.": "ჩატში",
+	"Toggle settings": "პარამეტრების გადართვა",
+	"Toggle sidebar": "გვერდითი ზოლის გადართვა",
+	"Top K": "ტოპ K",
+	"Top P": "ტოპ P",
+	"Trouble accessing Ollama?": "Ollama-ს ვერ უკავშირდები?",
+	"TTS Settings": "TTS პარამეტრები",
+	"Type Hugging Face Resolve (Download) URL": "სცადე გადმოწერო Hugging Face Resolve URL",
+	"Uh-oh! There was an issue connecting to {{provider}}.": "{{provider}}-თან დაკავშირების პრობლემა წარმოიშვა.",
+	"Understand that updating or changing your embedding model requires reset of the vector database and re-import of all documents. You have been warned!": "გაითვალისწინეთ, რომ თქვენი ჩაშენების მოდელის განახლება ან შეცვლა მოითხოვს ვექტორული მონაცემთა ბაზის გადატვირთვას და ყველა დოკუმენტის ხელახლა იმპორტს. ფრთხილად!",
+	"Unknown File Type '{{file_type}}', but accepting and treating as plain text": "უცნობი ფაილის ტიპი „{{file_type}}“, მაგრამ მიიღება და განიხილება როგორც მარტივი ტექსტი",
+	"Update": "განახლება",
+	"Update embedding model {{embedding_model}}": "განაახლე ჩაშენების მოდელი {{embedding_model}}",
+	"Update password": "პაროლის განახლება",
+	"Upload a GGUF model": "GGUF მოდელის ატვირთვა",
+	"Upload files": "ფაილების ატვირთვა",
+	"Upload Progress": "პროგრესის ატვირთვა",
+	"URL Mode": "URL რეჟიმი",
+	"Use '#' in the prompt input to load and select your documents.": "",
+	"Use Gravatar": "გამოიყენე Gravatar",
+	"Use Initials": "გამოიყენე ინიციალები",
+	"user": "მომხმარებელი",
+	"User Permissions": "მომხმარებლის უფლებები",
+	"Users": "მომხმარებლები",
+	"Utilize": "გამოყენება",
+	"Valid time units:": "მოქმედი დროის ერთეულები",
+	"variable": "ცვლადი",
+	"variable to have them replaced with clipboard content.": "ცვლადი, რომ შეცვალოს ისინი ბუფერში შიგთავსით.",
+	"Version": "ვერსია",
+	"Web": "ვები",
+	"WebUI Add-ons": "WebUI დანამატები",
+	"WebUI Settings": "WebUI პარამეტრები",
+	"WebUI will make requests to": "WebUI გამოგიგზავნით მოთხოვნებს",
+	"What’s New in": "რა არის ახალი",
+	"When history is turned off, new chats on this browser won't appear in your history on any of your devices.": "როდესაც ისტორია გამორთულია, ახალი ჩეთები ამ ბრაუზერში არ გამოჩნდება თქვენს ისტორიაში არცერთ მოწყობილობაზე.",
+	"Whisper (Local)": "ჩურჩული (ადგილობრივი)",
+	"Write a prompt suggestion (e.g. Who are you?)": "დაწერეთ მოკლე წინადადება (მაგ. ვინ ხარ?",
+	"Write a summary in 50 words that summarizes [topic or keyword].": "დაწერეთ რეზიუმე 50 სიტყვით, რომელიც აჯამებს [თემას ან საკვანძო სიტყვას].",
+	"You": "თქვენ",
+	"You're a helpful assistant.": "თქვენ სასარგებლო ასისტენტი ხართ.",
+	"You're now logged in.": "თქვენ შესული ხართ."
+}

src/lib/i18n/locales/ko-KR/translation.json

@@ -1,5 +1,5 @@
 {
-	"'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.": "'초', '분', '시간', '일', '주' 또는 만료 없음 '-1'",
+	"'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.": "'s', 'm', 'h', 'd', 'w' 또는 만료 없음 '-1'",
 	"(Beta)": "(Beta)",
 	"(e.g. `sh webui.sh --api`)": "(예: `sh webui.sh --api`)",
 	"(latest)": "(latest)",

src/lib/i18n/locales/languages.json

@@ -43,6 +43,10 @@
 		"code": "ja-JP",
 		"title": "Japanese"
 	},
+	{
+		"code": "ka-GE",
+		"title": "Georgian"
+	},
 	{
 		"code": "ko-KR",
 		"title": "Korean"

src/lib/i18n/locales/nl-NL/translation.json

@@ -62,7 +62,7 @@
 	"Click here to check other modelfiles.": "Klik hier om andere modelfiles te controleren.",
 	"Click here to select": "Klik hier om te selecteren",
 	"Click here to select documents.": "Klik hier om documenten te selecteren",
-	"click here.": "click here.",
+	"click here.": "klik hier.",
 	"Click on the user role button to change a user's role.": "Klik op de gebruikersrol knop om de rol van een gebruiker te wijzigen.",
 	"Close": "Sluiten",
 	"Collection": "Verzameling",

src/lib/i18n/locales/pl-PL/translation.json

@@ -86,9 +86,9 @@
 	"Customize Ollama models for a specific purpose": "Dostosuj modele Ollama do określonego celu",
 	"Dark": "Ciemny",
 	"Database": "Baza danych",
-	"DD/MM/YYYY HH:mm": "DD/MM/RRRR GG:MM",
+	"DD/MM/YYYY HH:mm": "DD/MM/YYYY HH:mm",
 	"Default": "Domyślny",
-	"Default (Automatic1111)": "Domyślny (Automatyczny1111)",
+	"Default (Automatic1111)": "Domyślny (Automatic1111)",
 	"Default (Web API)": "Domyślny (Interfejs API)",
 	"Default model updated": "Domyślny model zaktualizowany",
 	"Default Prompt Suggestions": "Domyślne sugestie promptów",
@@ -158,7 +158,7 @@
 	"Full Screen Mode": "Tryb pełnoekranowy",
 	"General": "Ogólne",
 	"General Settings": "Ogólne ustawienia",
-	"Hello, {{name}}": "Witaj, {{nazwa}}",
+	"Hello, {{name}}": "Witaj, {{name}}",
 	"Hide": "Ukryj",
 	"Hide Additional Params": "Ukryj dodatkowe parametry",
 	"How can I help you today?": "Jak mogę Ci dzisiaj pomóc?",
@@ -193,8 +193,8 @@
 	"Mirostat Eta": "Mirostat Eta",
 	"Mirostat Tau": "Mirostat Tau",
 	"MMMM DD, YYYY": "MMMM DD, YYYY",
-	"Model '{{modelName}}' has been successfully downloaded.": "Model '{{nazwaModelu}}' został pomyślnie pobrany.",
+	"Model '{{modelName}}' has been successfully downloaded.": "Model '{{modelName}}' został pomyślnie pobrany.",
-	"Model '{{modelTag}}' is already in queue for downloading.": "Model '{{nazwaModelu}}' jest już w kolejce do pobrania.",
+	"Model '{{modelTag}}' is already in queue for downloading.": "Model '{{modelTag}}' jest już w kolejce do pobrania.",
 	"Model {{embedding_model}} update complete!": "Aktualizacja modelu {{embedding_model}} zakończona pomyślnie!",
 	"Model {{embedding_model}} update failed or not required!": "Model {{embedding_model}} aktualizacja nie powiodła się lub nie jest wymagana!",
 	"Model {{modelId}} not found": "Model {{modelId}} nie został znaleziony",

src/lib/i18n/locales/pt-BR/translation.json

@@ -1,5 +1,5 @@
 {
-	"'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.": "'s', 'm', 'h', 'd', 's' ou '-1' para não expirar.",
+	"'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.": "'s', 'm', 'h', 'd', 'w' ou '-1' para não expirar.",
 	"(Beta)": "(Beta)",
 	"(e.g. `sh webui.sh --api`)": "(por exemplo, `sh webui.sh --api`)",
 	"(latest)": "(mais recente)",
@@ -190,7 +190,7 @@
 	"Mirostat": "Mirostat",
 	"Mirostat Eta": "Mirostat Eta",
 	"Mirostat Tau": "Mirostat Tau",
-	"MMMM DD, YYYY": "MMMM DD, AAAA",
+	"MMMM DD, YYYY": "DD/MM/YYYY",
 	"Model '{{modelName}}' has been successfully downloaded.": "O modelo '{{modelName}}' foi baixado com sucesso.",
 	"Model '{{modelTag}}' is already in queue for downloading.": "O modelo '{{modelTag}}' já está na fila para download.",
 	"Model {{modelId}} not found": "Modelo {{modelId}} não encontrado",

src/lib/i18n/locales/pt-PT/translation.json

@@ -1,5 +1,5 @@
 {
-	"'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.": "'s', 'm', 'h', 'd', 's' ou '-1' para nenhuma expiração.",
+	"'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.": "'s', 'm', 'h', 'd', 'w' ou '-1' para nenhuma expiração.",
 	"(Beta)": "(Beta)",
 	"(e.g. `sh webui.sh --api`)": "(por exemplo, `sh webui.sh --api`)",
 	"(latest)": "(mais recente)",
@@ -86,7 +86,7 @@
 	"Customize Ollama models for a specific purpose": "Personalize os modelos Ollama para um propósito específico",
 	"Dark": "Escuro",
 	"Database": "Banco de dados",
-	"DD/MM/YYYY HH:mm": "DD/MM/AAAA HH:mm",
+	"DD/MM/YYYY HH:mm": "DD/MM/YYYY HH:mm",
 	"Default": "Padrão",
 	"Default (Automatic1111)": "Padrão (Automatic1111)",
 	"Default (Web API)": "Padrão (API Web)",
@@ -190,7 +190,7 @@
 	"Mirostat": "Mirostat",
 	"Mirostat Eta": "Mirostat Eta",
 	"Mirostat Tau": "Mirostat Tau",
-	"MMMM DD, YYYY": "MMMM DD, AAAA",
+	"MMMM DD, YYYY": "DD/MM/YYYY",
 	"Model '{{modelName}}' has been successfully downloaded.": "O modelo '{{modelName}}' foi baixado com sucesso.",
 	"Model '{{modelTag}}' is already in queue for downloading.": "O modelo '{{modelTag}}' já está na fila para download.",
 	"Model {{modelId}} not found": "Modelo {{modelId}} não encontrado",

src/lib/i18n/locales/ru-RU/translation.json

@@ -2,39 +2,39 @@
 	"'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.": "'s', 'm', 'h', 'd', 'w' или '-1' для не истечение.",
 	"(Beta)": "(бета)",
 	"(e.g. `sh webui.sh --api`)": "(например: `sh webui.sh --api`)",
-	"(latest)": "(новый)",
+	"(latest)": "(последний)",
-	"{{modelName}} is thinking...": "{{modelName}} это думает...",
+	"{{modelName}} is thinking...": "{{modelName}} думает...",
 	"{{webUIName}} Backend Required": "{{webUIName}} бэкенд требуемый",
-	"a user": "юзер",
+	"a user": "пользователь",
-	"About": "Относительно",
+	"About": "Об",
 	"Account": "Аккаунт",
 	"Action": "Действие",
 	"Add a model": "Добавьте модель",
-	"Add a model tag name": "Добавьте тэг модели имя",
+	"Add a model tag name": "Добавьте имя тэга модели",
-	"Add a short description about what this modelfile does": "Добавьте краткое описание, что делает этот моделифайл",
+	"Add a short description about what this modelfile does": "Добавьте краткое описание, что делает этот моделфайл",
-	"Add a short title for this prompt": "Добавьте краткое название для этого взаимодействия",
+	"Add a short title for this prompt": "Добавьте краткий заголовок для этого ввода",
 	"Add a tag": "Добавьте тэг",
 	"Add Docs": "Добавьте документы",
 	"Add Files": "Добавьте файлы",
-	"Add message": "Добавьте message",
+	"Add message": "Добавьте сообщение",
 	"add tags": "Добавьте тэгы",
-	"Adjusting these settings will apply changes universally to all users.": "Регулирующий этих настроек приведет к изменениям для все юзеры.",
+	"Adjusting these settings will apply changes universally to all users.": "Регулирующий этих настроек приведет к изменениям для все пользователей.",
 	"admin": "админ",
 	"Admin Panel": "Панель админ",
 	"Admin Settings": "Настройки админ",
 	"Advanced Parameters": "Расширенные Параметры",
 	"all": "всё",
-	"All Users": "Всё юзеры",
+	"All Users": "Все пользователи",
-	"Allow": "Дозволять",
+	"Allow": "Разрешить",
 	"Allow Chat Deletion": "Дозволять удаление чат",
 	"alphanumeric characters and hyphens": "буквенно цифровые символы и дефисы",
-	"Already have an account?": "у вас есть аккаунт уже?",
+	"Already have an account?": "у вас уже есть аккаунт?",
 	"an assistant": "ассистент",
 	"and": "и",
 	"API Base URL": "Базовый адрес API",
 	"API Key": "Ключ API",
 	"API RPM": "API RPM",
-	"are allowed - Activate this command by typing": "разрешено - активируйте эту команду набором",
+	"are allowed - Activate this command by typing": "разрешено - активируйте эту команду вводом",
 	"Are you sure?": "Вы уверены?",
 	"Audio": "Аудио",
 	"Auto-playback response": "Автоматическое воспроизведение ответа",
@@ -88,7 +88,7 @@
 	"Database": "База данных",
 	"DD/MM/YYYY HH:mm": "DD/MM/YYYY HH:mm",
 	"Default": "По умолчанию",
-	"Default (Automatic1111)": "По умолчанию (Автоматический1111)",
+	"Default (Automatic1111)": "По умолчанию (Automatic1111)",
 	"Default (Web API)": "По умолчанию (Web API)",
 	"Default model updated": "Модель по умолчанию обновлена",
 	"Default Prompt Suggestions": "Предложения промтов по умолчанию",

src/lib/i18n/locales/vi-VN/translation.json

@@ -359,5 +359,20 @@
 	"Write a summary in 50 words that summarizes [topic or keyword].": "Viết một tóm tắt trong vòng 50 từ cho [chủ đề hoặc từ khóa].",
 	"You": "Bạn",
 	"You're a helpful assistant.": "Bạn là một trợ lý hữu ích.",
-	"You're now logged in.": "Bạn đã đăng nhập."
+	"You're now logged in.": "Bạn đã đăng nhập.",
+	"Accurate information": "Thông tin chính xác",
+	"Followed instructions perfectly": "Tuân theo chỉ dẫn một cách hoàn hảo",
+	"Showcased creativity": "Thể hiện sự sáng tạo",
+	"Positive attitude": "Thể hiện thái độ tích cực",
+	"Attention to detail": "Có sự chú ý đến chi tiết của vấn đề",
+	"Thorough explanation": "Giải thích kỹ lưỡng",
+	"Don't like the style": "Không thích phong cách trả lời",
+	"Not factually correct": "Không chính xác so với thực tế",
+	"Didn't fully follow instructions": "Không tuân theo chỉ dẫn một cách đầy đủ",
+	"Refused when it shouldn't have": "Từ chối trả lời mà nhẽ không nên làm vậy",
+	"Being lazy": "Lười biếng",
+	"Other": "Khác",
+	"Thanks for your feedback!": "Cám ơn bạn đã gửi phản hồi!",
+	"Tell us more:": "Hãy cho chúng tôi hiểu thêm về chất lượng của câu trả lời:",
+	"Feel free to add specific details": "Mô tả chi tiết về chất lượng của câu hỏi và phương án trả lời"
 }

src/lib/stores/index.ts

@@ -1,10 +1,10 @@
 import { APP_NAME } from '$lib/constants';
-import { writable } from 'svelte/store';
+import { type Writable, writable } from 'svelte/store';
 
 // Backend
 export const WEBUI_NAME = writable(APP_NAME);
-export const config = writable(undefined);
+export const config: Writable<Config | undefined> = writable(undefined);
-export const user = writable(undefined);
+export const user: Writable<SessionUser | undefined> = writable(undefined);
 
 // Frontend
 export const MODEL_DOWNLOAD_POOL = writable({});
@@ -14,10 +14,10 @@ export const chatId = writable('');
 
 export const chats = writable([]);
 export const tags = writable([]);
-export const models = writable([]);
+export const models: Writable<Model[]> = writable([]);
 
 export const modelfiles = writable([]);
-export const prompts = writable([]);
+export const prompts: Writable<Prompt[]> = writable([]);
 export const documents = writable([
 	{
 		collection_name: 'collection_name',
@@ -33,6 +33,109 @@ export const documents = writable([
 	}
 ]);
 
-export const settings = writable({});
+export const settings: Writable<Settings> = writable({});
 export const showSettings = writable(false);
 export const showChangelog = writable(false);
+
+type Model = OpenAIModel | OllamaModel;
+
+type OpenAIModel = {
+	id: string;
+	name: string;
+	external: boolean;
+	source?: string;
+};
+
+type OllamaModel = {
+	id: string;
+	name: string;
+
+	// Ollama specific fields
+	details: OllamaModelDetails;
+	size: number;
+	description: string;
+	model: string;
+	modified_at: string;
+	digest: string;
+};
+
+type OllamaModelDetails = {
+	parent_model: string;
+	format: string;
+	family: string;
+	families: string[] | null;
+	parameter_size: string;
+	quantization_level: string;
+};
+
+type Settings = {
+	models?: string[];
+	conversationMode?: boolean;
+	speechAutoSend?: boolean;
+	responseAutoPlayback?: boolean;
+	audio?: AudioSettings;
+	showUsername?: boolean;
+	saveChatHistory?: boolean;
+	notificationEnabled?: boolean;
+	title?: TitleSettings;
+
+	system?: string;
+	requestFormat?: string;
+	keepAlive?: string;
+	seed?: number;
+	temperature?: string;
+	repeat_penalty?: string;
+	top_k?: string;
+	top_p?: string;
+	num_ctx?: string;
+	options?: ModelOptions;
+};
+
+type ModelOptions = {
+	stop?: boolean;
+};
+
+type AudioSettings = {
+	STTEngine?: string;
+	TTSEngine?: string;
+	speaker?: string;
+};
+
+type TitleSettings = {
+	auto?: boolean;
+	model?: string;
+	modelExternal?: string;
+	prompt?: string;
+};
+
+type Prompt = {
+	command: string;
+	user_id: string;
+	title: string;
+	content: string;
+	timestamp: number;
+};
+
+type Config = {
+	status?: boolean;
+	name?: string;
+	version?: string;
+	default_locale?: string;
+	images?: boolean;
+	default_models?: string[];
+	default_prompt_suggestions?: PromptSuggestion[];
+	trusted_header_auth?: boolean;
+};
+
+type PromptSuggestion = {
+	content: string;
+	title: [string, string];
+};
+
+type SessionUser = {
+	id: string;
+	email: string;
+	name: string;
+	role: string;
+	profile_image_url: string;
+};

src/lib/utils/index.ts

@@ -20,9 +20,7 @@ export const getModels = async (token: string) => {
 		})
 	]);
 
-	models = models
+	models = models.filter((models) => models).reduce((a, e, i, arr) => a.concat(e), []);
-		.filter((models) => models)
-		.reduce((a, e, i, arr) => a.concat(e, ...(i < arr.length - 1 ? [{ name: 'hr' }] : [])), []);
 
 	return models;
 };
@@ -37,7 +35,6 @@ export const sanitizeResponseContent = (content: string) => {
 		.replace(/<\|[a-z]+\|$/, '')
 		.replace(/<$/, '')
 		.replaceAll(/<\|[a-z]+\|>/g, ' ')
-		.replaceAll(/<br\s?\/?>/gi, '\n')
 		.replaceAll('<', '&lt;')
 		.trim();
 };
@@ -187,7 +184,8 @@ export const generateInitialsImage = (name) => {
 	return canvas.toDataURL();
 };
 
-export const copyToClipboard = (text) => {
+export const copyToClipboard = async (text) => {
+	let result = false;
 	if (!navigator.clipboard) {
 		const textArea = document.createElement('textarea');
 		textArea.value = text;
@@ -205,21 +203,27 @@ export const copyToClipboard = (text) => {
 			const successful = document.execCommand('copy');
 			const msg = successful ? 'successful' : 'unsuccessful';
 			console.log('Fallback: Copying text command was ' + msg);
+			result = true;
 		} catch (err) {
 			console.error('Fallback: Oops, unable to copy', err);
 		}
 
 		document.body.removeChild(textArea);
-		return;
+		return result;
 	}
-	navigator.clipboard.writeText(text).then(
+
-		function () {
+	result = await navigator.clipboard
+		.writeText(text)
+		.then(() => {
 			console.log('Async: Copying to clipboard was successful!');
-		},
+			return true;
-		function (err) {
+		})
-			console.error('Async: Could not copy text: ', err);
+		.catch((error) => {
-		}
+			console.error('Async: Could not copy text: ', error);
-	);
+			return false;
+		});
+
+	return result;
 };
 
 export const compareVersion = (latest, current) => {

src/routes/(app)/+layout.svelte

@@ -106,6 +106,7 @@
 				// IndexedDB Not Found
 			}
 
+			await models.set(await getModels());
 			await settings.set(JSON.parse(localStorage.getItem('settings') ?? '{}'));
 
 			await modelfiles.set(await getModelfiles(localStorage.token));

src/routes/(app)/+page.svelte

@@ -39,6 +39,7 @@
 	import { RAGTemplate } from '$lib/utils/rag';
 	import { LITELLM_API_BASE_URL, OLLAMA_API_BASE_URL, OPENAI_API_BASE_URL } from '$lib/constants';
 	import { WEBUI_BASE_URL } from '$lib/constants';
+	import { createOpenAITextStream } from '$lib/apis/streaming';
 
 	const i18n = getContext('i18n');
 
@@ -599,38 +600,22 @@
 				.pipeThrough(splitStream('\n'))
 				.getReader();
 
-			while (true) {
+			const textStream = await createOpenAITextStream(reader, $settings.splitLargeChunks);
-				const { value, done } = await reader.read();
+			console.log(textStream);
+
+			for await (const update of textStream) {
+				const { value, done } = update;
 				if (done || stopResponseFlag || _chatId !== $chatId) {
 					responseMessage.done = true;
 					messages = messages;
 					break;
 				}
 
-				try {
+				if (responseMessage.content == '' && value == '\n') {
-					let lines = value.split('\n');
+					continue;
-
+				} else {
-					for (const line of lines) {
+					responseMessage.content += value;
-						if (line !== '') {
+					messages = messages;
-							console.log(line);
-							if (line === 'data: [DONE]') {
-								responseMessage.done = true;
-								messages = messages;
-							} else {
-								let data = JSON.parse(line.replace(/^data: /, ''));
-								console.log(data);
-
-								if (responseMessage.content == '' && data.choices[0].delta.content == '\n') {
-									continue;
-								} else {
-									responseMessage.content += data.choices[0].delta.content ?? '';
-									messages = messages;
-								}
-							}
-						}
-					}
-				} catch (error) {
-					console.log(error);
 				}
 
 				if ($settings.notificationEnabled && !document.hasFocus()) {
@@ -681,16 +666,18 @@
 				}
 			} else {
 				toast.error(
-					$i18n.t(`Uh-oh! There was an issue connecting to {{provider}}.`, { provider: model })
+					$i18n.t(`Uh-oh! There was an issue connecting to {{provider}}.`, {
+						provider: model.name ?? model.id
+					})
 				);
 				responseMessage.content = $i18n.t(`Uh-oh! There was an issue connecting to {{provider}}.`, {
-					provider: model
+					provider: model.name ?? model.id
 				});
 			}
 
 			responseMessage.error = true;
 			responseMessage.content = $i18n.t(`Uh-oh! There was an issue connecting to {{provider}}.`, {
-				provider: model
+				provider: model.name ?? model.id
 			});
 			responseMessage.done = true;
 			messages = messages;

src/routes/(app)/c/[id]/+page.svelte

@@ -42,6 +42,7 @@
 		OLLAMA_API_BASE_URL,
 		WEBUI_BASE_URL
 	} from '$lib/constants';
+	import { createOpenAITextStream } from '$lib/apis/streaming';
 
 	const i18n = getContext('i18n');
 
@@ -611,38 +612,22 @@
 				.pipeThrough(splitStream('\n'))
 				.getReader();
 
-			while (true) {
+			const textStream = await createOpenAITextStream(reader, $settings.splitLargeChunks);
-				const { value, done } = await reader.read();
+			console.log(textStream);
+
+			for await (const update of textStream) {
+				const { value, done } = update;
 				if (done || stopResponseFlag || _chatId !== $chatId) {
 					responseMessage.done = true;
 					messages = messages;
 					break;
 				}
 
-				try {
+				if (responseMessage.content == '' && value == '\n') {
-					let lines = value.split('\n');
+					continue;
-
+				} else {
-					for (const line of lines) {
+					responseMessage.content += value;
-						if (line !== '') {
+					messages = messages;
-							console.log(line);
-							if (line === 'data: [DONE]') {
-								responseMessage.done = true;
-								messages = messages;
-							} else {
-								let data = JSON.parse(line.replace(/^data: /, ''));
-								console.log(data);
-
-								if (responseMessage.content == '' && data.choices[0].delta.content == '\n') {
-									continue;
-								} else {
-									responseMessage.content += data.choices[0].delta.content ?? '';
-									messages = messages;
-								}
-							}
-						}
-					}
-				} catch (error) {
-					console.log(error);
 				}
 
 				if ($settings.notificationEnabled && !document.hasFocus()) {
@@ -693,16 +678,18 @@
 				}
 			} else {
 				toast.error(
-					$i18n.t(`Uh-oh! There was an issue connecting to {{provider}}.`, { provider: model })
+					$i18n.t(`Uh-oh! There was an issue connecting to {{provider}}.`, {
+						provider: model.name ?? model.id
+					})
 				);
 				responseMessage.content = $i18n.t(`Uh-oh! There was an issue connecting to {{provider}}.`, {
-					provider: model
+					provider: model.name ?? model.id
 				});
 			}
 
 			responseMessage.error = true;
 			responseMessage.content = $i18n.t(`Uh-oh! There was an issue connecting to {{provider}}.`, {
-				provider: model
+				provider: model.name ?? model.id
 			});
 			responseMessage.done = true;
 			messages = messages;

static/logo.svg

@@ -0,0 +1,4 @@
+<svg width="500" height="500" viewBox="0 0 500 500" fill="none" xmlns="http://www.w3.org/2000/svg">
+<rect x="347.666" y="139" width="44.3349" height="221.675" fill="black"/>
+<path fill-rule="evenodd" clip-rule="evenodd" d="M202.643 360.287C263.75 360.287 313.287 310.75 313.287 249.643C313.287 188.537 263.75 139 202.643 139C141.537 139 92 188.537 92 249.643C92 310.75 141.537 360.287 202.643 360.287ZM202.645 316.029C239.309 316.029 269.031 286.307 269.031 249.643C269.031 212.979 239.309 183.257 202.645 183.257C165.981 183.257 136.259 212.979 136.259 249.643C136.259 286.307 165.981 316.029 202.645 316.029Z" fill="black"/>
+</svg>

static/manifest.json

@@ -0,0 +1 @@
+{}