Oobabooga (TextGen WebUI) is both a frontend and a backend system for text generation inspired by AUTOMATIC1111's Stable Diffusion Web UI.
¶ Features
- Different interface modes: Default, Chat and Notebook.
- Supports multiple model backends such as transformers, llama.cpp, ExLLaMA, AutoGPTQ and GPTQ-for-LLaMA.
- Ability to switch between different models fast.
- LoRA support.
- Instruction templates for LLaMA 2, Alpaca, Vicuna, WizardLM, StableLM, and many others.
- Dark mode.
8-bitand4-bitprecision support.- Text streaming.
and more.
¶ Installation
- Download the one-click installer for Ooba based on your operating system or subsystem.
| Windows | Linux | macOS | WSL |
|---|---|---|---|
| Windows ZIP | Linux ZIP | macOS ZIP | WSL ZIP |
- Extract the ZIP files and run the start script from within the oobabooga folder and let the installer install for itself.
If you ever want to launch Oobabooga later, you can run the start script again and it should launch itself.
¶ Using Oobabooga as a API for Frontend Systems
To use Oobabooga as a backend for frontend systems like SillyTavern:
- Open webui.py inside of
[Oobabooga Folder]/text-generation-webuiwith a code editor or Notepad. - Find CMD_FLAGS and add
--apiafter --chat. It should look like this.CMD_FLAGS = '--chat --api'
If you want to make the API public (for remote servers), replace
--apiwith--public-api. For more flags, see this section of the Ooba Readme file here.
¶ Installing Models
Oobabooga supports both automatic downloads and manual downloads. See either section for your use case.
¶ Automatic Download
- Open a Terminal/CMD session and type in the following line:
python download-model.py organization/model
Replace organization/model with the model you wish to download. For example, if you want to download Pygmalion-6B, you will do the following.
python download-model.py PygmalionAI/pygmalion-6b
To download a protected model, setup the environment variables
HF_USERandHF_PASSto your Hugging Face username and password (or User Access Token). The model's terms must first be accepted on the HF website.
¶ Manual Download
Download a model of your choosing or from LLM Models following this guide.