This is a list of models recommended for creative writing, which includes RP, narration, adventure, etc... These models are open-source and can be run on your local machine or in a cloud server (via vast.ai or runpod.io).
This list only contains 2 (3 but base doesn't matter for 99% of people) types of models. GGUF models and exl2 models.
Tip:
If you don't know how much VRAM is needed to run a model, check out this calculator. It allows you to figure out approximately how much VRAM is needed in order to load a model at a defined context size. Keep in mind that it's not completely accurate and generally underestimates the required VRAM.
| Parameter size | Model | Base Model | GGUF for KoboldCPP | exl2 for TabbyAPI | Recommended Templates |
|---|---|---|---|---|---|
| 7B | Una-TheBeagle-7B-v1 | Base | GGUF | exl2 | Instruct Template |
| 8B | Hathor_Respawn-L3-8B-v0.8 | Base | GGUF | exl2 | Context Template - Instruct Template |
| 11B | Fimbulvetr-11B-v2 | Base | GGUF | exl2 | Instruct Template |
| 12B | Starcannon-v1 | Base | GGUF | exl2 | Context Template - Instruct Template |
| 20B | Rose | Base | GGUF | exl2 | Instruct Template |
| 20B | psyonic-cetacean | Base | GGUF | exl2 | Instruct Template |
| 35B | c4ai-commanr-r-v01 | Base | GGUF | exl2 | Context Template - Instruct Template |
| 103B | c4ai-command-r-plus | Base | GGUF | exl2 | Context Template - Unstruct Template |
| 120B | Goliath | Base | GGUF | exl2 | Instruct Template |