Differences

This shows you the differences between two versions of the page.

--- backends [2026/03/17 06:37] – created tys
+++ backends [2026/03/17 16:44] (current) – tys
@@ Line 9: / Line 9: @@
 These backends run locally. As a result, their capabilities are directly related to the quality of the hardware you have available.
-  * llama.cpp
+Additionally, each of these require a separate [[models|model]] to operate.
-  * koboldcpp
-  * oobabooga
+==== Inference Engines ====
+Each of the following perform inference without needing additional software. User friendliness is not the first priority with these utilities.
+^ Name ^ Notes ^
+|  **Oft-Recommended Engines**  ||
+| [[https://github.com/ggml-org/llama.cpp|llama.cpp]] | Reference backend. Invented GGUF format. |
+| [[https://github.com/LostRuins/koboldcpp|koboldcpp]] | Based on llama.cpp with RP focus. |
+|  **Other Engines**  ||
+| [[https://github.com/ikawrakow/ik_llama.cpp|ExLlamaV3]] | Created exl3 format, focused on GPU performance. |
+| [[https://github.com/ikawrakow/ik_llama.cpp|ik_llama]] | Improved CPU performance. |
+==== Engine Manangers ====
+These tools are designed to provide a user-friendly layer which handles backend needs and engine management simultaneously. If you're not sure what to pick, this is a good place to start.
+^ Name ^ Notes ^
+|  **Oft-Recommended Managers**  ||
+| [[https://github.com/oobabooga/text-generation-webui|text-generation-webui (Oobabooga)]] | Offers all other engines here and more. |
+| [[https://github.com/LostRuins/koboldcpp|koboldcpp]] | Good UI, koboldcpp engine only. |
+|  **Other Managers**  ||
+| [[https://localai.io/|LocalAI]] | Provides OpenAI-compatible API. |
+| [[https://ollama.com/|ollama]] | Wraps llama.cpp. |
+| [[https://github.com/theroyallab/tabbyAPI|tabbyAPI]] | Official API server for ExLlama engines. |
 ===== 3rd-Party API providers =====
-These are essentially remote backends. Everything you send and receive is, at minimum, available to the provider(s).
+These are essentially remote backends. Everything you send and receive is, at minimum, available to the provider(s). Censorship is often encountered to varying degrees.
+^ Name ^ Notes ^
+| [[https://aihorde.net/|AI Horde]] | Free, with limited performance and models. |
+| [[https://openrouter.ai/|OpenRouter]] | Large model selection. Low(er) cost. |
+| [[https://mancer.tech/|mancer]] | Low/no censorship. Free tier available. |
+| [[https://novelai.net/|NovelAI]] | Low/no censorship. |
+| [[https://pollinations.ai/|Pollinations]] | Free tier available with ads. |
-  * openrouter
+Additionally, most commercial APIs can be utilized such as ChatGPT, Claude, Perplexity, etc.
-  * ai horde
-More information to come.
+===== Additional Resources =====
+SillyTavern's page on [[https://docs.sillytavern.app/usage/api-connections/|API Connections]].