Differences

This shows you the differences between two versions of the page.

--- backends [2026/03/17 16:34] – tys
+++ backends [2026/03/17 16:44] (current) – tys
@@ Line 11: / Line 11: @@
 Additionally, each of these require a separate [[models|model]] to operate.
-=== Inference Engines ===
+==== Inference Engines ====
 Each of the following perform inference without needing additional software. User friendliness is not the first priority with these utilities.
 ^ Name ^ Notes ^
+|  **Oft-Recommended Engines**  ||
 | [[https://github.com/ggml-org/llama.cpp|llama.cpp]] | Reference backend. Invented GGUF format. |
-| [[https://github.com/LostRuins/koboldcpp|koboldcpp]] | Based on llama.cpp. Good UI. focused on RP. |
+| [[https://github.com/LostRuins/koboldcpp|koboldcpp]] | Based on llama.cpp with RP focus. |
+|  **Other Engines**  ||
 | [[https://github.com/ikawrakow/ik_llama.cpp|ExLlamaV3]] | Created exl3 format, focused on GPU performance. |
 | [[https://github.com/ikawrakow/ik_llama.cpp|ik_llama]] | Improved CPU performance. |
-=== Engine Manangers ===
+==== Engine Manangers ====
 These tools are designed to provide a user-friendly layer which handles backend needs and engine management simultaneously. If you're not sure what to pick, this is a good place to start.
 ^ Name ^ Notes ^
-| [[https://github.com/oobabooga/text-generation-webui|text-generation-webui (Oobabooga)]] | Most popular backend, for good reason. |
+|  **Oft-Recommended Managers**  ||
-| [[https://github.com/LostRuins/koboldcpp|koboldcpp]] | Good UI, capable of managing itself. |
+| [[https://github.com/oobabooga/text-generation-webui|text-generation-webui (Oobabooga)]] | Offers all other engines here and more. |
+| [[https://github.com/LostRuins/koboldcpp|koboldcpp]] | Good UI, koboldcpp engine only. |
+|  **Other Managers**  ||
 | [[https://localai.io/|LocalAI]] | Provides OpenAI-compatible API. |
 | [[https://ollama.com/|ollama]] | Wraps llama.cpp. |
 | [[https://github.com/theroyallab/tabbyAPI|tabbyAPI]] | Official API server for ExLlama engines. |
-**Note**: This list does not include backends that can directly accept Character Cards - those are considered [[interfaces]].
 ===== 3rd-Party API providers =====
-These are essentially remote backends. Everything you send and receive is, at minimum, available to the provider(s). Censorship can be expected, to varying degrees.
+These are essentially remote backends. Everything you send and receive is, at minimum, available to the provider(s). Censorship is often encountered to varying degrees.
@@ Line 45: / Line 46: @@
 Additionally, most commercial APIs can be utilized such as ChatGPT, Claude, Perplexity, etc.
+===== Additional Resources =====
+SillyTavern's page on [[https://docs.sillytavern.app/usage/api-connections/|API Connections]].