By itself, an LLM is just a block of data. The software required to perform inference with that data is typically called a “Backend”.

Sometimes the interface includes a backend (e.g. LocalTavern), other times it is a strict “Frontend” that requires a separate “Backend” to perform inference (e.g. SillyTavern).

Finally, most interfaces can be connected to 3rd-party APIs which provide backend capability for you. This can be incredibly useful when you want to run models which require more hardware than you have locally available.

These backends run locally. As a result, their capabilities are directly related to the quality of the hardware you have available.

• llama.cpp
• koboldcpp
• oobabooga

These are essentially remote backends. Everything you send and receive is, at minimum, available to the provider(s).

• openrouter
• ai horde

More information to come.