Self-hosted runtime

LocalAI

LocalAI is an open-source runtime that runs AI models on your own hardware.

Website (opens in a new tab)Source (opens in a new tab)

Checked 2026-10-31

LocalAI is an open-source runtime that runs AI models on your own hardware. Beyond text it covers voice, vision, image, video and agent workloads. Its site calls the API a drop-in replacement for most upstream APIs, with OpenAI compatibility plus Anthropic, Ollama and ElevenLabs styles, so most tools need only a URL change. Every feature has a CPU path on x86_64 or ARM64, and GPUs are optional: NVIDIA CUDA, AMD ROCm, Intel SYCL, Apple Metal and Vulkan are listed. You can install it as a Docker or Podman container, a macOS DMG, a Linux binary, or on Kubernetes with a manifest or Helm chart. The Docker command shown starts the API on port 8080. It is released under the MIT license and costs nothing. We would point it at teams that want one local endpoint for several model types and that are comfortable running a service. The limits are that drop-in covers most, not all, of the upstream APIs, so test the calls you rely on, and that you own the updates and the hardware. We have not run it ourselves for this note.

Mentioned here

Questions

How much does LocalAI cost?

Free and open source (MIT); you pay only for your own hardware (checked 31 October 2026).

Is LocalAI OpenAI compatible?

Yes, it offers an OpenAI-compatible API, so most agents and SDKs can point at it by changing the base URL.

Can I self-host LocalAI?

Yes, it can run on your own servers.

Reviews

Loading reviews…

Write a review

Your rating

Up to 1000 characters. Stars alone are enough.

Similar

  • Text Generation Inference

    Text Generation Inference, or TGI, is Hugging Face's open server for running large language models on your own GPUs.

    • Self-hosted runtime
    • OpenAI compatible
    • Self-hostable

  • LM Studio

    LM Studio is a desktop app for downloading and running open-weight models on your own computer.

    • Self-hosted runtime
    • OpenAI compatible
    • Self-hostable

  • Jan

    Jan is a desktop app from Menlo Research for running open models on your own machine.

    • Self-hosted runtime
    • OpenAI compatible
    • Self-hostable

  • SGLang

    SGLang is a serving framework for large language and multimodal models, aimed at low latency and high throughput from a single GPU up to large clusters.

    • Self-hosted runtime
    • OpenAI compatible
    • Self-hostable