Ollama

Ollama runs open models on your own machine and also offers hosted cloud models. The README gives installs for macOS, Windows and Linux, plus an official Docker image. The local server is OpenAI compatible at http://localhost:11434/v1, with chat completions, completions, embeddings, responses and audio endpoints, and the cloud endpoint is https://ollama.com/v1 with an API key. Running models on your own hardware is free and unlimited. Cloud use is billed per token on Pay as you go at $0 a month, while Pro is $20 a month with $60 in credits and Max is $100 a month with $300. The code is under the MIT license. We would suggest it to developers who want a simple local endpoint and the option to use bigger models in the cloud. The limits are in the compatibility docs: chat completions have no logprobs, tool_choice or n, and image input must be base64, not a URL. The pages we read do not list supported GPUs. We have not run it ourselves for this note.