Open-weight models: which to use and how to host them yourself
The best open-weight models are now close enough to the closed ones that, for many business tasks, you can run your own and keep every prompt inside your walls. The trade-off is that you become responsible for hardware, updates and quality. Here is what to run and where.
Updated 20 November 2026. Prices and features in this space change monthly; we re-check this page every month.
Open weights, open source, and why it matters
An open-weight model is one whose trained weights you can download and run. That is different from open source: the training data and code are usually not released, and the licence may restrict use. Weights are what you need to run, fine-tune and own a model.
Models worth testing (November 2026)
| Family | From | Size and hardware | Licence | Good for |
|---|---|---|---|---|
| Qwen 3.x (e.g. 27B) | Alibaba | Mid-size versions run on one data-centre GPU | Mostly Apache 2.0; check each model | The usual first choice for self-hosting next to sensitive data |
| gpt-oss-120b and 20b | OpenAI | 120b fits one 80 GB GPU; 20b runs on a strong workstation | Apache 2.0 | Reasoning and tool use with a permissive licence |
| DeepSeek V4.x | DeepSeek | Large; multi-GPU server | MIT | Agents and coding; widely used by enterprises |
| Kimi K3 | Moonshot AI | Very large (trillions of parameters); a full GPU server or more | Custom; read it | Top open-model scores on hard tasks |
| Mistral Large 4 | Mistral (France) | Large; multi-GPU | Custom; commercial terms apply | European companies that want an EU vendor |
| Muse Spark Open | Meta | Released October 2026 | Meta's custom terms | General use; check the terms for large companies |
| Ling 3.x | InclusionAI (Ant Group) | Fast "Flash" versions available | Check the model card | Speed-sensitive workloads |
Based on public open-model leaderboards (BenchLM, Artificial Analysis and others) and model cards, November 2026. Rankings change monthly; licence terms and hardware needs come from each model's official card and should be read there before you deploy. Some companies won't use models from certain countries; decide that policy before you test.
How much hardware a model needs
A rough rule: the weights take parameters × bytes per parameter of GPU memory, and you need about 20 to 30 percent more for the conversation in progress.
| Precision | Bytes per parameter | 30B model | 120B model |
|---|---|---|---|
| 16-bit (full quality) | 2 | ~60 GB | ~240 GB |
| 8-bit | 1 | ~30 GB | ~120 GB |
| 4-bit | 0.5 | ~15 GB | ~60 GB |
So a 30B model at 8-bit runs comfortably on one 48 GB or 80 GB GPU, and a 120B model at 4-bit fits on one 80 GB GPU. Mixture-of-experts models (most large ones now) only use part of their parameters for each word, which makes them faster, but all the weights still have to fit in memory. Quantising to 4-bit loses a little quality; test on your tasks.
Four ways to host it
| Option | Examples | Data stays… | Effort | Choose when |
|---|---|---|---|---|
| Your cloud's model catalogue | Amazon Bedrock, Google Vertex AI, Azure AI Foundry | In your cloud account | Low | You're already on that cloud and the model you want is listed |
| Managed open-model host | Together, Fireworks, Baseten, Groq | With the host, under contract | Low | You want speed and low cost per token without running servers |
| Rented GPU server | GPU instances on AWS, Google Cloud, Azure, CoreWeave, Lambda | In your account, on your server | Medium | Steady volume, a fine-tuned model, or strict data rules |
| Your own hardware | GPU servers in your data centre or office | Fully on your premises | High | Regulation or contracts forbid any cloud |
The software that serves the model is mostly free: vLLM and SGLang are the standard for servers, and Ollama or llama.cpp for laptops and small machines. All of them can expose an API that looks like OpenAI's, so your existing tools can point at your own model with a one-line change.
When self-hosting is worth it
- Data can't leave. Health, legal, defence, or client contracts that forbid third-party AI.
- High, steady volume. A GPU server costs the same whether it is busy or idle. At millions of requests a month on one task, it is usually cheaper than paying per token. At low or spiky volume, it rarely is.
- You've trained your own model. A model fine-tuned on your work has to run somewhere, and the weights are yours to deploy.
- You need it to never change. Hosted models get updated under you. Your own copy behaves the same until you decide otherwise.
When none of those apply, ChatGPT Business or Claude Team is cheaper and better for general staff use. Many companies run both: a hosted assistant for everyone, and a private model for the one task that needs it.
A sensible first project
- Pick one task with clear right answers, such as extracting fields from invoices or classifying support tickets.
- Collect 200 real examples with correct answers. This is your test.
- Score two or three open models and one closed model on it.
- If an open model is close, host it in your cloud's model catalogue or on one GPU server and run it beside the current process for two weeks.
- If none is close enough, that is the case for training your own.
Questions
What is an open-weight model?
A model whose trained weights are published so you can download, run and fine-tune it yourself. The licence may still limit how you use it, and the training data is usually not released.
What is the best open-weight model to self-host?
It changes every few months. In November 2026, mid-sized Qwen models and gpt-oss-120b are common first choices because they run on one GPU with permissive licences. Test on your own tasks before choosing.
What GPU do I need to run a 70B model?
About 70 GB of memory at 8-bit or 35 GB at 4-bit, plus room for conversations. One 80 GB GPU handles it at 8-bit; two 48 GB GPUs also work.
Is self-hosting cheaper than ChatGPT or the API?
Only at high, steady volume on one task. A GPU costs the same idle or busy, so for general staff use a ChatGPT or Claude seat is usually far cheaper.
Can we use Llama, Qwen or DeepSeek commercially?
Usually yes, but each has its own licence. DeepSeek uses MIT and most Qwen models Apache 2.0; Meta and some others add their own terms. Read the model card before deploying.