Preview for review. Not live, not indexed, and the form doesn't send anything yet.
Palo Alto Networks is buying Applied Compute for $4.5BWhat it means for you →

Open-weight models: which to use and how to host them yourself

The best open-weight models are now close enough to the closed ones that, for many business tasks, you can run your own and keep every prompt inside your walls. The trade-off is that you become responsible for hardware, updates and quality. Here is what to run and where.

Updated 20 November 2026. Prices and features in this space change monthly; we re-check this page every month.

Short answer. For most companies, start with a mid-sized open model (around 20 to 120 billion parameters) hosted in your own cloud account through a managed service or a single GPU server. It covers drafting, summarising, extraction and classification well, runs on one or two GPUs, and keeps data in your account. Use the giant open models only if a test on your own tasks shows the mid-sized ones fall short. Read each model's licence before commercial use: "open weights" doesn't always mean "do anything".

Open weights, open source, and why it matters

An open-weight model is one whose trained weights you can download and run. That is different from open source: the training data and code are usually not released, and the licence may restrict use. Weights are what you need to run, fine-tune and own a model.

Models worth testing (November 2026)

FamilyFromSize and hardwareLicenceGood for
Qwen 3.x (e.g. 27B)AlibabaMid-size versions run on one data-centre GPUMostly Apache 2.0; check each modelThe usual first choice for self-hosting next to sensitive data
gpt-oss-120b and 20bOpenAI120b fits one 80 GB GPU; 20b runs on a strong workstationApache 2.0Reasoning and tool use with a permissive licence
DeepSeek V4.xDeepSeekLarge; multi-GPU serverMITAgents and coding; widely used by enterprises
Kimi K3Moonshot AIVery large (trillions of parameters); a full GPU server or moreCustom; read itTop open-model scores on hard tasks
Mistral Large 4Mistral (France)Large; multi-GPUCustom; commercial terms applyEuropean companies that want an EU vendor
Muse Spark OpenMetaReleased October 2026Meta's custom termsGeneral use; check the terms for large companies
Ling 3.xInclusionAI (Ant Group)Fast "Flash" versions availableCheck the model cardSpeed-sensitive workloads

Based on public open-model leaderboards (BenchLM, Artificial Analysis and others) and model cards, November 2026. Rankings change monthly; licence terms and hardware needs come from each model's official card and should be read there before you deploy. Some companies won't use models from certain countries; decide that policy before you test.

How much hardware a model needs

A rough rule: the weights take parameters × bytes per parameter of GPU memory, and you need about 20 to 30 percent more for the conversation in progress.

PrecisionBytes per parameter30B model120B model
16-bit (full quality)2~60 GB~240 GB
8-bit1~30 GB~120 GB
4-bit0.5~15 GB~60 GB

So a 30B model at 8-bit runs comfortably on one 48 GB or 80 GB GPU, and a 120B model at 4-bit fits on one 80 GB GPU. Mixture-of-experts models (most large ones now) only use part of their parameters for each word, which makes them faster, but all the weights still have to fit in memory. Quantising to 4-bit loses a little quality; test on your tasks.

Four ways to host it

OptionExamplesData stays…EffortChoose when
Your cloud's model catalogueAmazon Bedrock, Google Vertex AI, Azure AI FoundryIn your cloud accountLowYou're already on that cloud and the model you want is listed
Managed open-model hostTogether, Fireworks, Baseten, GroqWith the host, under contractLowYou want speed and low cost per token without running servers
Rented GPU serverGPU instances on AWS, Google Cloud, Azure, CoreWeave, LambdaIn your account, on your serverMediumSteady volume, a fine-tuned model, or strict data rules
Your own hardwareGPU servers in your data centre or officeFully on your premisesHighRegulation or contracts forbid any cloud

The software that serves the model is mostly free: vLLM and SGLang are the standard for servers, and Ollama or llama.cpp for laptops and small machines. All of them can expose an API that looks like OpenAI's, so your existing tools can point at your own model with a one-line change.

When self-hosting is worth it

When none of those apply, ChatGPT Business or Claude Team is cheaper and better for general staff use. Many companies run both: a hosted assistant for everyone, and a private model for the one task that needs it.

A sensible first project

  1. Pick one task with clear right answers, such as extracting fields from invoices or classifying support tickets.
  2. Collect 200 real examples with correct answers. This is your test.
  3. Score two or three open models and one closed model on it.
  4. If an open model is close, host it in your cloud's model catalogue or on one GPU server and run it beside the current process for two weeks.
  5. If none is close enough, that is the case for training your own.

Questions

What is an open-weight model?

A model whose trained weights are published so you can download, run and fine-tune it yourself. The licence may still limit how you use it, and the training data is usually not released.

What is the best open-weight model to self-host?

It changes every few months. In November 2026, mid-sized Qwen models and gpt-oss-120b are common first choices because they run on one GPU with permissive licences. Test on your own tasks before choosing.

What GPU do I need to run a 70B model?

About 70 GB of memory at 8-bit or 35 GB at 4-bit, plus room for conversations. One 80 GB GPU handles it at 8-bit; two 48 GB GPUs also work.

Is self-hosting cheaper than ChatGPT or the API?

Only at high, steady volume on one task. A GPU costs the same idle or busy, so for general staff use a ChatGPT or Claude seat is usually far cheaper.

Can we use Llama, Qwen or DeepSeek commercially?

Usually yes, but each has its own licence. DeepSeek uses MIT and most Qwen models Apache 2.0; Meta and some others add their own terms. Read the model card before deploying.