Open Source vs Proprietary LLMs: What Each Choice Costs Your Margin

A proprietary API's cost is close to fully variable: you pay per token, the vendor handles scaling and uptime, and your engineering cost is limited to integration and prompt work. The tradeoff is that the per-unit rate is fixed by someone else, and at high volume that rate can outgrow what a self-hosted alternative would cost. It's also the option with the least operational risk, since you're not the one paging an on-call engineer at 2am for a model server.

Open weights remove the per-token licensing fee, but you now own GPU hosting, model serving infrastructure, monitoring, and periodically re-tuning as your use case shifts. Add up hosting cost per hour, the engineering hours to stand up and maintain serving infrastructure, and the fully loaded cost of whoever owns on-call for it. Below a certain volume, that fixed cost is more expensive per request than a proprietary API would have been.

For example, a company with one narrow classification feature and a small platform team might assume self-hosting will cut its bill. Once it counts the on-call rotation, the time spent patching and re-tuning the model, and the GPUs that sit idle overnight, the cost per request can land above what the proprietary API would have charged at that volume. A simple decision rule follows from this: self-host only when volume is high and consistent enough to keep the hardware busy, or when a compliance requirement leaves no alternative. If neither is true, stay on the proprietary API and revisit the comparison as volume grows.

Take your current or projected monthly token volume, price it against the proprietary rate, and separately estimate the fully loaded monthly cost of self-hosting at that volume, including engineering time. Where the two lines cross is your breakeven volume. Below it, proprietary wins on pure cost. Above it, self-hosting starts pulling gross margin the other direction, assuming your team can actually operate the infrastructure reliably.

Run the comparison in this order:

1. Estimate your current or projected monthly token volume and price it at the proprietary rate.
2. Estimate the fully loaded monthly cost of self-hosting at that volume, including GPU hosting, serving infrastructure, monitoring and the engineer who owns on-call.
3. Find the volume where the two cost lines cross, which is your breakeven point.
4. Add switching cost, meaning the engineering time to move providers or bring hosting in-house, as its own line item.
5. Set a calendar reminder to rerun the comparison as proprietary rates, open model quality and your own volume change.

Two situations flip the answer regardless of volume. First, data residency or contractual requirements that rule out sending data to a third party API, which makes self-hosting a compliance decision rather than a cost decision. Second, a use case narrow enough that a smaller fine-tuned open model matches a much larger proprietary model's quality at a fraction of the inference cost, which changes the whole comparison in the open source direction.

Whichever way you go, don't treat it as permanent. Model your switching cost, meaning the engineering time to move providers or bring hosting in-house, as its own line item. A margin comparison that ignores switching cost will consistently undercount the value of staying on a proprietary API a little longer, since flexibility has a real, if hard to quantify, price. Software companies generally carry meaningfully higher gross margin than services businesses, and this decision is one of the places that margin gets protected or eroded.

Revisit the comparison on a schedule rather than once. Proprietary rates change, open model quality improves, and your own volume grows, so a decision that was clearly right a year ago can quietly stop being right without anyone rerunning the numbers. Put a calendar reminder on it the same way you would a lease renewal.

Whichever path you pick shows up in a different place. Proprietary API spend is a clean, fully variable cost of goods sold line that scales directly with usage, which investors and boards generally find easy to read. Self-hosting splits your cost across infrastructure spend and the fully loaded cost of the engineers maintaining it, some of which may not even sit in cost of goods sold depending on how your team is structured. Decide up front how you'll classify self-hosted infrastructure and engineering time so your gross margin is comparable quarter to quarter regardless of which path you're on.