Skip to content
Theme
OnDevice Tools

SoftwareCloud & Services

AI decision models surge as customisation gains ground

TechCrunch reports that decision models are taking a growing share of AI traffic, while Microsoft and startups compete on custom tuning and enterprise use.

Illustration of AI decision model dashboard showing preset choices and confidence scores

Key takeaways

  • TechCrunch says over a quarter of Vercel AI router requests went to decision models in October.
  • Microsoft says Microsoft-Decision-1.1 is broadly available on Azure.
  • Sqwish announced a $3 million pre-seed round for its kettei fat model.
  • Custom tuning with customer data is emerging as a way to differentiate decision models.
In this article
  1. Decision models enter mainstream use
  2. Custom data becomes the differentiator
  3. Microsoft Azure joins the rollout
  4. Benchmarks versus real work

Decision models enter mainstream use

According to TechCrunch, over a quarter of requests handled by Vercel's AI router went to decision models in October. The router sends AI requests to different models. The share is notable because decision models emerged only about a month before the report.

Decision models are a specialised class of large language model, or LLM. They do not generate output tokens, the small text units produced by normal language models. Instead, they choose from preset options and report how confident they are. TechCrunch says this makes them useful in live production systems where fast, low-cost choices matter more than long explanations.

Custom data becomes the differentiator

TypeSafe's Jev family is a leading example of the category's growth. TechCrunch says TypeSafe's value reached $7.5 billion after the release of Jev. It also says OpenRouter processed over 1.5 billion queries for TypeSafe models last week. TypeSafe CEO Diogo Almedia said the hardest part was calibrating judgement with carefully adjusted synthetic data, which is computer-generated training material.

Sqwish is trying to stand out through customisation. The UK startup announced a $3 million pre-seed round, an early funding stage, for its decision model, kettei fat. The cofounders, Ushnish Sengupta and Federica Freddi, met while studying at Cambridge and had been building small LLMs before Jev arrived. Sqwish says its tools for extra training let customers use their own production data to improve results.

The report suggests data is central to success in this category. Sqwish argues that its approach makes it more than a commodity. Sengupta says customers can continue adding data so their models improve on priority tasks.

Microsoft Azure joins the rollout

Illustration: Microsoft logo, AI, cloud, business and the labels AI Decision Models, Intel, Jev

Microsoft is also moving quickly. On Tuesday, CEO Satya Nadella said Microsoft-Decision-1.1 would become broadly available on Azure straight away. Azure is Microsoft's cloud platform. The company says the model was built by balancing cost, delay, intelligence and calibration.

Achint Srivastava, the engineering vice president who led the project, said an excessive focus on benchmarks, or standard tests, can mislead. In an email to TechCrunch, he said models tuned mainly for benchmarks can struggle in real enterprise use. Microsoft says its model and Jev do well on accuracy, reliability, speed and cost, including on tests using unseen data.

The report says decision models are easy to build, and dozens appeared after Jev. An Amazon engineer built a model in a weekend. Soon every major cloud provider had an option. This gives customers more choices, but it also means they need to test models in their own settings.

Benchmarks versus real work

Measuring decision models is not straightforward. TechCrunch says ranking them is difficult. Sqwish says kettei fat has performed well on some benchmarks, but it treats those results as a starting point. Sengupta says a benchmark invites developers to try a model, while continual tuning gives them a reason to stay.

The wider trend is a division of labour in AI systems. Decision models may handle quick choices, while larger frontier models, the largest and most capable AI systems, handle more complex reasoning. Freddi describes decision models as a form of System One intelligence, meaning fast choices. She contrasts this with System Two intelligence, meaning slower and more deliberate reasoning.

Freddi predicts future systems will combine these two types of intelligence. She says the decision-model opportunity could be $10 billion to $50 billion. She contrasts that with OpenAI's IPO plans, which she says involve a $1 trillion to $2 trillion market value. For developers and businesses, the practical lesson is to test before committing. TechCrunch says it is generally simple to swap one model for another and see how it performs in a specific context.

Sources

  1. As decision model use skyrockets, is customization is the next frontier?TechCrunch

Tags

Spotted a mistake? See our corrections policy. Read how News by OnDevice works in our editorial policy.