You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Model router: name a model by its family, without a version - #348458
Builds on #347704. Model routers (.model-router.yaml, behind chat.copilot.modelRouter.enabled).
Change
A route's model had to be an exact model id, so a router went stale every time a new version shipped. A model can now also be a family: a model id without its version.
routes:
- model: opuswhen: Building or changing things over many steps.
- model: haikuwhen: Quick answers and small chores.
A family resolves to the newest released model of that family the account can use. It is resolved from the account's model list each time a message is routed, with no extra request and no decision-model call.
Rules
Exact ids still win. A reference that is a model's id is that model, as before.
A digit names a version.claude-opus-5 or gpt-4o asks for that model and never falls back to its family, so an unavailable id still moves on to the route's next model.
What a family is. A model belongs when its id has every word of the reference. The models with the fewest other words are the family, so a family is not its variants: gpt is not gpt-5-mini. The family is decided before looking at which of its models can be used, so a family with no usable model is unavailable; it is not replaced by a variant.
Newest. Version numbers are compared part by part. A number can follow a word and be followed by one letter, so llama3.2 is version 3.2 of llama and gpt-4o is version 4 of gpt. A snapshot date compares the same written together or apart. For dated snapshot ids on a user's own key, the date is kept apart from the version, and an undated alias beats a dated snapshot of the same version.
Models on a user's own key. The family is the last segment: anthropic/haiku, or anthropic/Work/haiku to stay within one group. What comes before the family is written in full or without the group, as for an exact id, and the same family in two groups is reported as two families.
Policy. A model turned off by policy is passed over while another of the family can be used.
Ambiguity. A name that fits more than one family, such as claude, is reported with the families to write instead.
Experimental models are never picked by a family. Otherwise a router would move to an internal or early-access model the day it appears in the account's list. They can still be used by their exact id. A family that has only experimental models picks none and says so.
How experimental models are known
The Copilot API reports the label Experimental in a model's vendor field for these models. The agent host dropped that field; it now keeps it on the model as a _meta flag (vscode.model.experimental). check_model_router lists the flag on each model, and the built-in model-router skill describes families.
Open chats
A chat on a route that names a family stays on that route when a newer model of the family ships, and runs on the newer model from its next message. For an unnamed route this needed a change: its identity was the resolved model id, so a new version made the chat look like it had no route, and it was routed from scratch. A routed turn now also saves the entry of the route's model that matched (routeReference) when that is not the model's id, and a chat's route is found by it first. When the newer model is already the model of an earlier route, the family's route is no route of its own, and the chat is on that earlier route.
How to test
Turn on chat.copilot.modelRouter.enabled and save a router with model: opus and model: haiku routes in ~/.agents/model-routers/.
Pick it in the model picker and send a message. The routing line names the newest released model of the family.
Ask the agent to run check_model_router on the file. Each route shows the model it matched.
Change a route to model: claude. The check reports that it matches more than one family and lists them.
pnpm run oxlint on the changed files and pnpm run valid-layers-check: passed.
In a local Code OSS build: a router written with families routed messages to the newest released model of each family, and a route written as an experimental-only family followed by a fallback ran on the fallback, with the check reporting why.
A route's `model` can be a family such as `opus`, `sol` or `claude-haiku`:
a model id without its version. It resolves to the newest released model
of that family the account can use, so a router keeps working when a new
version ships.
- An exact id still wins, and a reference with a digit, such as
`claude-opus-5` or `gpt-4o`, names a version and never falls back to
its family.
- The family is decided before looking at which of its models can be
used. A family with no usable model is unavailable, never replaced by
a variant of it, so the route's next model is tried.
- A name that fits more than one family is reported with the families
to write instead.
- A family never picks an experimental model, so a router does not move
to an internal or early-access model the day it appears. The Copilot
API marks those with the vendor label "Experimental", which the agent
host now keeps on the model as a flag. They are used by their exact id.
- A chat on an unnamed route that names a family stays on that route
when a newer model of the family ships.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Scoped model-family selectors are incorrectly rejected, while unstable route matching can prevent model upgrades or reroute chats despite first-turn-only switching.
Extends Agent Host model routers to accept versionless family names, reducing the need to update router files when new model versions become available.
Changes:
Resolves families to the newest eligible released model, with policy filtering and ambiguity diagnostics.
Preserves experimental-model metadata and exposes it through check_model_router.
Adds family-aware follow-up matching, tests, and authoring guidance.
…ts model entry
- A family for a model on the user's own key can be scoped to a group or
an id's own path: `anthropic/Work/haiku`. The same family in two
groups is reported as two families.
- A routed turn saves the entry of the route's `model` that matched when
it is not the model's id. A chat on a route without a name is found by
it, so the route can move to a newer model whatever that model's id
looks like, and a chat is not taken for one on another route that
names the earlier model.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Model routing can lose saved family selections during deduplication and skip available fallback models because alphanumeric versions are misclassified.
Alphanumeric versions are treated as family words: gpt-4o becomes words ['gpt', '4o'] with no version, whereas gpt-5 becomes words ['gpt']. If GPT-5 is policy-disabled but GPT-4o is available, model: gpt excludes GPT-4o at the fewest-words filter and returns the disabled GPT-5, so the route is skipped instead of using the available older model. Please recognize alphanumeric version tokens when deriving family membership and ordering, while retaining exact-only lookup for references containing digits, and cover this policy-fallback case.
… read 4o as a version
- A route without a name that lands on the model of an earlier one is
not a route of its own. A chat that was on it is now on that earlier
route, rather than routed from scratch.
- A letter after a number is part of that version, so `gpt-4o` is
version 4 of `gpt` and can be used while policy has a newer one
turned off.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On the previously missed issue (alphanumeric versions): fixed in 6eec59c. A letter after a number is part of that version, so gpt-4o is version 4 of gpt, and a reference with a digit is still exact-only. The family test covers the policy fallback.
model: ollama/llama cannot match ollama/llama3, an identifier already used in agentHostUtilityModelService.test.ts:39, because the tokenizer retains llama3 as a word. The same happens for llama3.2 and llama4, so a versionless route skips these models and falls through to its fallback. Recognize numeric versions attached to family names while preserving v-prefixed versions and suffixes such as 4o, and add coverage for this identifier shape.
Compact and hyphenated snapshot dates are compared in different units. For example, a BYOK family containing gpt-4o-20250805 and gpt-4o-2026-01-01 selects the older snapshot because [20250805] sorts above [2026, 1, 1]; even equivalent dates compare unequal. Normalize compact dates to year/month/day components before comparing them so family routing selects the newest snapshot regardless of its date format.
…t dates in one unit
- `llama3.2` is version 3.2 of `llama`, so a family finds models whose
id has the version attached.
- A date written together, such as 20250805, is its year, month and day,
so it compares with one written apart.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On the two previously missed issues: both fixed in 59430b7. A number can follow a word, so ollama/llama finds llama3.2, and a snapshot date written together is compared as its year, month and day. The family test covers both.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #347704. Model routers (
.model-router.yaml, behindchat.copilot.modelRouter.enabled).Change
A route's
modelhad to be an exact model id, so a router went stale every time a new version shipped. Amodelcan now also be a family: a model id without its version.A family resolves to the newest released model of that family the account can use. It is resolved from the account's model list each time a message is routed, with no extra request and no decision-model call.
Rules
claude-opus-5orgpt-4oasks for that model and never falls back to its family, so an unavailable id still moves on to the route's next model.gptis notgpt-5-mini. The family is decided before looking at which of its models can be used, so a family with no usable model is unavailable; it is not replaced by a variant.llama3.2is version 3.2 ofllamaandgpt-4ois version 4 ofgpt. A snapshot date compares the same written together or apart. For dated snapshot ids on a user's own key, the date is kept apart from the version, and an undated alias beats a dated snapshot of the same version.anthropic/haiku, oranthropic/Work/haikuto stay within one group. What comes before the family is written in full or without the group, as for an exact id, and the same family in two groups is reported as two families.claude, is reported with the families to write instead.How experimental models are known
The Copilot API reports the label
Experimentalin a model'svendorfield for these models. The agent host dropped that field; it now keeps it on the model as a_metaflag (vscode.model.experimental).check_model_routerlists the flag on each model, and the built-inmodel-routerskill describes families.Open chats
A chat on a route that names a family stays on that route when a newer model of the family ships, and runs on the newer model from its next message. For an unnamed route this needed a change: its identity was the resolved model id, so a new version made the chat look like it had no route, and it was routed from scratch. A routed turn now also saves the entry of the route's
modelthat matched (routeReference) when that is not the model's id, and a chat's route is found by it first. When the newer model is already the model of an earlier route, the family's route is no route of its own, and the chat is on that earlier route.How to test
chat.copilot.modelRouter.enabledand save a router withmodel: opusandmodel: haikuroutes in~/.agents/model-routers/.check_model_routeron the file. Each route shows the model itmatched.model: claude. The check reports that it matches more than one family and lists them.Validation
pnpm run typecheck-client: passed../scripts/test.sh --runGlob "vs/platform/agentHost/test/**/{modelRouter*,copilotModelRouter*,metadataCompatibility}.test.js": 162 passing../scripts/test.sh --runGlob "vs/platform/agentHost/test/**/{agentSideEffects,copilotModelRouterTools}.test.js": 316 passing../scripts/test.sh --run src/vs/platform/agentHost/test/node/copilotAgent.test.ts: 1103 passing, 1 pending.pnpm run oxlinton the changed files andpnpm run valid-layers-check: passed.