GitHub previews multi-model orchestration with HydraFusion

Combining models introduces a layer that decides which work goes where and how intermediate results are reconciled. That layer has its own behaviour, costs and failure modes. The relevant comparison is therefore between complete task-solving systems, rather than a list of model names.

Start by identifying why multiple models might help. Different components could handle planning, candidate generation or independent checking. Alternatively, a router might send routine requests to a smaller model and difficult ones to a more capable model. Each design makes a different tradeoff between latency, cost and complexity.

Specify the handoffs. What information is passed to the next component, what is omitted and how is disagreement handled? A polished final answer can conceal a weak intermediate assumption. Retaining a concise, inspectable record of the orchestration decisions makes it easier to understand whether the architecture adds value or simply introduces more steps between the user and the result.

Agreement between two model outputs is not the same as correctness. Models may share training patterns, accept the same misleading premise or repeat a plausible mistake. A verification stage is stronger when it can inspect evidence beyond the wording of another model's answer.

For coding tasks, executable checks provide one such source of evidence. A candidate change can be compiled, exercised against behavioural tests and inspected for effects outside its intended scope. The verifier should know which requirements it is checking and which remain untested. Passing a narrow test is not a general certificate of correctness.

Decide how the system responds when checks disagree. It may request another candidate, escalate to a person or report that it cannot establish a reliable result. Avoid a voting rule that hides uncertainty behind a majority. The goal of orchestration is to improve the chance of completing the user's task correctly, with understandable evidence and a bounded amount of additional work.

A router can make an otherwise strong collection of models perform poorly by assigning the wrong tasks or escalating too late. Evaluate routing decisions using a representative mix of workloads. Include ambiguous requests, long contexts and tasks that require access to external tools.

Record which path each task takes, the number of retries, tool usage, total latency and final quality. Compare the system with simpler alternatives under the same constraints. A larger architecture should earn its complexity through an observable improvement, not through a favourable example selected after the fact.

Pay particular attention to the tail. A small proportion of tasks can consume many retries or generate unusually long delays. Set explicit budgets and stopping rules. Give users meaningful feedback when a task is taking longer than expected. A system that knows when to stop and explain its limits can be more useful than one that continues indefinitely in pursuit of an uncertain improvement.

Each additional model connection introduces configuration, access requirements and an operational dependency. Consider what happens if one provider is unavailable, changes behaviour or reaches a rate limit. The fallback path should preserve the application's promised behaviour or clearly communicate a reduced capability.

Version the orchestration policy alongside prompts and tool definitions. Preserve evaluation results for changes to routing and verification, even when the underlying models remain the same. This makes it possible to trace a regression to the component that actually changed.

Document which inputs each provider receives and apply appropriate data controls at the boundary. Avoid sending the full task history to every component by default. Finally, provide operators with a view of the complete execution rather than disconnected logs from individual calls. Multi-model systems are most convincing when their additional machinery produces a measurable benefit and remains understandable to the people responsible for running them.