When Should Hermes Escalate to a Stronger Model?
Use Hermes model configuration and completed-run metadata to compare direct routing with one observable escalation rule on a matched batch. Set a per-task cost ceiling and a minimum number of accepted tasks.
Calibrate an escalation rule on a visible failure
Hermes exposes model and tool configuration separately. Use that separation to test a routing rule while keeping the toolset fixed. For example, give both routes a synthetic order lookup that must return a matching order ID and exact integer total. Record whether the lookup returned a usable payload and whether the answer passed the field checks. Escalate on a failed field check after a successful lookup. Log unavailable fixtures and tool timeouts in their own bucket since a second model still depends on that tool. Develop the rule on one batch and freeze it before measuring on another. Run the stronger route directly on every task in the measurement batch so you can see how many tasks the trigger misses and where escalation spends money without rescuing a result.
Install Hermes Agent: Official Guide | Nous ResearchFind the escalation price your batch can support
The Hermes run response distinguishes the requested model from the provider and model that served the completed turn. Keep those labels alongside the cost records so a provider fallback and your application’s escalation remain visible as separate events. Aggregate charges for every started attempt, including unresolved cases, and use the same billing basis on both routes. Suppose 100 illustrative tasks cost $2.00 on the first pass and a direct stronger run costs $5.00. If 20 tasks escalate, the average escalation can cost $0.15 before the cascade matches the direct route’s batch spend. Apply your acceptance floor before using that price ceiling. If the cascade accepts 89 tasks and the release floor is 90, the cheaper batch still fails the release gate even when spend comes in well under budget.
API Server | Hermes AgentRecord the route you actually run
Use hermes model to select the provider and model, and hermes tools to configure the tools needed by the routine. Record the exact model identifier, provider, API route, and relevant reasoning settings for each trial. The same display name can give you an ambiguous result if you change the provider or settings during the comparison.
Verify model availability, supported tool calling, and usable context length in the provider's documentation for that exact route. Run a plain reply, a read-only tool call, and a two-step task that depends on the result of the first call. Keep the prompt, tools, and fixture fixed across candidates. Mark a route as unqualified if it cannot complete the required tool sequence with the configuration you can use in production.
Install Hermes Agent: Official Guide | Nous ResearchScore the sequence and the final answer
The Hermes API server can stream tool progress to a client. For an illustrative test, give each candidate a fixture with ticket A17 marked Ready and B31 marked Blocked. Require the agent to fetch both tickets, look up the owner of A17, and return only the ready ticket ID and owner. Put both the expected answer and the expected tool arguments on the operator's checklist.
Include a second fixture where the directory has no owner for A17. Success means the run reports the missing owner and stops without guessing. A tool progress event helps you locate the step that started. Check the returned tool data and the final answer before scoring the run. Compare candidates on correct runs, unnecessary calls, and time spent correcting failures.
API Server | Hermes AgentCalculate cost per accepted run
Hermes documents a usage object with prompt and completion token counts in an API response example. Use the provider's usage export to verify billable quantities across all model calls in each run, including retries. Divide the total spend by the number of accepted runs. Track cached input or tool service charges separately when the provider bills them separately.
For illustrative arithmetic, assume 500 runs per month at 20,000 billable input tokens and 2,000 output tokens per run. At invented rates of $0.40 per million input tokens and $2 per million output tokens, that is $4 plus $2, or $6 in model charges. Replace those inputs using the provider's price sheet for the exact model and endpoint. For a subscription, confirm that the route is eligible and check quotas and overage terms.
API Server | Hermes AgentDiagnose failures before moving up a tier
The Hermes dashboard lets you inspect sessions and see the model name for each session. When a route answers plain questions but fails the tool test, open the failed session and confirm you tested the intended model. Check the provider response for a rejected request, then check whether your setup sent the expected tool schema. Change one setting at a time and rerun the same fixture.
For a reasoning model, verify whether the provider supports tool calls with reasoning enabled on the chosen endpoint before trying a different reasoning setting or endpoint. Give a higher-cost candidate the same failing fixture and compare the results. For a local Gemma or Qwen candidate, run the longest representative task on the actual host and record memory use, elapsed time, and tool completion before assigning unattended work.
Hermes Web Dashboard | Hermes Agent