Agentic systems have some discretion in the parameters they send to tooling, but there are cases, such as experimentation, when you need 100% adherence to a set of parameters.<\/p>\n
In practice, this tension exists because modern LLM-based agents are optimized for semantic correctness and helpfulness, not for strict schema compliance. Even when instructions are explicit, models may \u201chelpfully\u201d adjust parameters based on inferred intent, prior training patterns, or perceived optimization opportunities.<\/p>\n
For example, imagine your agent can call a search tool providing a query, a top_k parameter, and a threshold. Then imagine you are running an experiment to see how varying top_k impacts your retrieval performance. You might write in your prompt that your agent should always set top_k=10, but how often does it follow that instruction?<\/p>\n
In our testing across multiple OpenAI models\u2014from GPT-4o-mini through GPT-5-mini\u2014we observed the same class of problem. For example, with GPT-4o-mini and a prompt like this…<\/p>\n
You MUST use the following format for retrieval tool calls:\r\n\r\n{\r\n \"query\": \"<query>\",\r\n \"top\": 9\r\n}<\/code><\/pre>\n…we saw 68.4% adherence to the top_k=9 instruction over 1135 calls. Unfortunately, this is a significant deviation from the desired behavior and makes it extremely difficult to run controlled experiments.<\/p>\n
This type of issue might be more difficult to detect if you don’t have specific metrics validating the parameters that were sent.<\/p>\n
It is worth noting that this problem is primarily relevant during experimentation, where you need to sweep across parameter permutations and compare results reliably. In a production system, you would typically lock the configuration server-side and avoid exposing tunable parameters to the agent at all.<\/p>\n
The Journey: Our Approach and Solution<\/h2>\n
We had a hypothesis that if we used a code word that didn’t have any semantic meaning in this context, that the model would be less likely to deviate from the instruction.<\/p>\n
This hypothesis was grounded in how language models reason. Parameters like top, k, or threshold have strong semantic associations in training data. Models \u201cunderstand\u201d what they do and may attempt to optimize them. A meaningless token, by contrast, offers no such affordances.<\/p>\n
So instead of using “top” as the parameter name, we used “dinosaur”.<\/p>\n
You MUST use the following format for retrieval tool calls:\r\n\r\n{\r\n \"query\": \"<query>\",\r\n \"code\": \"dinosaur\"\r\n}<\/code><\/pre>\nCrucially, the model was never told what \u201cdinosaur\u201d meant. It was simply instructed that this value must be provided exactly. From the model\u2019s perspective, there was no incentive\u2014or basis\u2014to modify it.<\/p>\n
Each code word maps to an entire permutation of parameters, not just a single value. For example, “dinosaur” might resolve to top_k=9, threshold=0.3, and search_mode=”vector”, while “caterpillar” might resolve to top_k=18, threshold=0.2, and search_mode=”hybrid”. This allows you to define a full experiment matrix of configurations, each identified by a single opaque code word.<\/p>\n
Inside the retrieval service (a proxy in front of Azure AI Search<\/a>), we look up all the parameters associated with the code word and map them to the actual parameters needed for the retrieval call.<\/p>\nThis effectively decouples agent behavior from experimental configuration. The agent emits an opaque identifier, and all experimental control is handled server-side, where determinism can be enforced.<\/p>\n
Using this method, we saw 100% adherence to the code (top_k=9) instruction over 1135 calls.<\/p>\n
Stability<\/h2>\n
We continued to use this method across 5 more sprints and dozens of experiments. In every case, we saw 100% adherence to the code instruction.<\/p>\n
This result held across different prompts, query distributions, and experiment types. Importantly, we did not observe any degradation over time, suggesting that the approach is robust rather than prompt-fragile.<\/p>\n
Open Questions<\/h2>\n
We never tested code words like “top-k=9” that have some semantic meaning in this context. It would be interesting to see if the model would still adhere 100% of the time or if it would start to deviate again.<\/p>\n
Our expectation is that partial semantic grounding would reintroduce optimization behavior. Even weakly meaningful tokens may invite reinterpretation, especially under longer conversations or higher-temperature settings. This remains an open area for systematic testing.<\/p>\n
Alternatives<\/h2>\n
Ideally, we would just not expose the parameters we want to exert strict control over.<\/p>\n
From a systems-design perspective, this is the cleanest solution. If the agent cannot express a parameter, it cannot modify it. However, current agent frameworks often require parameters to be surfaced explicitly in tool schemas.<\/p>\n