
{"id":5541,"date":"2026-09-06T13:37:53","date_gmt":"2026-09-06T17:37:53","guid":{"rendered":"https:\/\/ikriv.com\/blog\/?p=5541"},"modified":"2026-09-06T13:37:53","modified_gmt":"2026-09-06T17:37:53","slug":"how-to-use-harness-engineering-to-tame-agents-that-create-wasteful-designs","status":"publish","type":"post","link":"https:\/\/ikriv.com\/blog\/?p=5541","title":{"rendered":"How to use harness engineering to tame agents that create wasteful designs"},"content":{"rendered":"<p>I will start with a joke.<\/p>\n<p>There is a room with a stovetop, a kettle, and a faucet. A programmer is asked to boil some water. He opens the faucet, pours water into the kettle, puts the kettle on the stove, turns on the stove, and waits.<\/p>\n<p>Then the same programmer is asked to solve a problem with different initial conditions: the kettle with water is already on the stove, and the stove is turned on. Without hesitation, the programmer turns off the stove, pours water out of the kettle, and declares that the problem has been reduced to the one we have already solved.<\/p>\n<p>This may seem like highly irrational behavior for a human, but agents do it all the time.<\/p>\n<p>The real story unfolded as follows:<\/p>\n<h3>Version 1<\/h3>\n<p>We had an API, let&#8217;s call it <code>\/api\/auth<\/code>, that took some credentials and returned a login token. Obtaining that token involved calling a third-party service that limits the number of calls per month, so we&#8217;ll call it <code>expensive_token<\/code>.<\/p>\n<pre><code>POST \/api\/auth =&gt; {expensive_token}<\/code><\/pre>\n<h3>Version 2<\/h3>\n<p>We added another kind of token called <code>cheap_token<\/code> that we minted ourselves.<\/p>\n<pre><code>POST \/api\/auth =&gt; {expensive_token, cheap_token}<\/code><\/pre>\n<h3>Version 3<\/h3>\n<p>A situation came up when we only needed the cheap token. An agent discovered <code>\/api\/auth<\/code> and decided to reuse it. We were obtaining both an expensive token and a cheap token, only to discard the expensive token, and we did this quite a lot. So, we started receiving &#8220;over budget&#8221; warnings from the third-party service: too many expensive tokens issued.<\/p>\n<p>What we actually needed was either a different endpoint like <code>\/api\/cheap-auth<\/code> or a flag on the existing endpoint indicating which tokens we wanted to obtain.<\/p>\n<h2>Harness Engineering<\/h2>\n<p>With traditional programming, it would end at that: a warning to the humans not to be like the guy in the joke: do not obtain expensive resources just to discard them. However, in the agentic world, it&#8217;s not enough: humans rarely write code these days; agents do. The problem with the current generation of agents is that they don&#8217;t always follow the rules and, worse yet, have little or no memory between sessions, so they repeat the same mistakes. The answer to that is adversarial review: after the code has been written, run <em>another<\/em> agent (or better, two) whose only goal is to find holes in the code. The code-writing agent will try to cut corners and finish the task as soon as possible. The code-reviewing agent will try to find as many holes as possible. Give both agents the same set of rules.<\/p>\n<p>Our rule that governs discarding expensive resources sounds like this:<\/p>\n<blockquote><p><strong>A response nobody fully consumes is a design finding, not a documentation task.<\/strong> When a caller ignores part of what an API returns, ask what it costs to produce the ignored part. If it is not trivial \u2014 a database query, an HTTP call, file I\/O, long computation \u2014 fix the interface: (1) split the API, or (2) add a mask or flag that explicitly requests a partial response, and skip producing the parts not requested. An unnecessary call is wasteful even when nobody bills for it. <em>[Examples omitted.]<\/em><\/p><\/blockquote>\n<p>So, to keep agents from pouring water out of the kettle in the name of code reuse:<\/p>\n<ol>\n<li>Write the rules down in the agent instructions file.<\/li>\n<li>Run an adversarial review based on the same rules.<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"<p>I will start with a joke. There is a room with a stovetop, a kettle, and a faucet. A programmer is asked to boil some water. He opens the faucet, <a href=\"https:\/\/ikriv.com\/blog\/?p=5541\" class=\"more-link\">[&hellip;]<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"Layout":"","footnotes":""},"categories":[32],"tags":[],"class_list":["entry","author-ikriv","post-5541","post","type-post","status-publish","format-standard","category-ai"],"_links":{"self":[{"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5541","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5541"}],"version-history":[{"count":1,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5541\/revisions"}],"predecessor-version":[{"id":5542,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5541\/revisions\/5542"}],"wp:attachment":[{"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5541"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5541"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5541"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}