{"id":5393,"date":"2025-11-19T15:02:57","date_gmt":"2025-11-19T20:02:57","guid":{"rendered":"https:\/\/ikriv.com\/blog\/?p=5393"},"modified":"2025-11-19T15:02:57","modified_gmt":"2025-11-19T20:02:57","slug":"llms-against-semantic-drift","status":"publish","type":"post","link":"https:\/\/ikriv.com\/blog\/?p=5393","title":{"rendered":"LLMs against semantic drift"},"content":{"rendered":"<p>Any group of people, development teams included, creates its own mini-language where existing words change meaning and new words are invented. In our case the word was \u201cunattended.\u201d Over a couple of months its meaning shifted from \u201cexecuting the plan from start to finish without any user involvement\u201d to \u201cstopping only to ask important questions.\u201d Similar processes happen in \u201creal\u201d languages, but they take decades or even centuries. For an example, look up the <a href=\"https:\/\/en.wiktionary.org\/wiki\/deer\">Old English meaning of \u201cdeer\u201d<\/a>. In both cases, the change is gradual and often happens imperceptibly to the members of the group.<\/p>\n<p>LLMs have no knowledge of this local mini-language and take words literally. When you tell an LLM to run a plan \u201cin unattended mode,\u201d it will not stop for questions. In fact, it has been observed creating synthetic data so it can proceed, even when the user input was critical.<\/p>\n<p>One can provide the LLM with a \u201cteamglish-to-English\u201d dictionary and explain that in our context \u201cunattended\u201d means something else, but this requires at least two conditions:<\/p>\n<p>(a) The group must realize their meaning of the word has shifted, and<br \/>\n(b) The LLM must not get confused and must consistently follow the instructions exactly.<\/p>\n<p>Unfortunately, LLMs are notoriously bad at following complex instructions exactly, so even providing a dictionary may not help.<\/p>\n<p>Interaction with the LLM thus acts as a brake on semantic drift and keeps the local team dialect closer to English than it otherwise would have been.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Any group of people, development teams included, creates its own mini-language where existing words change meaning and new words are invented. In our case the word was \u201cunattended.\u201d Over a <a href=\"https:\/\/ikriv.com\/blog\/?p=5393\" class=\"more-link\">[&hellip;]<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"Layout":"","footnotes":""},"categories":[1],"tags":[],"class_list":["entry","author-ikriv","post-5393","post","type-post","status-publish","format-standard","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5393","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5393"}],"version-history":[{"count":1,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5393\/revisions"}],"predecessor-version":[{"id":5394,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5393\/revisions\/5394"}],"wp:attachment":[{"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5393"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5393"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5393"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}