{"id":5376,"date":"2025-12-08T03:00:54","date_gmt":"2025-12-08T08:00:54","guid":{"rendered":"https:\/\/ikriv.com\/blog\/?p=5376"},"modified":"2025-12-08T03:00:54","modified_gmt":"2025-12-08T08:00:54","slug":"3-months-of-vibe-coding","status":"publish","type":"post","link":"https:\/\/ikriv.com\/blog\/?p=5376","title":{"rendered":"3 Months of Vibe Coding"},"content":{"rendered":"

TL;DR<\/b> I use a lot<\/b> of AI-assisted coding now. It is a huge productivity boost, but the agents still do a lot of things wrong and need to be guided by humans.<\/p>\n

Background<\/h2>\n

Genesis Computing<\/a>, my new employer, encourages the use of AI tools, specifically the Cursor IDE. We have virtually unlimited access to the most powerful \u201cMAX\u201d models. The last several weeks have been VERY different from everything I\u2019ve done before. I often start by telling AI what features I need implemented or what bugs I need fixed in plain English, and the agent does 80% of the work. My job is to supervise it, make sure it does not go wild, review code, maintain proper design, and provide direction. However, it is still far from \u201cvibe coding\u201d in the pure sense. Agents don\u2019t see the big picture, and if given full freedom, they will ruin your project quite quickly.<\/p>\n

I\u2019ve accumulated a list of battle stories that could fill several long posts, so I\u2019ll focus on the most important things.<\/p>\n

What is an agent<\/h2>\n

An \u201cagent\u201d is a tool like Cursor or Claude Code that calls an underlying LLM model such as ChatGPT or Sonnet and uses it to work with your code base. It provides a set of tools to the model\u2014things like \u201cgrep\u201d or \u201cedit file\u201d\u2014using which it can examine and manipulate your code. The agent also breaks up big tasks into smaller chunks and makes sure the model actually finishes the job. By themselves, models don\u2019t handle long lists of TODO items very well; they tend to stop in the middle and declare victory too early.<\/p>\n

Things agents do well<\/h2>\n

Boilerplate code and prototypes<\/h3>\n

Agents are very<\/b> good at building boilerplate code from scratch. They know every technology under the sun. Want a FastAPI server that implements the OpenAI protocol? Done in 5 minutes. Want a NextJS application that sends messages over WebSockets? No problem. Simple gRPC client\/server pair? Right away!<\/p>\n

This opens the door to prototyping at astronomical speeds. You can have a skeleton application in minutes and a decent proof-of-concept in hours. But don\u2019t confuse a prototype with production code. Agents are not yet capable of building reliable production applications of any meaningful size on their own.<\/p>\n

Code analysis<\/h3>\n

Agents can reason about code and answer questions like \u201cdescribe the sequence of events when the user presses the STOP button,\u201d or \u201chow can I navigate to the Questions dialog in the UI,\u201d or \u201cdo we ever read the GARBAGE_PILE table from the database.\u201d The nice thing is that the agent does not judge: you can ask stupid questions, repeat questions, and it will patiently answer everything.<\/p>\n

Model quality matters a lot<\/b> here. Cheaper\/faster models tend to be less precise. Even expensive ones like Opus 4.1 or Sonnet 4.5 may make incorrect assumptions without fact-checking. If you have a component named DetailsView<\/code> and a tab named \u201cDetails,\u201d they may assume the tab shows the DetailsView<\/code> component without checking. If your code follows some non-trivial convention\u2014e.g., \u201ccolor\u201d is always RGB and \u201ccolour\u201d is always CMYK\u2014they may treat it as a spelling variation. My favorite example is when Sonnet and Opus told me that the frontend distinguishes two types of messages using a field I had added to the backend 30 seconds earlier, which the frontend did not even know about.<\/p>\n

Log analysis<\/h3>\n

In complex systems, logs quickly become too large for humans to analyze efficiently. Agents are better than people at searching for relevant lines, finding patterns, and connecting dots in a wall of text. Most of this is not magical\u2014they use the same techniques a person would\u2014but they do it faster and with fewer mistakes.<\/p>\n

Keep in mind, however, that agents use LLMs that emulate human reasoning. If your log message would confuse a person, it will likely confuse the model as well. If your log says \"Thread 12345 stopped\"<\/code>, the LLM will assume the thread actually stopped and will base further reasoning on that. If your team decided that \u201cthread stopped\u201d really means \u201cthread finished processing one line and will then continue,\u201d the LLM will not know that and may arrive at the wrong conclusions.<\/p>\n

Things agents do OK<\/h2>\n

Fixing bugs<\/h3>\n

The best results come when the agent can check on its own whether the bug is fixed, so the human is out of the loop entirely. It\u2019s fun to watch the agent make hypotheses, write a fix, test it, fail, try again, and so on until it finds the solution. Sometimes it takes 3\u20134 iterations or more. More expensive models do better. Don\u2019t try to fix nasty bugs with composer-1<\/code>.<\/p>\n

If human interaction is required, things slow down dramatically. The agent will apply a fix and declare victory (\u201cthe button should be right-aligned now\u201d), only for you to run the code and find that the button hasn\u2019t moved a pixel. Then it asks you to report what\u2019s in the console, paste logs, etc. The more of this you can automate, the better.<\/p>\n

Like humans, agents often jump to conclusions without verification. My favorite was: \u201cOh, you don\u2019t see file creation messages? That must be because we\u2019re not showing enough fields. Let me show the data length and timestamp in addition to the file name.\u201d Of course, this had nothing to do with the issue.<\/p>\n

Unit tests<\/h3>\n

Agents tend to build OK unit tests, but they need specific instructions. The best approach is to create an .md<\/code> file explaining how you want your tests structured (e.g., given-when-then), how to name them, not to test private functions, not to mock private functions, not to touch real databases or networks, how to properly clean up mocks, etc. Once you\u2019ve done this groundwork, the agent will write decent unit tests. Even then, it tends to give them awkward names like test_adder_handles_rational<\/code> instead of test_adder_can_add_two_rational_numbers<\/code>.<\/p>\n

Things agents do badly<\/h2>\n

Seeing the big picture<\/h3>\n

Every time you invoke an agent, it starts from a clean slate. It has no long-term memory. It sees your code through the peephole of the context window and is laser-focused on the current task. It excels at tactics and struggles with strategy. You\u2019re dealing with a very well-educated junior developer who knows everything about everything and speaks 150 languages but cannot teach itself not to copy-paste code or modify items in place after they\u2019ve been put into an asynchronous queue.<\/p>\n

Whenever there is a difficult problem, the LLM will prefer a local quick fix and will not recognize systemic failure. For example, it will introduce fallbacks, artificial delays, or repeated messages, but it will never tell you \u201cdude, you can\u2019t do this with polling reliably\u2014switch to a push-based model.\u201d<\/p>\n

Keeping the code clean<\/h3>\n

LLMs strongly prefer local modifications and dislike refactoring. They may produce an excellent first version, but after a few iterations, spaghettification is inevitable. Here’s an actual AI-produced piece of code with only slight editing:<\/p>\n

\r\nif function is not None:\r\n    try:\r\n        s_arguments = json.loads(arguments)\r\n    except json.JSONDecodeError as e:\r\n        if fixed_args:\r\n            s_arguments = json.loads(fixed_args)\r\n            arguments = fixed_args\r\n        else:\r\n            err_msg = f"Malformed JSON in tool arguments for {func_name}: {e!s}"\r\n            safe_completion_callback({"success": False, "error": err_msg})\r\n            return\r\n    if getattr(function, "is_mcp", None):\r\n        safe_completion_callback(function(s_arguments, status_update_callback))\r\n        try:\r\n            elapsed = time.time() - start_time\r\n            logger.info(...)\r\n        except Exception:\r\n            pass\r\n        try:\r\n            from mycompany.core.metrics import metrics\r\n            metrics.tool_call_finished(thread_id, func_name)\r\n        except Exception:\r\n            pass\r\n        return\r\n<\/pre>\n
Automatic linters help but give no guarantees. If your linter warns about deep nesting, long functions, swallowed exceptions, etc., the agent will generally try to minimize those\u2014but not always.<\/p>\n
Following software design principles<\/h3>\n
Agents struggle with adhering to design principles that require global understanding. No matter how much you instruct them, they will:<\/p>\n