Buy @ Amazon

The Truth About AI Coding Agents

on

The Truth About AI Coding Agents

What They Can and Can’t Do for Your Engineering Teams

Image generated by Microsoft's Copilot inbuilt into Edge Browser

A not so recent "warning" email from the CEO of a leading tech company to its employees, proclaiming that AI was already handling a significant percentage of development work and would inevitably lead to widespread developer redundancy, ignited a wildfire of anxiety among developers and unfortunate rash decisions by department heads. Suddenly, CEOs of other companies were following suit, downsizing its development team and pouring investments into the promise of a magical AI coding agent. In our age of virality, it seems bad news and exaggerated claims spread with alarming speed and influence.

Let's be clear: this was, at best, a brilliant marketing play by a CEO aiming to dominate the burgeoning AI coding agent market – and it undeniably paid off handsomely. In my conversations with executive leadership, I advise them with lighted spirit to not fall prey for such ploy which is similar to their own that they play out to their customers ;) Past narratives of productivity-enhancing tools, however positive, never achieved this level of market frenzy. The short-sighted cost-cutting achieved by prematurely firing developers is a temporary illusion. Nothing truly replaces human collaboration when it comes to creatively and uniquely solving complex problems. 

Companies that strategically leverage AI tools to boost their existing team's productivity will thrive, while those chasing the pipe dream of wholesale developer replacement are heading for a painful reckoning.

The noise surrounding AI coding agents like GitHub Copilot, Amazon's CodeWhisperer (now renamed as Q Developer), and others has reached a fever pitch, and this post is my attempt to inject some much-needed clarity into the conversation for executive leadership. While these AI coding agents offer undeniably significant advantages, falling into the trap of believing they deliver unrealistic 100x productivity gains or can replace a substantial portion of your skilled development team is a dangerous misconception that can lead to fundamentally flawed strategic decisions.


What AI Coding Agents CAN Do Today?

Think of these tools as highly efficient and knowledgeable assistants embedded within your development workflow. They can:
  • Dramatically Accelerate Code Completion: Suggesting and completing lines or entire blocks of code in real-time or even generating routine utility functions from function name/comment, they cut down on repetitive typing and boilerplate, freeing up developers to focus on core logic. (In teams I led, I used to nudge folks to leverage plugins or powerful IDEs like Pycharm, IntelliJ etc for most if not all of this.)
  • Enable Faster Prototyping Through Code Generation: Translating natural language comments into functional code allows for rapid experimentation and quicker translation of ideas into tangible solutions. This is hands down a fabulous thing that I am wowed by. This is Vibe Coding to me and it shouldn't go beyond this, if you care for safety of your business and the business of your end-customer.
  • Improve Team Understanding with Code Explanation: Demystifying complex or unfamiliar code, these agents aid in faster onboarding, easier maintenance of legacy systems, and smoother team collaboration. As a polyglot developer, I have leveraged this power of AI Agent to quickly understand and appreciate the idiomatic code when I move from one language to another or even when you move from one coding style to another (like writing FP-style coding).
  • On-demand Code Review: Offering suggestions for improved readability, identifying potential errors, and even proposing basic optimizations, they help elevate code quality without consuming valuable senior developer time on routine tasks. In dynamic programming languages like Python, I continue to influence teams to using linting tools like Pylint, and Flake8 for this - checkout my recent PR on this for more insights into this.
  • Unit Tests Generation: Generating unit tests isn't something I'm a fan boy of. However, if you are like me who don't code often, generating these unit tests come handy when you sit on different code bases using different programming languages and tools. Interesting fact is that even in teams that have been in the habit of writing unit tests, I have seen different writing styles and approaches to it apart from the tools. This is thread on its own, so to say. 
  • Assist in More Efficient Debugging: By analyzing errors and suggesting potential solutions through chat interfaces, they can significantly shorten debugging cycles. From clicking through the links in search results for error to mining for its reasons in forums like Stackoverflow, Reddit, etc., one has to now only check out their AI Agent window within their own IDE. What a game changer it is to go distraction free and be more focused at work even so effortlessly?
  • Contribute to Better Documentation: Generating API docs for your publicly accessible methods is now a jiffy. This is such a mundane task that it often is done in a shoddy manner. This should be gone for good with the adoption of AI Agent like Copilot.
  • Code Review on Pull Request: Summarizing changes in pull requests allows reviewers to grasp the essence of the modifications swiftly, accelerating the review process. Personally not a fanboy of this feature even in open-source world and don't think those motivated bunch would care for this. However, most companies are stuck to this legacy Git workflow style in their engineering culture and code review is done mostly for the sake of compliance. This feature of AI Agent hopefully makes the process a little interesting for the reviewer and opportunity to shift the blame on the AI Agent when things go wrong ;)  
In quick summary, prior to this AI Agents, I have nudged my teams to leverage tools where required to improve their productivity. You can read more on this from my earlier blogs titled - Python Development Tools You Must Leverage For Productivity and Java Development Tools You Must Leverage For Productivity. Depending on the language you must leverage such tools as they continue to hold their importance for continuous and automated code quality checks. I bet 80% of the companies still don't have it in place. If you have come this far reading this, take a moment to ask yourself "Do I have it all in place in my product's SDLC?".

What AI Coding Agents CANNOT Do? - The Critical Reality Check

Despite their impressive capabilities, these AI tools are not a substitute for the strategic thinking, deep understanding, and collaborative spirit of experienced software engineers. They currently cannot:
  • Produce Flawless Code Without Human Scrutiny: AI-generated code, while often helpful, is not always optimal, secure, or even correct (in terms of completeness, logic or approach). Thorough human review, testing, and refinement are crucial to prevent errors and ensure quality.
  • Error-prone When Code Refactoring Involves Multiple Files: In two years time from the time of generic ChatGPT like Agents to  Specialist Agents embedded in IDE, Coding Assist AI Agents have come a long way to give its due credits. As an embedded agent it started from aa single file as context to open files as context to growing the context across the project is a big leap in better outcomes for refactoring that affect across files in the same project codebase. But in my experience of refactoring legacy code bases, when the refactoring spans across files as context, it is error prone in that some of the code logic often gets missed out. This gets worse with projects having large classes and methods. This is crux of ongoing development effort for developers and AI Agent has a long way to go in being resourceful to developers here.
  • Grasp the Nuances of Complex Business Logic: While they understand code syntax, they lack the inherent understanding of your specific business requirements, user needs, and strategic objectives that guide effective software development. Take for example the legacy code bases, that has methods that runs into hundreds of LoC and classes that runs into thousands of LoC, written by yesteryears experts and today's engineering leaders (You!). This code often is littered with dead code, buggy-code that bombs from time to time whose symptoms are fixed else-where making the codebase akin to minesweeper game, bad or dated naming to variables, classes, methods, dated and incorrect documentation, etc. This problem is compounded by logic that spreads across layers and modules and services. No AI Agent can save you from this gigantic mess but good engineering discipline. 
  • Independently Solve Novel and Complex Problems: These agents are powerful assistants but cannot independently solve your problems that are specific to your domain understanding your custom language. Take the example of lending in fintech - in any company you would see people coming from different other fintechs using their own set of jargons in conversations silently leading to nurturing chaos. Don't get it? Take the case of a fintech company where Personal Loans, Unsecured loans, Short-term personal loans, STPL, etc. all refer to the same thing. Often team members are left without a source of documentation to validate their understanding. In any domain, there are thousands of concepts and if each concept is referred by five or six synonymous words, it leads to miscommunication and misunderstanding. It requires human discipline and no AI Agent can solve this. Why? For any computer program, the adage is, "Garbage in, garbage out!". You AI Agent can replicate basic version of E-commerce website but not build production grade e-commerce systems in a jiffy. Great things take time and efforts with human involvement. AI Agents are for chores, or imitation at least until AGI (Artificial General Intelligence) becomes mainstream. 
  • Integrate External Knowledge and Context Autonomously: Harping on the above two points leads us to say this one. AI Agent's understanding is limited to the provided code and context. They cannot independently access and synthesize broader business intelligence, external APIs (beyond code usage), or real-world implications without explicit human direction.
  • Handle Entirely New or Unforeseen Scenarios Effectively: Trained on existing data, they may struggle with problems or situations that are custom to your working model and system that is different from the training dataset used for the AI Agent.
  • Operate Effectively Without Clear Human Guidance: They are tools that amplify human skills, not replacements for them. Developers must provide clear instructions, context, and feedback to guide the AI agent effectively.
  • Address Complex Ethical and Legal Considerations: Issues surrounding code ownership, licensing, and potential biases in AI-generated code require human judgment and policy enforcement. Humans alone can do it but do it badly any which ways. This requires fix in our value system. But lo behold, you will see AI Agents being abused in this area subtly altering the way humans think and act. This is the threat that all of us are going through even today in many areas of our lives without our realization. 

In conclusion, AI coding agents represent a significant leap forward in developer tooling, offering tangible benefits in terms of efficiency and productivity that I listed earlier. However, the narrative of wholesale developer replacement is a dangerous oversimplification. Smart executive leadership will adopt a balanced and well-informed perspective, in recognizing these tools as powerful enablers of human ingenuity, not substitutes for it. By strategically integrating AI and continuing to invest in your skilled development teams (ask them, "What is it that they learned last to make workplace better?". I am known to ask this to my team members), you will position your company for genuine and sustainable success in the evolving landscape of software development. 

The future is not about replacing human developers, but about empowering them with intelligent AI assistance.