Understanding and Eliminating Technical Debt (Mark Heath)

Full article

Just completed the “Understanding and Eliminating Technical Debt” course by Mark Heath on Pluralsight.

What is technical debt?

I would imagine most of the I.T. community has heard of “technical debt” by now. The term originated about 20 years ago with Ward Cunningham. He was trying to explain to his boss about the benefits of refactoring a piece of software he was developing… financial software. (Get it? Financial Debt… Technical Debt…)

As it turns out, the comparison really is a good one. Probably because we’ve all experienced financial debt to some extent.

  • We go into financial debt when we borrow money to get something right now which we wouldn’t have otherwise been able to get. (A house, a car… maybe a pool or set of furniture.)
  • If we don’t pay it back, interest accrues. The longer we wait, the harder it is to pay back… until one day it may be nearly insurmountable.

Likewise, in the development world:

  • We go into technical debt when we (hopefully) make an intentional, purposeful decision to take a shortcut, in order to get something now which we would’ve had to wait for.
  • Whether or not we repay the debt (refactor the “shortcuts” into something maintainable), future projects will build upon it anyway. The longer we wait to repay the debt, the crappier the foundation we’re building on, and the more the debt (and interest) compounds.

On the off-chance you haven’t heard the term, then most likely you’ve felt it, if you’ve worked on anything large enough for a long enough time. As Philippe Kruchten put it:

Most people who are very new to software have a hard time understanding this concept of technical debt. If ITs speak to undergrads they don’t get it. But if you’ve been in organizations that have large amounts of software written ten, fifteen years ago that you try to drag into the new year with adding new technologies, you feel technical debt. Maybe you haven’t yet found a word to describe this strange feeling, but that’s it.

I like that last line. I’d say even before I heard the term, I definitely felt it. You realize you’re fixing the same bug several times, or there’s areas of the code that no current employee wrote and no one wants to touch… that’s debt in the application.

I’ve been more aware of it lately, and when I was looking for the next Pluralsight course to take, Mark’s jumped right out at me. It was several hours well spent. First, you’ll realize you’re not alone; second, you’ll learn how to start reversing it (paying back the debt).

Without giving too many of the goodies away, I thought I’d share a few take-away points that resonated with me (yours will differ, depending on your own experiences).

On types of debt

Mark breaks debt down into several categories during his presentation.

  • Regarding “test” debt, manual testing can only cover so much of the application. There are only so many hours in the day. If we’re lacking in automated tests, bugs are more likely to get through. And then its the customers who end up reporting bugs, instead of us catching them before release. His way of looking at how these decisions affect the customer makes it all the more practical. It’s not just about developers doing something for its own sake… there really is a net effect on the users.
  • Regarding “knowledge” debt, I hadn’t thought of this as another form of debt, but I’ve definitely “felt” it too. Like when the team stares at a certain segment of code, reluctant to touch it, wondering what it does and whether anyone’s using it. This can especially happen when the original authors are gone and the documentation is outdated.

He breaks things down into several other types of debt too. You may find it useful for wrapping your head around things you probably already experience in your own code base.

On measuring debt

  • One simple way to measure the possible existence of debt is to monitor source control changes. If a file is constantly being changed by members of the team, then it may be doing too much and needs refactoring, or it may be full of bugs that keep requiring fixes.
  • Your bug tracking system should be able to report how many bugs were found in the current iteration and how quickly were they fixed (if they were fixed at all). Those and other bug metrics could indicate whether the number of bugs are increasing as time goes on, and how well the team is handling them.
  • He mentioned a couple tools for measuring code metrics which I’d like to check out; namely, NDepend and Source Monitor.

On testing

  • Mark suggests aiming for 100% coverage of business logic. Great advice, IMO. The business logic is where we’ve translated the customer’s request into code, and testing helps verify that the output is correct in all scenarios.
  • He mentions using the built-in code coverage tool in VS. For us NUnit users, that means installing the NUnit Test Adapter, which enables running NUnit tests inside Visual Studio.

On communication

Working with code that’s riddled with technical debt is no fun, and it certainly can affect the team’s morale. He lists out a few possible ways developers might respond:

  • Cynicism – We’re doomed to fix bugs forever and it may never get sorted out. This may lead to an exodus of developers (and knowledge debt).
  • Accusatory – It’s all your / his / her fault.
  • Extreme Caution – Change is too risky. Something may break. We can’t risk refactoring.
  • Cavalier – Go hog-wild with refactoring. Throw caution to the wind.
  • Clueless – Assume it’s just the way things are.

I wanted to list these because I’ve been in every single one of those positions, and you probably have been too. Sometimes for a few hours while looking at the same bug again for the umpteenth time; sometimes for a few weeks while trying to get a project to play nice with legacy code.

I definitely spent at least the first few years as a programmer in the “clueless” category, before talking to others and learning about best practices, technical debt and so forth. Not that I always follow them, but hey, knowledge is half the battle. :p

Mark also suggests demonstrating how to write cleaner code (unit tested, SOLID, DRY, YAGNI etc) by performing live refactoring demos, using actual code from the codebase. That’s a great idea. It’s easy to mock up some code and show the “happy-path” refactoring. But using the real thing, full of whatever crap your codebase suffers from? Everyone gets to see first-hand how to start removing those paint points.

On devs and managers

  • While we developers might feel managers just don’t care about maintainability and “doing it right”, it helps to put yourself in others’ shoes.
  • Management, more focused on the customers, and feeling pressure from shareholders and other interested groups, may feel developers don’t care about the customer’s needs.

It’s important to think about the effects of debt on things non-developers are concerned about too.

  • Technical debt is a hidden input that injects itself into the start of any and every project, and can result in significantly longer development times.
  • If programs are buggy, customers may get sick of it and not renew their service, nor recommend it to others, which can lead to financial debt.
  • Be realistic. Companies must stay profitable and so stuff must get done. If debt is to be repaid, we should be strategic about which debt gets repaid first.
  • Communicate with management when you know you’re about to take on technical debt.

The technical debt document

When faced with a particular debt you want to stamp out, Mark suggests writing a short document explaining the risks involved, proposed solutions, etc.

Writing stuff down can force us to think hard about what we’re trying to communication, and then you can actually hand that off to someone to review or share with others. May not work for everyone, but I like the idea.

  • What’s the problem and how/why is it slowing us down?
  • What’s the risk to the customer?
  • What’s the solution(s)?
  • What are the benefits to future development?

In his course, he walks through samples of what you might include in a document for each type of debt he defined earlier.

Practical techniques

His last module covers some practical ways to repay debt.

I liked a point he made about adding code coverage before making modifications to legacy code, where perhaps no one on the team knows its purpose (knowledge debt). First, assume it’s working as designed. From there, exercise it with a bunch of inputs, observe the outputs, and write tests that pass. This won’t tell you that it was correct to begin with, but it will prove that after your refactoring it functions the same way.

He also discusses a newer concept known as the micro-service architecture, which is about getting away from the monolithic, “does everything” architecture, and going towards smaller services that can be source controlled separately, deployed separately, and helps avoid tight coupling. More on micro-services from Martin Fowler.

Celebrating accomplishments

Debt is no fun, and repaying it is can be about one step above no fun. Mark encourages the team to keeping track of what’s been fixed, and celebrate success, which can help especially on days when debt seems piled up.

I’ve seen team members (myself included) get downright depressed about legacy, buggy code. I think part of the issue is not seeing how much progress we’ve made. There’s no indication of how many bugs were successfully fixed… they drop off the radar and are replaced by others, and the march goes on.

There are probably plenty of opportunities to get those metrics and more (such as total test coverage) in front of the developers, so everyone can see just how much progress they are making.

Okay, that was more than just a few take-away points! But we’ve got some serious legacy code and tech debt in our decade-old code-base, and it is depressing at times, and it does get old, and I would love to do something about it. If you find yourself in the same boat, check it out.


Grant Winney

I write when I've got something to share - a personal project, a solution to a difficult problem, or just an idea. We learn by doing and sharing. We've all got something to contribute.

Comments / Reactions