A less commonly discussed side effect of adopting an iterative and experiment-driven design process is the incurring of something called Design Debt. This is where the cohesion and consistency of a design deteriorates over time as new experiments are run and new elements are introduced into the design. Strong design direction and regular refactoring can help to periodically unify the design and uphold its structural integrity.
I’ve always been a major proponent of iterative design (namely Lean UX) and at this point, I’d be hard-pressed to name a successful SaaS company that isn’t leveraging experimentation, rapid iteration, and data-backed decision making in their design process. Derivatives of Agile methodologies can be found all over the place, and when coupled with tactics like A/B testing and hypothesis-driven experiments, they’re pretty powerful. But when applied over time, they can yield an important side effect that’s rarely discussed.
The side effect that I’m referring to is something called Design Debt. You’ve probably heard of Technical Debt before and this is a lot like that. But where Technical Debt affects the integrity of the codebase, Design Debt affects the integrity of the user experience. This is what happens when a bunch of incremental changes collect over time and yield a disjointed, inconsistent, and patched-together experience. With each successful experiment, a small amount of Design Debt is incurred as the design moves slightly away from it’s original, cohesive roots.
When a new design is born, it’s debt free. Think of a website or a product after a full design refresh. Everything is cohesive, intentional, and well-thought out. It’s as the designer originally intended, and as such, each element in the design was created with a certain awareness of its surrounding elements. The navigation is aware of the CTA, the CTA is aware of the video, the video is aware of the social feed, and so on and so forth.
This sort of reciprocal awareness between elements is what creates a notion of consistency and cohesiveness in the design. But while new designs excel in consistency, they also leave a ton of room for optimization. We collect data on the new design’s performance, determine where areas of weakness or opportunity may exist, and release an experiment. When a new variation of an element wins, we integrate it into the design, bringing us one step away from the original. This is good because we’re ultimately improving the design. But it comes at a bit of a price.
While these new elements may be created with an awareness of their surrounding elements (sometimes they aren’t), no matter what we do, the original elements simply won’t share that critical awareness of the new elements. The original elements weren’t created within the context of the new elements. This is where we start to incur Design Debt.
As Design Debt builds, it accrues interest that compounds on itself over time. The more new elements that we introduce into the design, the less elements were created within the context of each other, and the more disjointed the design becomes (despite the fact that it may be performing better quantitatively). Eventually, we have to break the cycle of interest and pay down the debt by refactoring the design.
When designs aren’t regularly refactored, they start to collectively form what may look like a digital version of Times Square (an entertaining example of this would be Ling’s Cars). Numerous disjointed and unrelated elements in a single field of view, all of which are competing for the user’s attention and may even be solving for different goals. Projects that are especially at risk for this commonly include a few of the following characteristics:
An interesting phenomenon in data-backed design is that while data can take a good design and make it great, it can’t take a bad design and make it good. Data isn’t a means for creative direction; it’s a means for optimization. And any great design is going to need a solid amount of both in order to be successful.
This is why, in recent months, many designers have publicly shared an angst for data-driven design, and expressed the importance of intuition in design. They’re emphasizing the need for a balance between data and creative direction; machine and man.
Just as design used to be run exclusively by intuition, the dawn of big data caused it to be run exclusively by numbers. Neither of these approaches are healthy or responsible. More data does not necessarily equate to better design, and neither does more ego or more intuition. The best approach exists somewhere in between, where we focus on creating designs that are both data-informed and human-centered. In the iterative or experiment-driven process, the best way to do this is through multiple small iterations followed by a refactoring.
During the small iterations that lead up to a refactoring, there are several things that we can do to extend the life of a design and maintain it’s health along the way. Practically speaking, these preventative measures just ensure that we’re always doing our best to produce high quality designs and only take on debt when it’s absolutely necessary.
When a design has taken on a substantial amount of debt and it needs to be paid down, the best approach is to initiate a refactoring as a large iteration in a process that otherwise contains small iterations.
Adopting an iterative and experiment-driven process is undoubtedly one of the best ways to learn about an audience and develop a design that truly works. By all measures, designers should be doing more of it. But it’s important to always consider the debt that can be taken on by a design over time, and plan for refactoring as necessary. This stuff doesn’t come in the “A/B testing manual” but it’s a critical part of the experimentation process.