Concept design in an Agile environment

Mar 15 2009

“When one has finished building one’s house, one suddenly realizes that in the process one has learned something that one really needed to know in the worst way – before one began.” (Friedrich Nietzsche)

Design for digital media differs from many other fields of design in that turning the design into the final solution requires the involvement of programmers, who are by definition designers themselves (as opposed to, for example, furniture design where the “implementation” requires craftsmen). Digital media design therefore is a process of multiple layers of design, bringing along challenges for the communication and cooperation between those involved.

While designers in the field often have programming skills and sometimes even produce smaller solutions themselves, specialized professionals are always part of the process beyond one-man productions. In the context of really big projects, there may even be several teams of programmers, subcontractors included, not to talk about the different levels of management that big corporations bring along. Designers are more often than not restricted by both prerequisite requirements and the possibilities and/or capabilities for implementing concepts as designed.

This essay aims at evaluating the background of Agile processes as well as to present concepts for the integration of designers into these processes. The focus hereby is specifically on concept designers, referring to interaction designers involved in the high-level design of software solutions or products, as opposed to ‘UI designers’ in the classical sense. Based on my own experiences from several Agile projects I have been working on as a designer in digital agencies, I would then like to propose a model based on literature review how I see designers can cope with these very popular ways of doing (programming) things.

1. Agile processes

Agile models of software development – initially referred to as IID, “Iterative and Incremental Development” – have been around already since the 1970s and earlier. The methodology was suggested, tested and proven suitable as an alternative for the waterfall model. While serving designers with a very reliable model of ‘design first, then implement’, the waterfall is challenging for software development. However, it took until the late 1990s before the new methods, most prominently Scrum with its iterative 30-day increments, were adopted widely. (Larman, 2003)

While so called “lightweight methods” were already in use in different forms, for example Extreme Programming (XP) or the aforementioned IID, the term “Agile” was coined in the “Agile manifesto” authored at a convention of a group of lightweight method advocates in Utah 2001: “The manifesto is a rallying cry: it says what we stand for and also what we are opposed to. Several items were worded to clearly make a distinction between our views and that of many others in the software industry. ” (Fowler 2006)

1.1 Philosophy

In their article “Iterative and Incremental Development: A Brief History”, Larman and Basili (2003) well describe the benefits of IID methods, quoting an earlier article by Basili and Turner:

The basic idea behind iterative enhancement is to develop a software system incrementally, allowing the developer to take advantage of what was being learned during the development of earlier, incremental, deliverable versions of the system. Learning comes from both the development and use of the system, where possible. Key steps in the process were to start with a simple implementation of a subset of the software requirements and iteratively enhance the evolving sequence of versions until the full system is implemented. At each iteration, design modifications are made along with adding new functional capabilities. (Basili and Turner 1975 cited Larman and Basili 2003, p.49)

Still a few years ago the waterfall model was considered a best practice by many. “Do the requirements, then design, and then implement.”, Larman and Basili (2003, p.55) describe the idea of “an orderly, accountable, and measurable process, with simple document-driven milestones”. The waterfall guaranteed clearly defined steps, followed by handovers, so that for example a designer would not make any more changes to a design after the programmers had started their work.

Visualization of the designer's involvement in the traditional waterfall model: the process allows a lot of time for research and ideation; once the design is finalized, it is handed over to the developers.

Figure 1: Visualization of the designer’s involvement in the traditional waterfall model: the process allows a lot of time for research and ideation; once the design is finalized, it is handed over to the developers.

But the waterfall model, theoretically a perfectly streamlined process, has its shortcomings. As Frederick Brooks, a critic of the waterfall model, puts it:

Much of present-day software acquisition procedure rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it. I think this assumption is fundamentally wrong, and that many software acquisition problems spring from that fallacy. (Brooks 1987 cited Larman and Basili 2003, p.52).

The philosophy of Agile methods is therefore not only a different method of programming software, but also includes assumptions regarding the designer’s role in software development. In the ‘Agile Manifesto’, the assumed benefits of the ‘new’ method over the waterfall model are formulated as follows:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan (Cunningham et al. 2001)

Feedback, collaboration, understanding, commitment and instant value creation were the main thoughts behind the Agile Manifesto. (Thomas 2007) With a method that relies on continuous change, this new model appears a lot more suitable to software development in general. For the designer, however, this means an entirely new way of doing things – there is no more ‘final specification’ that can be delivered for approval.

Visualization of the designer's involvement in the Agile process: the process is based on incremental design phases, including the iteration of the parts already created.

Figure 2: Visualization of the designer’s involvement in the Agile process: the process is based on incremental design phases, including the iteration of the parts already created.

1.2 Practicalities

In practice, the implications of the Agile method can be derived almost directly from the four lines of the manifesto:

a) Software is implemented one interaction at a time, as opposed to the creation of an entire solution at once. For this purpose, requirements are usually defined as so called ‘user stories’, brief sentences expressing one particular action the user wants to do with the system. These small chunks of functionality are then considered the single tasks for the programmers. When needed, chunks that are considered too big can even be split into smaller tasks.

b) Documentation is reduced to an absolute minimum, not only because it is considered obsolete, but also due to the fact that any part of the design may change at any given time, rendering all existing documents outdated. The reduction of ‘waste’ is one of the key elements in agile methods, aiming at the creation of value (working software features) rather than documents that are not part of the software product itself.

c) Instead of agreeing on the implementation of an ‘approved’ design, the scope of a project may change. All the specifications are added to a backlog, from where then small chunks are implemented. As the software grows (one of the agile principles is ‘release something every day’, though the release intervals may be significantly longer), the current state can directly be reviewed – possibly even released – by the customer. Based on the state of the implemented features, the backlog may be updated and decisions are made what features are taken to implementation next.

d) The requirements of the project can change at any time, on behalf of the technologist or the designer. This enables to react directly on technical challenges or design flaws emerging during the implementation and on the other hand is a necessary step to adapt the design to the selective implementation of features from the backlog (i.e. the omission of a certain feature may require an update to another functionality that is being implemented).

2. Designer involvement models

In this chapter, I want to analyze some articles and discussions on the subject of “Designers in agile environments”. Starting from an attempt to discover why agile is a somewhat uncomfortable terrain for many designers, I summarize some of the suggested methods of cooperation brought up in literature.

2.1 Background: The clash of cultures

As already mentioned in the previous chapter, the shift to iterative and incremental development methods implies leaving the designer’s traditional terrain. As Bowles (2008) puts it:

“Time, research, and ideation have historically been designers’ comfort zones. They suit our natural aptitude for vision and intuition, and allow us to carefully synthesize nebulous things like ‘brand’ and ‘user behavior’ into objects developers love: specifications, sitemaps, layouts, and graphics.”

The last part of the quote only on the first glance contradicts what I elaborated on the programmers’ desires in the first chapter. The fact that Agile emerged from the perception that ready-made specifications are not flexible enough for the realities of software development does not mean that specifications wouldn’t be needed any more. The change that is happening is more about their extent and form, which seems to be what requires quite some adaptation by the designers.

But the problems do not only reside in the different way and rhythm of working. Agile was initially designed to serve the programmers rather than the conceptual designer, a base pillar of the method that cannot easily be turned upside down.

Jeff Patton uses a Gartner quadrant diagram to explain the field where agile development methods have their origin: the quadrant he calls “traditional IT where an internal development organization builds or contracts for the construction of products used internally by the company” (Patton 2008b). As he describes, agile methods were invented for efficiency, not for user experience. And this particular area of software development (internal tools) is not exactly famous for producing highlights of user experience design.

Jeff Patton explains where Agile methods emerged and that this is not the area where UX practice is most critical. (Patton 2008a)

Figure 3: Jeff Patton explains where Agile methods emerged and that this is not the area where UX practice is most critical. (Patton 2008a) [Image source]

Of course, the need for sustainable models of collaboration between programmers and designers has been acknowledged by both sides, leading to the question what are the possible models that enable iterative software development while preserving the designer’s control over the entirety of the product.

2.2 Developers and designers together

Aaron Smith discusses the value and limitations of UI designers in an agile environment. He suggests their ideal use to be in the mitigation of risk areas, designing flows, specifying main views and creating demos where necessary. Thereby, their major value is in ensuring a cohesive look and feel as well as an elegant usability, through checking and balancing the results and creation of templates. (Smith 2007)

Smith – who in the same text also makes clear that the UI designer’s role cannot be to design every single screen – makes an important point here: While the Agile environment is in a certain way chaotic, allowing the developers to concentrate on single features rather than the entire product, the involvement of the designer is important in order to maintain control over ‘the big picture’.

The integration of the development and design activities therefore is the key to successful design work in an Agile environment. In a document called ‘Agile Web Design Manifesto’, in reference to the Agile manifesto, Emily Chang and Max Kiesler describe agile design as: “user interface design strategy meets agile programming methodologies.” (Chang and Kiesler 2006). One important distinction the authors make is that design should not be a separate process from the implementation, but embedded into the iterative cycle: “Design isn’t just a phase you do at the end of a development project.”. In transferring the principles of Agile development to design, they propose a set of values:

Core principles of Agile Web Design
- Design the system not the surface
- Design as evolutionary and user-driven
- There is no page, only pathways
- Rapid and iterative over final
- Simplicity over complexity
- Collaborative and open design (Chang and Kiesler 2006)

But how about the early phases of design projects, with research, high-level concepting etc.? The approaches presented above appear to be limited chiefly to UI design, where the designer’s role is to shape the interface between the user and the computer. In these approaches, the UI designer becomes part of the development team, adding their specific expertise to the process. The points made by Smith, Chang and Kiesler surely are valid for concept design as well, but the iterative and incremental approach cannot be applied directly on high level design decisions.

2.3 Agile concept design

The challenge of getting design and development processes unified into one common model is not new. There is a bunch of models to be found from literature on the topic, suggesting different ways of coping with the different needs of both sides. Later in this chapter, I am presenting two specific approaches that I found noteworthy. I will discuss these, together with my own experiences, in chapter 4.

In his article “Twelve emerging best practices for adding UX work to Agile development”, Jeff Patton lists methods that he has recognized as patterns emerging from their use by practitioners in the field:

1. Drive: UX practitioners are part of the customer or product owner team
2. Research, model, and design up front – but only just enough
3. Chunk your design work
4. Use parallel track development to work ahead, and follow behind
5. Buy design time with complex engineering stories
6. Cultivate a user validation group for use for continuous user validation
7. Schedule continuous user research in a separate track from development
8. Leverage user time for multiple activities
9. Use RITE to iterate UI before development
10. Prototype in low fidelity
11. Treat prototype as specification
12. Become a design facilitator (Patton 2008a)

The most important patterns described, from my point of view, are the close enclosure of the designers in the product owner team and allotting time for researching, modeling and designing ahead of the Agile process. Basically, these both go hand in hand – aiming to ensure that the designer is and remains the owner of a concept based on strong foundation. Pattern #12 is closely related to that as well: the designer – due to the change in her role – should no longer consider herself the instance writing specifications but rather a catalyst for handling constantly changing design requirements (including change requests coming from the developers’ side).

No less important are the remaining patterns. I consider them more practical guidelines, as opposed to the aforementioned three basic mindsets. The chunking of tasks, the different ways of gaining time for design work and the utilization of user testing and prototypes all are means to keep the designer in control of the conceptual entity created, whereas the programmers can apply their iterative methods for developing the technical parts.

2.3.1 Parallel track development

“Parallel track development” is one of the patterns mentioned by Patton, referring to a concept first described by Lynn Miller in 2005. She describes a “double-track method”: a Developer Track and an Interface Designer Track. (Miller 2005, p.5)

The process described by Miller starts with a so called “Iteration Zero”, a phase to be used for basic design tasks without parallel development. This phase is often used for the initial setup of the development environment by the software team.

Starting from phase 1, the idea of the model is that the development team always implements what the designers have been working on during the previous phase. Meanwhile, the interaction designers review the outcome of the previous phase’s development efforts, design features for the upcoming phase’s development and gather input for the phase after next. This way, there is a crossed handover between the two tracks after every phase of the project: the designers receive the latest development version from the programmers for review, while the programmers are handed over the designs for the next phase. As an important feature, these designs also include the designer’s reaction to mutations the development version experienced since the previous handover.

In phase 1, the interaction designers still have time to work on essential design elements, as the Development Track schedule during phase 1 is mainly reserved for high level features with little UI relevance.

The 'Dual tracks' as described by Lynn Miller. (Miller 2005, p.9)

Figure 4: The “Dual tracks” as described by Lynn Miller. (Miller 2005, p.9) [Image source]

This method has successfully been used by Lynn’s company. In her reflections she emphasizes that they have been “able to maximize the quantity and impact of customer input by having the interaction designers work in a parallel and highly connected track alongside of the developers.” (Miller 2005, p.9). At the same time she also mentions possible shortfalls: not every feature is possible to design or implement within one phase and the dependence on the synchronized allocation of design and development resources can be challenging. The model relies heavily on good timing, not only in resourcing but also in the scheduling of the deliverables to be handed over to the other team.

2.3.2 Waterfall first, then Agile

In a presentation titled “Human-centered design meets Agile Development”, Maria Giudice discusses how to integrate Agile methods into a human-centered design and research process. Quoting designers and engineers, Giudice points out a few crucial differences between the waterfall and the agile methods. The waterfall model is described to easier enable the allocation of time for “thinking big”, while a quote on Agile states it “can take off pressure off designers from getting it right the first time”. (Giudice 2008)

Giudice presents a list of seven so called “Big disconnects”, the contraditions in the mindsets of designers vs. engineers:

Holistic vs. modular thinking
Research-based assumptions/decisions vs. hunches/intuition
Enumeration and alternatives vs. iterations
User research vs. user validation
Scheduled change vs. continual change
Artifacts vs. working software
Quality vs. quantitiy (Giudice, 2008, slide 30)

Derived from the comparison of waterfall vs. agile and the discussion of the aforementioned disconnects as well as the similarities between the two parties involved, her presentation suggests a hybrid model, the “Waterfall-Agile Combo”: in this model, the phases of Discovery, Strategy and Design are organized in a waterfall matter, whereafter the process shifts to the Build and Transfer phases utilizing Agile iterations.

Figure 5: Maria Giudice's 'Waterfall-Agile Combo'. (Giudice 2008, slide 34)

Figure 5: Maria Giudice’s “Waterfall-Agile Combo”. (Giudice 2008, slide 34) [Image source]

This model is not to be misunderstood as two strictly separated phases, where the waterfall part’s outcome is a traditional concept specification that is then implemented using Agile. Giudice’s idea is about giving the designers time to do the base work they need before switching into an “agile” mode together with the developers. At the end of the “Discovery, Strategy and Design” phase, the designer would be equipped with a set of traditional design variables (such as different models, a basic flow etc.), ready to turn these design goals into something real – which is then being done in the “Build and Transfer” phase.

3. Experiences

In this chapter I will describe some of the challenges I have experienced myself. These are based on a variety of Agile-based projects, from small internet start-ups over medium-sized web application development to huge projects for big multi-national corporations. I have worked in these as a designer with different grades of responsibility from UI design to lead concept design.

My experiences described here reflect both my own observations and those reflected on as a team. Some of these were identified and solved before they caused problems, others emerged from discrepancies between theory and practice of a particular process. And some are things that had to be learned “the hard way”, causing avoidable headaches.

3.1 Using “Waterfall” specifications with “Agile” production

This is an issue that is prevalent in subcontracting situations, where the “design” part of a project is outsourced: the design project is bought/sold as a “traditional” project with pre-defined requirements, a time frame for creating a design and a concept documentation as the desired deliverable. The Agile method is then only applied when the concept is handed over to the programmers.

In this kind of situation, the final product may resemble the initial design only remotely. From the moment the design is handed over for implementation, design decisions in this setup are made in the software team. Even though this team may include UI design specialists, the pressure of timetable/budget easily leaves them with the task to brush up the outcome of the production sprints rather than giving these designers control over what the programmers are going to do.

Projects where the concept designer is fully integrated with the development team have been the only instances where I experienced the full absence of this issue. This fits very much with the observations of Lynn Miller, who talks about the importance of having parallel tracks and their timing. Also, this underlines how Giudice’s “Waterfall-Agile Combo” does not suggest a separation of the waterfall and agile phases, but refers to a sequential process with no handover in between.

3.2 Sprint planning without involvement of the designer

In almost every agile project I have been involved so far, the decision making process when planning backlog priorities has been a major challenge. Naturally, when time and resources are limited, the concept designer is not the only person with an opinion on what are the main priorities. Depending on the size of the company, marketing department, management, related product lines and other stakeholders have a strong influence on the backlog planning.

Consequently, decisions are not always made based on what is best for the concept, but what is best in terms of market, offering or bigger scale development plans. Again, the risk is big that the designer ends up being the person receiving implemented features to “make them fit with the concept” after the code has been written.

It appears to be a good practice to make some kind of prioritization list an essential part of the concept itself, showing up not only what would be the ideal solution, but also what steps should be taken in what order to achieve an optimal incremental development. Ideally, of course, the concept designer is involved in the sprint planning itself to be able to react directly to changes of scope.

3.3 Lack of insight into the long-term strategy

The aforementioned idea of making priorities a part of the concept is not always realistic. It shall be mentioned that this “roadmapping” of a concept by the designer requires quite a lot of insight into the client’s roadmap and strategy – information that is not always given or possible to be given to a vendor.

A designer working on a high-level concept (or for a matter of fact: any designer) quickly develops her own visions on a product. It can be a major challenge if it is not possible to synchronize the visions of the designer and those responsible for setting out the overall roadmap. While these problems are not specifically related to agile methods as such, but a potential issue in any design project that involves more than one party, the risk with Agile resides in the process: both sides, developers and designers, are working in small increments. This bears the risk that, while in consensus on the design of a particular feature, the programmers may end up having a different understanding of a feature than the designer.

A solution to the described situation is to communicate not only the detail-level design decisions and visions for single features, but to regularly ensure that both sides have a common understanding of what role these features play in the big picture. For example, it makes a major difference whether some feature is supposed to be a temporary solution that will later be replaced with something “bigger” or whether the same feature is intended to be the backbone of an entire product.

3.4 Failure to identify and fix feature dependencies on partial implementation

As described earlier, the key idea of agile processes is change. Nothing is locked into a “final” design before it has been implemented, and specifications may even change on behalf of the technologists. But “being agile” does not only mean that everybody can do changes. It also includes the responsibility of both sides, designers and developers, to accept changes made by the other side and take them into consideration.

This is where another problem may arise: since the Agile process allows to prioritize small parts of a bigger concept, as well as to digress from the design during implementation, changes carried out on one part of a concept will most likely affect other parts as well. Some UI updates may be required, even an entire feature or feature set might need to be realigned with the change. Failure to either communicate changes made or to identify and fix the parts of the concept dependent on what has been changed necessarily may lead to major inconsistencies in the outcome.

Once more, communication is the solution. Avoiding this problem requires the developers to actively communicate where a feature has been changed from its initial design or where a prioritization decision led to a limited implementation of some feature. Then, the designer has to accept the change as a part of the process and see how this affects the overall concept (which should not negate the right of the designer to insist on making another change to the feature in an upcoming phase or to strongly suggest dependent tasks from the backlog for implementation). The software developers again have to accept that their changes, while legitimate from their point of view, may affect the product’s entirety in a way that requires another change or correction.

4. Reflection

In chapters 1 and 2, I discussed the history and principles of Agile methods as well as patterns and models for the integration of concept designers in these. In summary, I see two important basic requirements for a successful involvement of concept design:

a) The role of the designer has to be clearly defined in a way that allows her to keep control of the overall concept of the product, as well as of UX and UI matters. This requires an adaption of new mindsets on both ends. Getting rid of the traditional thinking of a static specification, created by the designer and turned into code by the programmer, there is no longer a pre-defined “interface” between the two parties. As suggested in the literature, the designer has to be closely integrated with the product team, avoiding situations where programmers have to make their own interpretations of the concept or where solutions that came up during a development cycle remain without verification against the concept by the designer.

b) In terms of methods, the iterative nature of Agile methods should be used for the benefit of the designer’s needs to regularly evaluate the latest version of the product and cross-check it with the overall concept. The methods mentioned in the literature, including but not limited to user testing and validation, prototyping and asynchronous research, are all a crucial part of ensuring that a project remains “on track” and that prioritization decisions made by the development team do not negatively affect the user experience or the concept in its entity. At the same time, the designer has to give up control to a certain degree. Not designing every single screen and view into the smallest detail (as in the classical waterfall model, even though these details rarely get implemented as designed) means to “let the concept live” to a certain extent and then validate the results afterwards.

Derived from above requirements, it is safe to say that successful concept design in an Agile environment is only possible if both the design and the development process are closely synchronized; this includes the need for both sides to slightly move away from their ideal modes of conduct in order to establish a true team where both sides have competence to make changes, but these are always evaluated and – where needed – altered afterwards by the other.

This – and chapter 3 supports that argument with some examples of challenges from real life – leads to the conclusion that conversation is the most important variable for successful cooperation between designers and developers in an agile environment. But the “communica­tion” I am hereby referring to is not only on an individual level, but above all has to be the foundation of any process model created for carrying out a design project using agile methods.

5. Conclusion

The main question of this essay was how concept designers can best do their work in an environment where the implementation is done using Agile. As I presented, concept design – referring to the design of and control over a high-level concept – can be done to the same extent than with more traditional models. I even claim that the continuous participation in the development process gives the designer more control over the outcome than for example the waterfall model, where prioritization, changes and updates carried out after the “final concept delivery” happen beyond the designer’s control.

But this requires both sides to work closely together, using a process that ensures a well-timed parallelism of the incremental development and design phases, as suggested by Lynn Miller. If the two tracks are out of sync, the mindset of those involved is still in “waterfall” mode (i.e. the designer considers her designs “final” or the programmer expects to get “approved specifications” to the last detail) or the project environment does not allow fluent communication, the outcome of the work is likely to be dissatisfying for everybody involved.

What does this mean for the early stages of concept design? All authors on the subject acknowledge the need of the designer for proper research, planning and concepting before the implementation begins. While the so called “Iteration Zero” is an essential part of most Agile processes, Maria Giudice’s suggestion of preceding the agile phase with a waterfall-oriented design phase is essentially a more elaborate definition of just that.

Maria Giudice's 'Human-centered-Agile' model. (Giudice 2008, slide 36)

Figure 6: Maria Giudice’s “Human-centered-Agile” model. (Giudice 2008, slide 36) [Image source]

What she calls “Cycle 0″ is a phase reserved for different modeling tasks (goals, users, scenarios, tasks) as well as the creation of flows, schematics and prioritization. This phase is even preceded by another phase of research and assessment. An important element of the “Human-centered-Agile” model is the five parallel channels of project lead, client, UX, visual design and engineering that only together serve as enablers for the model as a whole.

Combing the “Human-centered-Agile” model with Lynn Miller’s “Dual tracks” leads to a process model I believe to be very well suited for doing Agile development with a strong role of the concept designer.

Schematic overview of my proposed combination of 'Human-centered-Agile' and 'Dual tracks', combining the most suitable parts of the waterfall model and Agile processes.

Figure 7: Schematic overview of my proposed combination of “Human-centered-Agile” and “Dual tracks”, combining the most suitable parts of the waterfall model and Agile processes.

In this model, the needs of the designer are served by a phase of research, modeling and concepting (in Giudice’s terminology: Discovery, Strategy, Design), as known from the traditional waterfall model. However, the modeling and concept phase does not only represent stages of the waterfall process, but also form the so called cycle zero of the Agile process. The following implementation cycles then apply the idea of parallel tracks, where the outcome of each design and implementation iteration serves as input for the next cycle on the other end.

Bibliography

Tweet this Share on Facebook Send by e-mail More bookmarking/sharing

Leave a comment:

(this address is not published here). .