Building Better Software Through Collaboration: Whose Job Is It, Anyway?

How do we break down the artificial walls that keep us from creating great things together? How can organizations foster environments that encourage natural, unforced collaboration?
There are no quick fixes, but these are far from insurmountable problems. I propose the following five-level hierarchy as a solution:

There are no shortcuts to breaking down silos. You can’t fix the environment if the organization doesn’t understand the problem. You can’t improve the development process if the right environment doesn’t exist to enable healthy guidelines. You have to climb the pyramid brick by brick to the ultimate goal: better software through true collaboration.
Let’s look at each of these levels.

Level 1: Make Sure Everyone Understands The Problem

Photo credit: David Buckingham
Most organizational leaders would probably admit that collaboration is not as good as it should be, but they might try to solve the problem incorrectly. As Louis Rosenfeld recently said in “The Metrics of In-Betweenness”:

Many senior leaders recognize the silo problem, but they solve it the wrong way: if one hierarchical approach to organizing their business doesn’t work, try another hierarchy. Don’t like the old silos? Create new ones. This dark tunnel leads to an even darker pit: the dreaded — and often horrifically ineffective — reorg.

The first level is admitting that there’s a problem and that the current problem-solving methods just aren’t working. This isn’t about moving branches of the organizational tree around. It’s about planting the tree in more fertile ground to establish the right foundation in order to start looking for solutions.

Level 2: Empower Teams To Do Great Work

Once the organization is united around a common understanding of the problem, then the starting point for breaking down silos is to take a healthy look at the culture and work environments. Above all, the needs of makers (such as designers and developers) should be taken seriously by managers (those who direct and enable the work). Mike Monteiro takes on this issue by attacking the calendar in “The Chokehold of Calendars”:

Meetings may be toxic, but calendars are the superfund sites that allow that toxicity to thrive. All calendars suck. And they all suck in the same way. Calendars are a record of interruptions. And quite often they’re a battlefield over who owns whose time.

Paul Graham takes a more holistic view in “Maker’s Schedule, Manager’s Schedule.” He explains that managers break their days up into hour-long stretches of time, while makers need large blocks of time in order to focus:

When you’re operating on the maker’s schedule, meetings are a disaster. A single meeting can blow a whole afternoon, by breaking it into two pieces, each too small to do anything hard in. Plus you have to remember to go to the meeting. That’s no problem for someone on the manager’s schedule. There’s always something coming on the next hour; the only question is what. But when someone on the maker’s schedule has a meeting, they have to think about it.

Makers need long stretches of uninterrupted time to get things done, and get them done well. Most siloed environments don’t support this because of an insatiable need for everyone to agree on everything (more on this later). So first, helping managers understand why this is such a big deal for makers is important, so that the managers can effect cultural change.
Michael Lopp talks about this in his article “Managing Nerds.” Substitute the word “nerds” in this article with “designers and developers” (no offense intended). Michael describes how nerds are forever chasing two highs.
The first high is unraveling the knot: that moment when they figure out how to solve a particular problem (“Finally, a simple way to get users through this flow.”). But the second high is more important. This is when “complete knot domination” takes place — when they step away from the 10 unraveled knots, understand what created the knots and set their minds to making sure the knots don’t happen again (“OK, let’s build a UI component that can be used whenever this situation occurs.”).

Chasing the Second High is where nerds earn their salary. If the First High is the joy of understanding, the Second High is the act of creation. If you want your nerd to rock your world by building something revolutionary, you want them chasing the Second High.

And the way to help designers and developers chase the second high is to “obsessively protect both their time and space”:

The almost-constant quest of the nerd is managing all the crap that is preventing us from entering the Zone as we search for the Highs.

So, how do you change a culture built around meetings and interruptions? How do you understand what designers and developers need in order to be effective, and how do you relentlessly protect them from distractions? Here are two ways to start:

1. Ask the makers what’s missing from their environment that would help them be more effective.
   Find out what your designers and developers need, and then make it happen. A quiet corner to work in? Sure. A bigger screen? Absolutely. No interruptions while the headphones are on? Totally fine. Whatever it takes to help them be as creative as possible and to be free to chase that second high.
2. Start working on a better meeting culture.
   This one is a constant struggle for organizations of all sizes, and there are many ways to address it. I try to adhere to two simple rules. First, a meeting has to produce something: a wireframe, a research plan, a technical design, a strategic decision to change the road map, etc. Secondly, no status meetings. That’s what Google Docs and wikis are for (agile standups are an exception to this, for reasons best left to a separate article).

Level 3: Create A Better Design And Development Lifecycle

Once a team is optimized for creativity, then it becomes possible to build appropriate guidelines around that culture to take an idea from vision to shipped product. This includes the development process as well as the prioritization of projects, so that you only work on things that are important.

The Design And Development Process

Formalizing the design and development process is so critical: from the identification of user, business and technical needs, to the UX design cycle, to technical design and, ultimately, to development, QA and launch. By formalizing this process based on a common understanding of the needs of the business, the team will have no excuse for skipping the UX phase of a project because “it would take too long.” By providing different scenarios for whatever timelines are available, the team ensures that design and UX always remain a part of the development lifecycle.
But one particular aspect of the development process has a direct impact on organizational silos: ensuring the right balance between design and engineering, and how designers and developers work together. This isn’t explored enough, yet it can have far-reaching implications on the quality of a product.
Thomas Petersen describes the ideal relationship between designers and developers in “Developers Are From Mars, Designers From Venus”:

They are the developers who can design enough to appreciate what good design can do for their product even if it sometimes means having to deviate from the framework and put a little extra effort into customizing certain functionality. If they are really good developers they will actually anticipate that they have to deal with it and either use a framework that allows them be more flexible or improve the framework they prefer to work in.
And they are the designers who learn how to think like a programmer when they design and develop an aesthetic that is better suited to deconstruction rather than composition. They know that composition in the Web world is not like composition in the print world and that what they are really doing is solving problems for customers, not manifestations of their creative ego.

Beyond the usual discussion of whether designers should be able to code, one of the main causes of bad blood between the groups is that developers are rarely asked what they need in order to write the best possible code. Designers should always ask their development teams two questions:

1. “How would you like to contribute to the product development process?”
   It is amazing how few people actually ask this question, as is how the opinions of the people who understand the product at its most detailed level often don’t have a voice in ideation or prioritization. Any cycle that doesn’t include developers from the beginning will likely fail, because the conflict between design and utility cannot be resolved with detailed specifications. It can be resolved only around a table, with plenty of paper to draw on and time to argue about the best way to do things. Of course, designers need time on their own to create, but developers need to be given a chance to contribute to the product that they’re building in a way they’re comfortable with.
2. “What information do you need to start coding?”
   The theoretical discussion about low-fidelity versus high-fidelity mock-ups or prototypes is largely misguided when it comes to real-world development. The goal is right-fidelity specifications, and that all depends on the maturity of the application you’re working on and the style of the developer. Some developers need perfect PSDs before they start coding; others are fine with back-of-the-napkin sketches along with a solid UI component library. Find out what they need, and provide just that — anything more than what they need will not get looked at, and that’s when tempers can really flare up.

Bringing such diverse worlds together is hard work. But, in “So Happy Together: Designers and Engineers,” Dave Gustafson warns what might happen if you don’t invest in this:

What’s the alternative to this kind of collaboration? Keeping design and engineering separate, where the pass-off from one to the other is aptly called “throwing it over the wall.” Designers may enjoy an unhindered blue-sky design process, but they’ll likely be disappointed with what actually gets made. Without engineers in the design process, there are bound to be some unrealistic features in the concept — and without an understanding of the designers’ intentions and priorities, engineers are likely to compromise the design with changes to meet cost goals. Some money may have been saved on the engineering and manufacturing — but not enough to offset a product that misses the mark.

The Prioritization Process

When there is no clear development process, “prioritization” can end up being a complex algorithm consisting of the last email request sent, the job title of the requestor, and proximity to the development team. This is no way to build a product. One way to address the difficulties of prioritization is through the concept of a “product council.”
At a start-up, the entire company could make up this group — even if it’s a group of one. In large companies, the group should include the CEO and the VPs of each department, including marketing, product, engineering, support, etc. The name is not important — the purpose is. The product council would have weekly or by-weekly meetings with two goals:

1. Review the current product road map to assess whether the right priorities are being addressed.
2. Introduce new ideas (if any) that have come up during the week and discuss business cases and priorities.

This meeting would have several very positive outcomes:

• It would give the management team complete insight into what the product or design team is working on, and would allow for anyone to make a case for a change in priorities. This eliminates the vast majority of the politics you see at many organizations, and it frees up the teams to do what they do best: execute.
• No one in the company would be able to go straight to a designer or developer to sneak things onto the road map. The user, technical or business impact of every big idea must be demonstrated.
• It would prevent scope creep. Nothing would be put on the road map without something else moving out or down. This is absolutely critical to the development cycle.

From there, projects would move to small dedicated teams, which would have complete ownership of the design and implementation. The product council sets the priorities, not the details of implementation — those are up to the teams themselves. I’m always reminded of what Jocelyn K. Glei says in her excellent article “What Motivates Us to Do Great Work?”:

For creative thinkers, [there are] three key motivators: autonomy (self-directed work), mastery (getting better at stuff), and purpose (serving a greater vision). All three are intrinsic motivators.… In short, give your team members what they need to thrive, and then get out of the way.

In pursuit of collaboration, we run the risk of overshooting our target and gaining the false sense of security that “consensus” brings. Consensus too often results in mediocre products, because no one really gets what they want, so the result is a giant compromise. Marty Cagan says this very well in his article “Consensus vs. Collaboration”:

In consensus cultures people are rarely excited or supportive. Mostly because they are very frustrated at how slow things move, how risk-averse the company is, how hard it is to make a decision, and especially how unimpressive the products are.

So, even though everyone agreeing on something is great, having someone be responsible for the decisions in that particular project is infinitely more important. This person does not do all the work, but rather is the one who owns the product’s fate — its successes and failures.

In many organizations, this person is the product manager, but it doesn’t have to be. Whoever it is, the role should be clearly defined and well communicated to the rest of the organization. The role is not that of a dictator, but of a diplomat, working with UX, business functions and engineering to build products that are driven by user, business and technology needs.

Level 4: Communicate Better

Once the appropriate guidelines are in place and the teams are working effectively, it’s time to root out any other causes of mistrust that might still exist. And one of the best ways to build trust in an organization is to eradicate secrets.
I am huge believer in full transparency, and I see little need in keeping any relevant information about a project from anyone. (The prerequisite? Hire trustworthy employees!)
If plans, progress and problems are published for all to see, there is no need to hide anything and no need to play politics to get things done. Here are just some of the things that should be published on a public wiki for anyone to view and comment on (written from the perspective of a UX team):

• Roles and responsibilities of the product and UX teams.
• How road-map planning and the prioritization process work.
• How the product development process works, including (critically) where UX fits into even the smallest projects.
• The goals and success metrics for every product line.

Publish everywhere, invite anyone. The tools at our disposal make this so easy, from Dropbox and Google Docs to ConceptShare and Campfire. There is no excuse for keeping things to ourselves.

Level 5: Prove That It Works

When the groundwork is laid for silos to start crumbling down, one last piece of dynamite will blow it all up: it’s time to start proving that it works. People will believe you for only so long if you say, “Trust me, this is the right thing to do!” At some point you have to show them the money.
A common theme throughout this series has been that better collaboration results in better software. The only way to cement these changes into the organizational culture is to show that you’re actually shipping a better product because of it.
Here’s what I do to demonstrate the business value of a collaborative development process that includes a tightly integrated UX cycle:

1. Share case studies from other companies or projects that clearly show the business benefits of working this way. Showing that it’s been successful elsewhere should buy enough time and resources for the team to put in place its plans to follow a proper collaborative design and development process in one or more of its projects.
2. Start on a project where changes can be measured by an improvement in one of the three A’s of revenue generation:
   • Acquisition
     Getting new users to sign up for your product.
   • Activation
     Getting those new users to make their first purchase.
   • Activity
     Getting those first-time purchasers to come back for more.
3. Benchmark well before the start of the project, and set clear goals to measure the success of the project.
4. Follow through on the commitment to collaboration, and measure your results. See “How to Measure the Effectiveness of Web Designs” for ideas on which measurement tools to use.
5. Publish your metrics widely so that everyone in the organization can see the results. And don’t hide the failures. There will be failures — the trick is to own those mistakes, learn from them and get better.

Summary (aka “Whose Job Is It, Anyway?”)

So, whose job is it to break down organizational silos and build a collaborative development process?
Guess what? It’s your job. Whether you’re a designer, developer or manager, this isn’t something that someone else will get around to — if it was, then it would have happened already. Building collaboration is best done through a groundswell in the organization, led by the makers, the people who build the actual product. It might be difficult, but I hope there are enough case studies and examples in this series to help you get started on this journey of organizational change.
Collaboration only works if everyone in the organization is open about their processes and workflows, without fear of being judged unfairly. Arguments over what’s right and how to do things differently will happen. And they should — it’s the only way to get better at what we do.
Building collaborative environments is not easy, because change management is not easy. But the positive outcomes of doing this far outweigh the pain of making it happen. You’ll end up with happy, creative teams that feel a sense of ownership over what they’re building and a sense of pride in its quality.
I often remind my team that we are judged on the products we ship, not on the number of times we ask for help along the way. So, what possible reason could there be not to collaborate and create a better product — because it will make us all look better in the end anyway? (Hint: there isn’t one.)
Go. Make it happen.