Thinking Like An App Designer

There’s more to designing mobile apps than meets the eye. The task requires a deep knowledge of devices, and it often means changing the way we think — even if that means leaving behind much of what we’ve learned from designing for the web.
I started my career like many designers: working on print design projects. Shortly thereafter, I discovered the world of websites, which fascinated me and became the focus of my work for some time. Along the way I learned concepts related to interaction design and user experience, which I hardly knew existed until then.
At some point I felt stuck in a kind of repetitive work cycle: I had found a formula, an almost surefire way of designing, that enabled me to deliver each new project more easily and quickly than ever before, but in some way or another, all of the designs looked similar. It was then that the first versions of iOS and Android were released.
Thereafter, looking for a breath of fresh air, my life took an unexpected turn, focusing me on something that seemed completely different: mobile app design. That was what I needed, a real challenge. I not only had to learn new things, but somehow had to forget part of that magical recipe for designing websites that I had been using till then.
A few years after that aha moment, just days ago in fact, the design community met the launch of new apps for Apple’s smartwatch with mixed emotions. On the one hand, seeing a device that offers a world of new design possibilities is exciting. On the other, the challenge of understanding a medium with unprecedented interactions and use cases is daunting.
It’s not the first time we’ve seen a device like this on the market. But as new technologies that require new design approaches appear more and more frequently, we will have to adapt at an increasingly quicker rate.

Of course, the first big push came nearly 20 years ago, with ubiquitous Internet access, which ushered in the first websites. Before then, most designers, like me, focused on print, and the arrival of the Internet required us to learn how to design for an interactive medium. The user had become much more than just a spectator.
The transition was harder for some than for others, and that’s why so many websites of the time looked like signs with buttons on them — designers didn’t immediately understand the characteristics of this new medium.

The same thing is happening today with apps. One often comes across applications that look like mini-websites: They’re constructed as web pages “translated” for smartphones, and in turn, they fail with contrast, font size, touch targets and gestures. They also miss the mark with who will be using the app, where and on what devices.
When designing for mobile, escaping from a web structure is imperative because it can work against us. We have to face projects with a different mentality and fully comprehend the mobile devices in order to take full advantage of them and deliver delightful experiences.
So, what do you need in order to be an app designer? Beyond knowledge and tools, you need to change the way you think. Below are a few recommendations for adapting to the world of app design.

Change The Way You Work

Hundreds of apps are entering the market at this very moment — you’ve got no time to lose.
As professionals, we can no longer afford to spend weeks or months on detailed fancy designs before launching a product, only then to realize that other apps have already solved the problem (and most likely solved it in a similar way).
This is where lean UX comes in, a way of working in short quick cycles. The approach entails continual iteration on design and development, keeping one single focus in mind: that nothing is certain until users try it out.

Thus, app design can’t begin with interface design in Photoshop or Illustrator. It has to start before that, with wireframe-based prototypes and basic designs. That way, if change is necessary, it will take moments — not months — to apply.
Beginning with the visual design is a normal tendency for designers: That’s the stage that is usually most interesting to us. In some way, we also got used to working in this way because it was the only way possible.
Recently, in speaking with many designers, most look at me with surprise when I recommend that they start working on projects without even using the computer, but instead by simply drawing on paper. It sounds so natural that we often forget it’s even a possibility. Designing like this has proven to be very useful in preventing us from thinking in terms of variables (such as the size of the design document, colors and fonts used, etc.) — variables and details that can actually block our creativity when we are starting a project.
Personally, I find sketching on paper to be much more useful because we focus only on the idea and the problem to be solved, without falling into the trap of considering design details, or at least not in that first stage.
“Mobile first,” or starting with the small screen, is also a good strategy to employ. Even if your app will run on both smartphones and tablets, beginning by designing for the smaller phone is more effective. Starting the design process from a mobile standpoint forces us to prioritize, choose and above all alter our mental structure, which is accustomed to the larger screen of a desktop computer as a general point of departure.

The first time I started a project this way, I almost ended up with a headache. Perhaps it’s a similar situation when you go to the gym for the first time; muscles that you haven’t used for a while start to hurt, due to the effort of working differently. If you haven’t done it yet, try it some day. You’ll notice how your priorities change, and you will sooner realize what are the truly important things in your app, the things that are indispensable.
You could also draw a parallel with travelling with luggage. If you have a small suitcase, what should you bring? You’d bring the most important things, of course, the things you are sure to need during the trip, and leave everything else behind. Believe it or not, this ordinary example is similar to what happens when we design for a smartphone.

Learn From The Other Half

The success of an app depends on both the designer and the developer, who must work together to innovate and achieve the highest level of quality. To accomplish this while working efficiently, you can shorten iterations by working in parallel. For this to work, you must both be conscious of the complexities of the design as well as of the actual implementation.
One way to do this is to learn to use development tools, not necessarily to develop apps from start to finish, but at least to build screens that can be transferred quickly and faithfully, while accurately communicating the design’s intent.

Shortly after arriving in Barcelona, I started working at a tech startup. I was surprised to find that I was the sole designer there — I was literally surrounded by developers, and at first there were a lot of bottlenecks because everybody was waiting for me to finish my design work in order to continue their own.
With time, I learned how to use tools such as Xcode and some basic programming languages. These enabled me to make more complex proposals and to work at certain moments at the same pace as the rest of the team.
After a while, once I had been with the company for months, developers learned the way I work, too. They would now open my design documents directly and would know how to export design assets without my help.
Finally, I would like to teach you a little trick: Learn to use the same terminology as the rest of the team. Doing so, you will feel that everybody in the office speaks the same language. Designers and developers often refer to the same thing but in different words, which makes understanding between parties more difficult.

Recognize The Value Of Teamwork

Going back a bit to the previous topic, teamwork goes far beyond sharing office space. It entails collaboration between roles and constant communication. Lean app design entails putting an end to cascading processes, where the work of one individual begins where another one’s ends.
But in practice, most teams have settled into an efficient hybrid process, with developers following an agile methodology (working with a goal to accomplish by a specific deadline) and designers following a lean one (rapid iteration, usually without a specific deadline).
While there is still a vestige of a “cascade” (or waterfall) here, with designers usually working just a bit ahead of the rest of the team, developers can certainly start without waiting for every design to be ready.
That’s why an app designer must recognize that their coworkers — especially programmers — are allies, not enemies. Understanding one another and growing accustomed to working together will lead to a better, more synergetic result.
We are often afraid to share our work. We tend to be somewhat defensive of the reviews and comments that may arise, especially coming from fellow designers. The truth is that many of the things that others say are vital to improving our proposals, including feedback from those who do not have a design background, such as programmers. Their vision, generally more pragmatic than ours, can provide a good balance.
People in a company I visited a while ago told me that designers and developers work on separate floors, communicating through various systems of instant messaging. I suggested that they try something else: Designers and developers who work on the same projects should try sitting all mixed together. A few months after that, they told me that the quality of their products had dramatically improved, as did even the relationships between team members.

Use Various Operating Systems

When you’re designing a mobile app, you can’t just use your own phone for reference. If your target is the global mobile market, you’ve got to have access to at least Android, iOS and Windows Phone devices and grow accustomed to using them constantly in order to fully understand how to resolve different design scenarios on each platform.
The truth is that the world doesn’t begin and end with the iPhone. So many designers limit themselves to iOS, and when the time comes to design for Android, they just translate — often almost literally — the original app.

Interaction patterns (i.e. established solutions to problems of design) are different on each operating system. As such, designers must alternate frequently in order to assimilate them and use them correctly.
To name just a few examples, the location of tabs, how to show menus, and when and how to display primary and secondary actions vary on each of these mobile operating systems.
To learn the different patterns on each mobile OS, here’s a practical exercise you can follow: Download and install the same app on various operating systems, and analyze their distinctions and similarities critically; the patterns should be different. So, if you find that an app looks and works practically the same on iOS and Android, it’s possible that something has not been done correctly on one of them.

Moreover, continually looking at and trying out other applications is a great way to discover how to solve your own interface problems. For example, many times when I feel some sort of a creative block and don’t know how to fix a problem, I’ll start playing with my phone, carefully analyzing the apps to see their approach to the problem I’m facing.
If you don’t have a phone near you, you could check websites such as Pttrns (for iPhone), Android App Patterns and Windows Phone UI Design Patterns, which have plenty of examples that might be useful to you.

Prototype Everything

An agile methodology requires understanding how something will look and function prior to implementation. Likewise, prototypes help us to evaluate (via user testing) an app’s usability. Ideally, we would not have to wait long before building our first prototype.

Few of us probably remember the Palm Pilot: One of the first prototypes was a simple piece of wood with a sheet of paper on it with the printed design of the interface. Employees would carry the prototype in their pockets as if it were a real device.
For what it was intended to test — size, weight and comfort of use — that prototype was more than enough. The point is to determine the extent to which it’s necessary to design in order to test what you have in mind, and then get down to work!
Numerous tools exist for prototyping and even viewing a design on smartphones as though it were a finished product. That being said, a prototype has to do more than just show a static design; it must also include images, transitions and gestures.

Choosing the most appropriate tool will depend a lot on your workflow, what you want to achieve and the result you are expecting. For example, I often use POP when I have no more than simple wireframes drawn on paper. With this app, I can take pictures of designs, then add gestures and transitions — quickly and easily!
But when a concept is more advanced, I find it more convenient to work in design-specific software, such as Sketch, and then build a prototype using Marvel, Flinto or InVision, making for a more polished prototype. (Note that sometimes one tool is not necessarily better than another. Choosing among them will partly depend on which you feel most comfortable with. However, if you’d like to learn more about prototyping with Sketch and InVision specifically, Smashing Magazine has you covered!)
As mentioned, a ton of prototyping apps are out there, so many that I decided a few months ago to create a small website, Prototyping Tools, that lists all of them that I am aware of. The website provides an overview of which features each tool provides and so should help you choose one.

On the other hand, be aware that operating systems like Android — with its “material design” — are bringing more focus to micro-interactions. Prototyping micro-interactions is the responsibility of many designers today. (A tool like Keynote enables you to prototype these kinds of details with relative ease. Keynote’s Magic Move option comes in handy here.)

Don’t Trust What You See

When designing for the web, it’s possible to watch a design come to life as you work. But with apps, you have to test on a mobile device in order to see how an interface will look (and function). Above all, such testing is imperative for choosing contrast and sizes.
Tools for both Android and iOS will help designers view their work on mobile screens with varying resolutions and densities. For iOS, one of the most well-known options is LiveView. Another, the one I use most, is Skala (available for both iOS and Android), which comes with a desktop version that you can use on a Mac. If you use Sketch, you could also try Sketch Mirror.
Unfortunately, a tool of this kind is not available for Windows and Windows Phone (at least not that I’m aware of).

In my workflow, I always preview a design on my phone to be sure of how it will look before taking the next step. Likewise, I usually start designing for one operating system at a time, and always with the phone close to me most of the time. That way, I can frequently review how my work will appear on that screen, and from the moment the first wireframes are ready, I can be sure that the sizes (of graphics, text and touch targets), contrast and colors will be the ones I’m expecting. If you don’t do this, you might notice it later, when making changes and corrections would require much more time.
Also important, start working with a design document that is the same size (in width and height) as the phone on which you are previewing your work. (Note: If you are designing for different screen sizes, then keep in mind that pixels are no longer the best way to measure sizes, because they change from one OS to another — for example, see the difference between points on iOS and density-independent pixels on Android. This forces you in some way to fully understand the differences in how to design and how to treat images in the current mobile landscape.)
Once your designs have been implemented by the developers, check how the app looks and performs on different phones and OS versions, and not only in the best phone you have next to you.

Design Humbly

App designers must abandon the concept of creating something final. An app is never finished, because it’s a digital product that evolves over time. For this very reason, we can’t approach interface design with a definitive stance. The truth is discovered through usability testing and deciphering what problems users encounter.
Sometimes we designers tend to think that when a user makes a mistake using an app, they are at fault, thus freeing us from all liability. The truth is that we more likely have done something wrong.
Detecting problems (and discovering our mistakes) isn’t a bad thing. Rather, it is an opportunity to learn and correct our work, thereby improving it and building something that’s easier to use. We must approach design with an attitude of humility.
In one company where I worked, every time I told my boss that I had finished a design, he would ask me, “Have you tested it with users already?” I hadn’t always tested it yet, but in the end I incorporated it as a habit, and each time I tested, I went in ready with an open mind, willing to see what problems I would find.
This can only be done if you are mature enough to accept your mistakes and learn from them. It’s difficult, but if you succeed it’s really worth it.

Stay At The Forefront

The amount of information out there makes it nearly impossible to stay current on everything when designing for tomorrow. Nevertheless, as app designers, we must stay engaged and curious about what’s new and what’s to come.
This includes trends in app design, as well as new versions of operating systems. Like it or not, we have to install new operating systems and stay ahead of the curve in order to see where app design is headed.
One good way to get valuable design information is by reading official design guides for the latest OS versions. You can also learn from industry leaders, like Josh Clark and Luke Wroblewski, to name just a couple.
Staying on top of things also means trying tools that simplify design and make our work more efficient as they come onto the market. We can’t be afraid of getting out of our comfort zone and abandoning our old ways of working and designing.

Along these lines, a while ago I finally dared to use Sketch. It was a bit hard at first, because it meant learning something new from scratch, when — I have to admit — I was already comfortable (sort of) using the tools I already knew.
A few months later, I now realize that this software has quite a gentle learning curve. It saves me time and enables me to work in a very agile manner — and all of that has been a reward for being willing to learn something new.

Conclusion

At the end of the day, changing one’s mindset to adapt to apps comes from the inside. No one will force you to do it. If you don’t adjust, you run the risk of becoming little more than an icon factory, just a small step in a larger process of design.
Above all, designing apps requires a new way of thinking. It’s time to leave our web design boxes and to understand smartphones, tablets and even watches as separate and distinct devices. It’s the only way to truly design complete, comprehensive mobile experiences.

Additional Resources

• “7 Lessons Learned From Interviewing 100+ App Developers,” Steve P. Young
• Lean UX: Applying Lean Principles to Improve User Experience, Jeff Gothelf with Josh Seiden
• “Motion UI Design Principles,” Grant Liddall, BeyondKinetic
• Prototyping Tools, Javier Cuello
  This is my personal project where I maintain a list of modern prototyping tools.
• “Designer’s Toolkit: Prototyping Tools,” Emily Schwartzman, Cooper
  A short list maintained and updated by the design agency Cooper
• “Three Stages of Making Wireframes,” Alexander Mescheryakov
• “Designer’s Guide to DPI,” Sebastien Gabriel
• “Mobile Behavior and Design Trends,” User Interface Engineering
  A podcast episode featuring Luke Wroblewski

Browser Input Events: Can We Do Better Than The Click?

Responding to user input is arguably the core of what we do as interface developers. In order to build responsive web products, understanding how touch, mouse, pointer and keyboard actions and the browser work together is key. You have likely experienced the 300-millisecond delay in mobile browsers or wrestled with touchmove versus scrolling.
In this article we will introduce the event cascade and use this knowledge to implement a demo of a tap event that supports the many input methods while not breaking in proxy browsers such as Opera Mini.

Overview

Three primary input methods are used to interact with the web today: digital cursors (mouse), tactile (direct touch or stylus) and keyboards. We have access to these in JavaScript through touch events, mouse events, pointer events and keyboard events. In this article we are primarily concerned with touch- and mouse-based interactions, although some events have standard keyboard-based interactions, such as the click and submit events.
You have very likely already been implementing event handlers for touch and mouse events. There was a time in our not too distant pass when the recommended method was something akin to this:

/** DO NOT EVER DO THIS! */
$('a', ('ontouchstart' in window) ? 'touchend' : 'click', handler);

Microsoft has led the charge to create a better, future-facing event model with the “Pointer Events” specification. Pointer events are an abstract input mechanism that is now a W3C recommendation. Pointer events give the user agent (UA) flexibility to house numerous input mechanisms under one event system. Mouse, touch and stylus are all examples that easily come to mind today, although implementations extending to Myo or Ring are imaginable. While web developers seem to be really excited about this, not all browser engineers have felt the same. Namely, Apple and Google have decided not to implement pointer events at this time.
Google’s decision is not necessarily final, but there is no active work being done on pointer events. Our input and usage of pointer events through polyfills and alternative solutions will be part of the equation that could eventually tip the scale the other way. Apple made its statement against pointer events in 2012, and I am unaware of any more public response from Safari’s engineers.

The Event Cascade

When a user taps an element on a mobile device, the browser fires a slew of events. This action typically fires a series of events such as the following: touchstart → touchend → mouseover → mousemove → mousedown → mouseup → click.
This is due to backwards compatibility with the web. Pointer events take an alternative approach, firing compatibility events inline: mousemove → pointerover → mouseover → pointerdown → mousedown → gotpointercapture → pointerup → mouseup → lostpointercapture → pointerout → mouseout → focus → click.
The event specification allows for UAs to differ in their implementation of compatibility events. Patrick Lauke and Peter-Paul Koch maintain extensive reference material on this topic, which is linked to in the resources section at the bottom of this article.
The following graphics show the event cascade for the following actions:

1. an initial tap on an element,
2. a second tap on an element,
3. tapping off the element.

Please note: This event stack intentionally ignores where focus and blur events fit into this stack.

Applying the Event Cascade

Most websites built today for the desktop web “just work” because of the efforts of browser engineers. Despite the cascade looking gnarly, the conservative approach of building for mouse events as we previously have will generally work.
Of course, there is a catch. The infamous 300-millisecond delay is the most famous, but the interplay between scrolling, touchmove and pointermove events, and browser painting are additional issues. Avoiding the 300-millisecond delay is easy if:

• we optimize only for modern Chrome for Android and desktop, which use heuristics such as <meta name="viewport" content="width=device-width"> to disable the delay;
• we optimize only for iOS, and the user does a clear press, but not a quick tap and not a long tap — just a good, normal, clear press of an element (oh, it also depends on whether it’s in a UIWebView or a WKWebView — read FastClick’s issue on the topic for a good cry).

If our goal is to build web products that compete with native platforms in user experience and polish, then we need to decrease interaction response latency. To accomplish this, we need to be building on the primitive events (down, move and up) and creating our own composite events (click, double-click). Of course, we still need to include fallback handlers for the native events for broad support and accessibility.
Doing this requires no small amount of code or knowledge. To avoid the 300-millisecond (or any length of) delay across browsers, we need to handle the full interaction lifecycle ourselves. For a given {type}down event, we will need to bind all events that will be necessary to complete that action. When the interaction is completed, we will then need to clean up after ourselves by unbinding all but the starting event.
You, the website developer, are the only one to know whether the page should zoom or has another double-tap event it must wait for. If — and only if — you require the callback to be delayed should you allow a delay for the intended action.
In the following link, you will find a small, dependency-free tap demo to illustrate the effort required to create a multi-input, low-latency tap event. Polymer-gestures is a production-ready implementation of the tap and other events. Despite its name, it is not tied to the Polymer library in any way and can easily be used in isolation.
To be clear, implementing this from scratch is a bad idea. The following is for educational purposes only and should not be used in production. Production-ready solutions exist, such as FastClick, polymer-gestures and Hammer.js.

• Demo: The tap event
• Code: taps.js

The Important Bits

Binding your initial event handlers is where it all begins. The following pattern is considered the bulletproof way to handle multi-device input.

/**
 * If there are pointer events, let the platform handle the input 
 * mechanism abstraction. If not, then it’s on you to handle 
 * between mouse and touch events.
 */

if (hasPointer) {
  tappable.addEventListener(POINTER_DOWN, tapStart, false);
  clickable.addEventListener(POINTER_DOWN, clickStart, false);
}

else {
  tappable.addEventListener('mousedown', tapStart, false);
  clickable.addEventListener('mousedown', clickStart, false);

  if (hasTouch) {
    tappable.addEventListener('touchstart', tapStart, false);
    clickable.addEventListener('touchstart', clickStart, false);
  }
}

clickable.addEventListener('click', clickEnd, false);

Binding touch event handlers could compromise rendering performance, even if they don’t do anything. To decrease this impact, binding tracking events in the starting event’s handler is often recommended. Don’t forget to clean up after yourself and unbind the tracking events in your action-completed handlers.

/**
 * On tapStart we want to bind our move and end events to detect 
 * whether this is a “tap” action.
 * @param {Event} event the browser event object
 */

function tapStart(event) {
  // bind tracking events. “bindEventsFor” is a helper that automatically 
  // binds the appropriate pointer, touch or mouse events based on our 
  // current event type. Additionally, it saves the event target to give 
  // us similar behavior to pointer events’ “setPointerCapture” method.

  bindEventsFor(event.type, event.target);
  if (typeof event.setPointerCapture === 'function') {
    event.currentTarget.setPointerCapture(event.pointerId);
  }

  // prevent the cascade
  event.preventDefault();
  
  // start our profiler to track time between events
  set(event, 'tapStart', Date.now());
}

/**
 * tapEnd. Our work here is done. Let’s clean up our tracking events.
 * @param {Element} target the html element
 * @param {Event} event the browser event object
 */

function tapEnd(target, event) {
  unbindEventsFor(event.type, target);
  var _id = idFor(event);
  log('Tap', diff(get(_id, 'tapStart'), Date.now()));
  setTimeout(function() {
    delete events[_id];
  });
}

The rest of the code should be pretty self-explanatory. In truth, it’s a lot of bookkeeping. Implementing custom gestures requires you to work closely with the browser event system. To save yourself the pain and heartache, don’t do this à la carte throughout your code base; rather, build or use a strong abstraction, such as Hammer.js, the Pointer Events jQuery polyfill or polymer-gestures.

Conclusion

Certain events that used to be very clear are now filled with ambiguity. The click event used to mean one thing and one thing only, but touchscreens have complicated it by needing to discern whether the action is a double-click, scroll, event or some other OS-level gesture.
The good news is that we now understand much better the event cascade and the interplay between a user’s action and the browser’s response. By understanding the primitives at work, we are able to make better decisions in our projects, for our users and for the future of the web.
What unexpected problems have you run into when building multi-device websites? What approaches have you taken to solve for the numerous interaction models we have on the web?

Additional Resources

• “Pointer Events Finalized, But Apple’s Lack of Support Still a Deal Breaker,” Peter Bright
• “Getting Touchy: An Introduction to Touch and Pointer Events,” including slides and talk, Patrick E. Lauke
• “Apple’s Web?” by Tim Kadlec
• “Avoiding the 300ms Click Delay, Accessibly,” Tim Kadlec
• “Touch Table,” Peter-Paul Koch
• “Making the Web ‘Just Work’ With Any Input: Mouse, Touch, and Pointer Events,” Jacob Rossi
• FastClick library
• Hammer.js
• polymer-gestures
• PointerEvents jQuery polyfill
• “Implement Custom Gestures,” Google Developers

Stop Wasting Users’ Time

Our users are precious about their time and we must stop wasting it. On each project ask two questions: “Am I saving myself time at the expense of the user?” and “How can I save the user time here?” What is the single most precious commodity in Western society? Money? Status? I would argue it is time.
We are protective of our time, and with good reason. There are so many demands on it. We have so much to do. So much pressure. People hate to have their time wasted, especially online. We spend so much of our time online these days, and every interaction demands a slice of our time. One minor inconvenience on a website might not be much, but, accumulated, it is death by a thousand cuts.
Steve Jobs claimed that improving the boot time on the Macintosh would save lives. A 10-second improvement added up to many lifetimes over the millions of users booting their computers multiple times a day.
Millions of people might not use your website, but millions do use the Web as a whole. Together, we are stealing people’s lives through badly designed interactions. When I work on a website, one question is front and center in my mind:

“Am I saving myself time at the expense of the user?”

That is the heart of the problem. In our desire to meet deadlines and stay on budget, we often save ourselves time by taking shortcuts via our users’ time. Let’s explore some examples of what I mean.

Taking Time To Improve Performance

The most obvious example of wasting users’ time is website performance. This is what Jobs was getting at with boot times. If our websites are slow, then we’ll waste our users’ valuable time and start to irritate them. One more cut, so to speak.
The problem is that improving performance is hard. We became lazy as broadband became widespread. We cut corners in image optimization, HTTP requests and JavaScript libraries. Now, users pay the price when they try to access our websites on slow mobile devices over cellular networks.
Making our websites faster takes time and effort, but why should users suffer for our problems? On the subject of making our problems the users’ problem, let’s take a moment to talk about CAPTCHA.

CAPTCHA: The Ultimate Time-Waster

CAPTCHA is the ultimate example of unloading our problems onto users. How many millions of hours have users wasted filling in CAPTCHA forms? Hours wasted because we haven’t addressed the problem of bots.
Just to be clear, I am not just talking about traditional CAPTCHA either. I am talking about any system that forces the user to prove they are human. Why should they have to prove anything? Once again, another inconvenience, another drain on their precious time.
We could solve this problem if we put the time into it. The honeytrap technique helps. There are also server-side solutions for filtering out automated requests. The problem is that throwing a CAPTCHA on a website is easier.
Not that CAPTCHA is the only way that we waste the user’s time when completing forms.

Don’t Make Users Correct “Their” Mistakes In Forms

Sometimes we even waste the user’s time when we are trying to help them. Take postal-code lookup. I have been on websites that try to save me time by asking me to enter my postal code so that it can auto-populate my address. A great idea to save me some time — great if it works, that is.
The problem is that some lookup scripts require the postal code to have no spaces. Instead of the developer configuring the script to remove any spaces, they just return an error, and the user has to correct “their” mistake. Why should the user have to enter the data in a particular way? Why waste their time by requiring them to re-enter their postal code? This doesn’t just apply to postal codes either. Telephone numbers and email addresses come with similar problems.
We also need to better help mobile users interact with forms. Forms are particularly painful on touchscreens, so we need to explore alternative form controls, such as sliders and the credit-card input system in Square’s mobile app.
Then, there are passwords.

Why Are Passwords So Complicated?

Why do we waste so much of the users’ time with creating passwords? Every website I visit these days seems to have ever more complex requirements for my password. Security is important, but can’t we come up with a better solution than an arcane mix of uppercase, numbers and symbols?
Why couldn’t we ask users to type in a long phrase instead of a single word? Why can’t my password be, “This is my password and I defy anyone to guess it”? The length would make it secure, and remembering and typing it would be much easier. If your system doesn’t like the spaces, strip them out. You could even provide an option for people to see what they’re typing.
If you can’t do that, at least provide instructions when the user tries to log in. Remind them of whether your website wants uppercase or a certain number of characters. That would at least help them remember their password for your website.
The important thing is to recognize that people have to log in all the time. The task demands extra attention so that it is as painless as possible.

Pay Special Attention To Repetitive Tasks

We should ask ourselves not only whether we are unloading our problems onto users, but also how we can save our users time.
Take those common tasks that users do on our websites time and again. How can we shave a quarter of a second off of those tasks? What about search? If the user enters a search term on your website, will hitting the “Return” key submit the query? They shouldn’t have to click the “Search” button.
Drop-down menus are another good example. Navigating country-pickers can be painful. Could we display countries differently, or make the most common countries faster to access? In fact, so much could be done to improve country-pickers if we just take the time.
For that matter, a more robust solution to “Remember me” functionality would be nice, so that users are, in fact, remembered!
I am aware that this post might sound like a rant against developers. It is not. It is a problem faced by all Web professionals. Designers need to pay close attention to the details of their designs. Web managers need to ensure that the budget exists to refine their user interfaces. And content creators need to optimize their content for fast consumption.

Help Users Process Our Content Faster

We waste so much of our users’ time with verbose, poorly written and dense copy, making it hard for them to find the piece of information they need. The real shame is that we could do so much to help. For a start, we could give the user a sense of approximately how long a page will take to read. I offer this functionality on my personal blog, and it is the feature most commented on. Users love knowing how much of their time a post will take up.
We can also make our content a lot more scannable, with better use of headings, pullout quotes and lists. Finally, we can take a leaf out of Jakob Nielsen’s website. At the beginning of each post, he provides a quick summary of the page.

The Tip Of The Iceberg

We could do so much more in all aspects of Web design to save users’ time. From information architecture to website analytics, we waste too much of it. Sometimes we even know we are doing it! We need to be forever vigilant and always ask ourselves:

“How can I save the user time in this situation?”

What are your thoughts on this topic? Please share your experiences and opinions with us, and join in the discussion in the comments section below.
(al, il)

What Leap Motion And Google Glass Mean For Future User Experience

With the Leap Motion controller being released on July 22nd and the Google Glass Explorer program already live, it is obvious that our reliance on the mouse or even the monitor to interact with the Web will eventually become obsolete.
The above statement seems like a given, considering that technology moves at such a rapid pace. Yet in 40 years of personal computing, our methods of controlling our machines haven’t evolved beyond using a mouse, keyboard and perhaps a stylus. Only in the last six years have we seen mainstream adoption of touchscreens.
Given that emerging control devices such as the Leap Controller are enabling us to interact with near pixel-perfect accuracy in 3-D space, our computers will be less like dynamic pages of a magazine and more like windows to another world. To make sure we’re on the same page, please take a minute to check out what the Leap Motion controller can do:

“Introducing the Leap Motion”
Thanks to monitors becoming portable with Google Glass (and the competitors that are sure to follow), it’s easy to see that the virtual world will no longer be bound to flat two-dimensional surfaces.
In this article, we’ll travel five to ten years into the future and explore a world where Google Glass, Leap Motion and a few other technologies are as much a part of our daily lives as our smartphones and desktops are now. We’ll be discussing a new paradigm of human-computer interface.
The goal of this piece is to start a discussion with forward-thinking user experience designers, and to explore what’s possible when the mainstream starts to interact with computers in 3-D space.

Setting The Stage: A Few Things To Consider

Prior to the introduction of the iPhone in 2007, many considered the smartphone to be for techies and business folk. But in 2013, you’d be hard pressed to find someone in the developed world who isn’t checking their email or tweeting at random times.
So, it’s understandable to think that a conversation about motion control, 3-D interaction and portable monitors is premature. But if the mobile revolution has taught us anything, it’s that people crave connection without being tethered to a stationary device.
To really understand how user experience (UX) will change, we first have to consider the possibility that social and utilitarian UX will be taking place in different environments. In the future, people will use the desktop primarily for utilitarian purposes, while “social” UX will happen on a virtual layer, overlaying the real world (thanks to Glass). Early indicators of this are that Facebook anticipates its mobile growth to outpace its PC growth and that nearly one-seventh of the world’s population own smartphones.
The only barrier right now is that we lack the technology to truly merge the real and virtual worlds. But I’m getting ahead of myself. Let’s start with something more familiar.

The Desktop

Right now, UX on the desktop cannot be truly immersive. Every interaction requires physically dragging a hunk of plastic across a flat surface, which approximates a position on screen. While this is accepted as commonplace, it’s quite unnatural. The desktop is the only environment where you interact with one pixel at a time.
Sure, you could create the illusion of three dimensions with drop shadows and parallax effects, but that doesn’t change the fact that the user may interact with only one portion of the screen at a time.
This is why the Leap Motion controller is revolutionary. It allows you to interact with the virtual environment using all 10 fingers and real-world tools in 3-D space. It is as important to computing as analog sticks were to video games.

The Shift In The Way We Interact With Machines

To wrap our heads around just how game-changing this will be, let’s go back to basics. One basic UX and artificial intelligence test for any new platform is a simple game of chess.

(Image: Wikimedia Commons)
In the game of chess below, thanks to motion controllers and webcams, you’ll be able to “reach in” and grab a piece, as you watch your friend stress over which move to make next.

Now you can watch your opponent sweat.
In a game of The Sims, you’ll be able to rearrange furniture by moving it with your hands. CAD designers will use their hands to “physically” manipulate components (and then send their design to the 3-D printer they bought from Staples for prototyping.)
While the lack of tactile feedback might deter mainstream adoption early on, research into haptics is already enabling developers to simulate physical feedback in the real world to correspond with the actions of a user’s virtual counterpart. Keep this in mind as you continue reading.
Over time, this level of 3-D interactivity will fundamentally change the way we use our desktops and laptops altogether.
Think about it: The desktop is a perfect, quiet, isolated place to do more involved work like writing, photo editing or “hands-on” training to learn something new. However, a 3-D experience like those mentioned above doesn’t make sense for social interactions such as on Facebook or even reading the news, which are more becoming of mobile.
With immersive, interactive experiences being available primarily via the desktop, it’s hard to imagine users wanting these two experiences to share the same screen.
So, what would a typical desktop experience look like?

Imagine A Cooking Website For People Who Can’t Cook

With this cooking website for people who can’t cook, we’re not just talking about video tutorials or recipes with unsympathetic instructions, but rather immersive simulations in which an instructor leads you through making a virtual meal from prep to presentation.
Interactions in this environment would be so natural that the real design challenge is to put the user in a kitchen that’s believable as their own.
You wouldn’t click and drag the icon that represents sugar; you would reach out with your virtual five-fingered hand and grab the life-sized “box” of Domino-branded sugar. You wouldn’t click to grease the pan; you’d mimic pushing the aerosol nozzle of a bottle of Pam.
The Tokyo Institute of Technology has already built such a simulation in the real world. So, transferring the experience to the desktop is only a matter of time.

“Cooking simulator will help you cook a perfect steak every time”
UX on the future desktop will be about simulating physics and creating realistic environments, as well as tracking head, body and eyes to create intuitive 3-D interfaces, based on HTML5 and WebGL.
Aside from the obvious hands-on applications, such as CAD and art programs, the technology will shift the paradigm of UX and user interface (UI) design in ways that are currently difficult to fathom.
The problem right now is that we currently lack a set of clearly defined 3-D gestures to interact with a 3-D UI. Designing UIs will be hard without knowing what our bodies will have to do to interact.
The closest we have right now to defined gestures are those created by Kinect hackers and John Underkoffler of Oblong Technology (the team behind Minority Report’s UI).
In his TED talk from 2010, Underkoffler demonstrates probably the most advanced example of 3-D computer interaction that you’re going to see for a while. If you’ve got 15 minutes to spare, I highly recommend watching it:

John Underkoffler’s talk “Pointing to the Future of UI”
Now, before you start arguing, “Minority Report isn’t practical — humans aren’t designed for that!” consider two things:

1. We won’t likely be interacting with 60-inch room-wrapping screens the way Tom Cruise does in Minority Report; therefore, our gestures won’t need to be nearly as big.
2. The human body rapidly adapts to its environment. Between the years 2000 and 2010, a period when home computers really went mainstream, reports of Carpal Tunnel Syndrome dropped by nearly 8%.

(Image: Minnesota Department of Health)
However, because the Leap Motion controller is less than $80 and will be available at Best Buy, this technology isn’t just hypothetical, sitting in a lab somewhere, with a bunch of geeks saying “Wouldn’t it be cool if…”
It’s real and it’s cheap, which really means we’re about to enter the Wild West of true 3-D design.

Social Gets Back To The Real World

So, where does that leave social UX? Enter Glass.
It’s easy to think that head-mounted augmented reality (AR) displays, such as Google Glass, will not be adopted by the public, and in 2013 that might be true.
But remember that we resisted the telephone when it came out, for many of the same privacy concerns. The same goes for mobile phones and for smartphones around 2007.
So, while first-generation Glass won’t likely be met with widespread adoption, it’s the introduction of a new phase. ABI Research predicts that the wearable device market will exceed 485 million annual shipments by 2018.
According to Steve Lee, Glass’ product director, the goal is to “allow people to have more human interactions” and to “get technology out of the way.”
First-generation Glass performs Google searches, tells time, gives turn-by-turn directions, reports the weather, snaps pictures, records video and does Hangouts — which are many of the reasons why our phones are in front of our faces now.
Moving these interactions to a heads-up display, while moving important and more heavy-duty social interactions to a wrist-mounted display, like the Pebble smartwatch, eliminates the phone entirely and enables you to truly see what’s in front of you.

(Image: Pebble)
Now, consider the possibility that something like the Leap Motion controller could become small enough to integrate into a wrist-mounted smartwatch. This, combined with a head-mounted display, would essentially give us the ability to create an interactive virtual layer that overlays the real world.
Add haptic wristband technology and a Bluetooth connection to the smartwatch, and you’ll be able to “feel” virtual objects as you physically manipulate them in both the real world and on the desktop. While this might still sound like science fiction, with Glass reportedly to be priced between $299 and $499 and Leap Motion at $80 and Pebble being $150, widespread affordability of these technologies isn’t entirely impossible.

Social UX In The Future: A Use Case

Picture yourself walking out of the mall, and your close friend Jon updates his status. A red icon appears in the top right of your field of vision. Your watch displays Jon’s avatar, which says, “Sooo hungry right now.”
You say, “OK, Glass. Update status: How about lunch? What do you want?” and keep walking.
“Tacos.”
You say, “OK, Glass. Where can I get good Mexican food?” 40 friends have favorably rated Rosa’s Cafe. Would you like directions? “Yes.” The navigation starts, and you’re en route.
You reach the cafe, but Jon is 10 minutes away. Would you like an audiobook while you wait? “No, play music.” A smart playlist compiles exactly 10 minutes of music that perfectly fits your mood.
“OK, Glass. Play Angry Birds 4.”
Across the table, 3-D versions of the little green piggies and their towers materialize.
In front of you are a red bird, a yellow bird, two blue birds and a slingshot. The red bird jumps up, you pull back on the slingshot, the trajectory beam shows you a path across the table, you let go and knock down a row of bad piggies.
Suddenly, an idea comes to you. “OK, Glass. Switch to Evernote.”
A piece of paper and a pen are projected onto the table in front of you, and a bulletin board appears to the left.
You pick up the AR pen, jot down your note, move the paper to the appropriate bulletin, and return to Angry Birds.
You could make your game visible to other Glass wearers. That way, others could play with you — or, at the very least, would know you’re not some crazy person pretending to do… whatever you’re doing across the table.
When Jon arrives, notifications are disabled. You push the menu icon on the table and select your meal. Your meal arrives; you take photos of your food; eat; publish to Instagram 7.
Before you leave, the restaurant gives a polite notification, letting you know that a coupon for 10% off will be sent to your phone if you write a review.

How Wearable Technology Interacts With Desktops

Later, having finished the cooking tutorial on the desktop, you decide it’s time to make the meal for real. You put on Glass and go to the store. The headset guides you directly to the brands that were advertised “in game.” After picking out your ingredients, you receive a notification that a manufacturer’s coupon has been sent to your phone and can be used at the check-out.
When you get home, you lay a carrot on the cutting board and an overlay projects guidelines on where to cut. You lay out the meat, and a POW graphic is overlaid, showing you where to hit for optimal tenderness:

You put the meat in the oven; Glass starts the timer. You put the veggies in the pan; Glass overlays a pattern to show where and when to stir.
While you were at the store, Glass helped you to pick out the perfect bottle of wine to pair with your meal (based on reviews, of course). So, you pour yourself a glass and relax while you wait for the timer to go off.
In the future, augmented real-world UX experiences will be turned into real business. The more you enhance real life, the more successful your business will be. After all, is it really difficult to imagine this cooking experience being turned into a game?

What Can We Do About This Today?

If you’re the kind of UI designer who seeks to push boundaries, then the best thing you can do right now is think. Because the technology isn’t 100% available, the best you can do is open your imagination to what will be possible when the average person has evolved beyond the keyboard and mouse.
Draw inspiration from websites and software that simulate depth to create dynamic, layered experiences that can be easily operated without a mouse. The website of agency Black Negative is a good example of future-inspired “flat” interaction. It’s easy to imagine interacting with this website without needing a mouse. The new Myspace is another.
To go really deep, look at the different Chrome Experiments, and find a skilled HTML5 and WebGL developer to discuss what’s in store for the future. The software and interactions that come from your mind will determine whether these technologies will be useful.

Conclusion

While everything I’ve talked about here is conceptual, I’m curious to hear what you think about how (or even if) these devices will affect UIs. I’d also love to hear your vision of future UIs.
To get started, let me ask you two questions:

1. How will the ability to reach into the screen and interact with the virtual world shape our expectations of computing?
2. How will untethering content from flat surfaces fundamentally change the medium?

I look forward to your feedback. Please share this article if you’ve enjoyed this trip into the future.

10 Signs of a User-Focused Design

When building a new website or re-designing an existing one, web designers are faced with all kinds of significant challenges, including keeping the client happy, creating a site that gets results, and catering to users at the same time. During the design and development process it’s easy to get distracted and overlook the significance of usability. Since we’re the ones creating the site, we know it much better than any visitor will, and sometimes it’s easy to forget about the impact on users.
Building a truly usable website requires the right kind of priorities and testing. With the emphasis only on creating a great-looking site or one that is effectively monetized, usability can easily wind up taking a back seat. Throughout the design and development process, make an effort to view the site as if you were a first-time visitor and see if it changes your opinion on the strengths and weaknesses of the site.
In order to improve the usability of your work, focus on specific aspects of the design that will influence users the most, and make sure that the site holds up in these areas. Here is a look at 10 things to consider.

1. Clear and Logical Navigation

When usability is mentioned, navigation is usually part of the discussion. Without the ability to easily move through the website, visitors are unlikely to find what they want and they’re more likely to leave quickly without really getting to know more about the website or the company. Clear navigation on the other hand will make it simple for visitors to maneuver through the site as they choose.
The navigational approach will greatly depend upon the type of site. Blogs, for example, tend to have navigation that is not quite like any other type of website. The typical blog visitor knows that post titles on the front page are links to full posts, that category links are usually in the sidebar and that date-based archives will allow them to browse through older posts. E-commerce websites will have a different approach entirely, which will focus on helping visitors to find the right products. All of this comes back to usability. When visitors arrive at a specific type of site, they’ll expect certain things, and navigation is a big part of this issue.

2. Clear Communication

Have you ever been to a website where you have spent a few minutes looking at several pages before you could even determine the site’s purpose? I sometimes come across sites that do a poor job of communicating the products or services provided, and I usually wind up leaving if I can’t find something that’s understandable. One of the key issues involved with usability is communication.
There is so much that is involved in communication online. Headlines, taglines, images, colors, design styles, they all say something to visitors. An effective and usable website will clearly communicate it’s message to visitors, and ideally that message should be consistent throughout the website.

3. Targeted, Focused Content

The actual content of the website is often overlooked in discussions on usability, but it will have a strong impact, whether positive or negative, on the experience of visitors. A good website will have a specific audience that it is targeting, and content that is catered to suit this audience. Whether the content consists of blog posts, products for sale, service descriptions, tutorials, videos, or any other type of content, it should be created with the audience in mind.
A website with content that isn’t focused on a particular audience will lack a purpose and will be of little value to visitors. Remember that visitors are coming to a website to find something of interest to them. Hopefully, many of the visitors share similar interests (which will be the case if the audience is appropriately targeted), which makes it fairly simple to provide the right kind of content, you just have to keep the audience in mind.

4. Style that Suits the Audience

Every website will have some people that love the design and others that hate it. No design will ever please everyone, but it should be suited to fit the interests of the target audience. Although no two people will ever have exactly the same opinions on different styles of design, generally people who have similar interests or fit into different demographics will have similar likes and dislikes.
The style that is used for a site’s design should always be influenced by the target audience. For example, grungy websites are often used for extreme sports and for rock bands, two types of sites that are frequently visited by young people that will appreciate the grunge style. You won’t see an effective website for a retirement home with the same style of design. Similar differences in style can be seen in the likes of various cultures.
The look and feel of a website will have an instant impact on users. It’s possible that the minute they first visit a site they’ll feel more comfortable based strictly on the style of design. When a website uses a style that’s embraced by the visitor, it’s a subtle message telling them that they belong at this website and that they’ll fit in.

5. Lack of Interference

A user-focused website will not include such interferences as flashy or annoying advertisements, intrusively located ads, pop ups or pop unders, or anything else that can damage the ability of the user to read and interact with the site. Yes, some of these items are effectively used for legitimate purposes on many websites, but it’s my opinion that they help to accomplish the goals of the website owner and are not used for the purpose of improving the user experience.
If your priority is to create a site places the user’s needs and interests above anything else, interferences should be minimized or eliminated. In the real world, websites usually have other goals, such as making money, that don’t always match up perfectly with the requirements for maximum usability. In these cases you’ll have to use your best judgment, but be aware that any type of interference can take away from the user’s experience.

6 – Ease of Contact

A percentage of visitors to a website will want to contact the owner for one reason or another. Maybe it is to inquire about services that are offered, or maybe it’s about an issue they’re having with the site. Whatever the case may be, some people will want to contact you, and the ease with which they can do so will have a strong influence on their experience on the site.
Not being able to find a way to contact the owner can be extremely frustrating, and it sends a clear message to the visitor that their feedback is not a priority. A few times when I first started blogging I tried to track down other bloggers that were stealing my content. In most cases these people had no contact form or email address listed on their sites. Often times they had very little information about themselves or about their blog. What they did typically have was AdSense. The lack of a contact form communicated to me that their site was built to make money from AdSense and they didn’t have any interest in other bloggers contacting them about stolen content.
Since visitors will be entering your site on various pages, it’s a good practice to include a link to contact information on every page. Think about where you typically will go on a website to find this information. Many websites include a link to a contact page in their main navigation menu and/or in the footer. If the site is large and a link to the contact page isn’t feasible in the primary navigation, most visitors will go to an About page if they’re trying to make contact.

7 – Readability

The user experience is greatly impacted by the readability of a site’s content, particularly on blogs. Two different websites can provide the same exact content, but if one is formatted well for readability and the other one is not, there is a huge difference between the two for visitors.
Most visitors will have no interest in reading word-for-word all of the content on a page. Being able to quickly scan a page or an article for relevant content is very important. You can make this easier on visitors by using whitespace, bold text, headers and sub-headers, lists, etc.

8 – Not Focused on the Website Owner’s Wants

A user-focused website is, by definition, giving priority to the wants and needs of the users. This means that the priorities of the owner will be secondary. Of course, it’s still important to achieve the purposes for which the website exists, such as selling products, but it needs to be done in a way that focuses on the user. Rather than thinking about a website being created around a product or service, think of it as being created around meeting the needs of the user and allowing the product or service to be a part of that solution.
A website’s visitors will be able to tell when they are not being considered by the owner or developer and when the only priority is to get something from them. Obviously, this is not the way to enhance the user experience and make visitors feel at home on the site. In most cases it’s possible to accomplish the goals and objectives of a website and the company as the same time as meeting the needs of users.

9 – Anticipation of User Issues/Questions

Regardless of how well the designer creates a site to meet the needs of users, some visitors are bound to have questions or issues while on the site. The development of a user-focused website should include consideration of likely issues so that they can be addressed easily. Although you will never be able to avoid all of these situations, minimizing the number of unresolved issues will go a long way in improving the user experience.
If you’re seeing signs of problems, such as an unusually high number of questions through a contact form or a very high exit rate from a particular page, steps should be taken to address the issues that exist and make the site more user-friendly.

10 – Accessibility

The topic of user-focus can never be complete without mentioning accessibility. If a visitor is unable to access the site, it is seen as the ultimate sign of a lack of user-focus. In truth, most websites are not 100% accessible. What’s most important is that your site is built to be accessible to your target audience. For example, a slow-loading, flash-based site with lots of elements to hog bandwidth may not be usable to absolutely every visitor, but they can still be effectively used in some cases where the target audience will have access to the site and all of its features.
The issue of accessibility is obviously too big to cover in depth in this post, but it is extremely significant in the context of user focus. Ideally, a site will be accessible and usable for all visitors, but you certainly want to make sure that it’s at least catering to the target audience.

What’s Your Opinion?

On the subject of user-focused websites, what do you look for as signs of designer and website owner’s priorities? As a designer, what are your goals for creating a website that meets the needs and desires of users?

Creating An Adaptive System To Enhance UX

In computer science, the term “adaptive system” refers to a process in which an interactive system adapts its behavior to individual users based on information acquired about its user(s), the context of use and its environment. Although adaptive systems have been long-discussed in academia and have been an aspiration for computer scientists and researchers, there has never been a better time than today to realize the potential of what future interaction with computer systems will be like.
The abilities of today’s network information technologies to create rich, immersive personalized experiences to track interactions and aggregate and analyze them in real time, together with the data collected by the sensors we carry in our smart devices, provides us an opportunity like never before to design adaptivity in order to ultimately offer a better user experience that is both unobtrusive and transparent.
This article will cover the fundamental concepts for utilizing smart device technologies and sensor data in order to understand context and introduce “adaptive thinking” into the UX professional’s toolset. I will demonstrate the importance of context when designing adaptive experiences, give ideas on how to design adaptive systems, and perhaps inspire designers to consider how smart devices and context aware applications can enhance the user experience with adaptivity.

Examples Of Adaptive Systems

An early example of an adaptive feature can be found in GPS navigational devices. Using one of these devices, a user is able to easily locate and navigate to any location they can drive to. When the sun sets or while driving through a tunnel, the system automatically changes the interface color to a dark “night mode” so as not to blind the driver with a bright light from the device. The system knows the user’s exact location and the position of the sun, and by understanding these two factors, the system maintains a safe driving environment by adapting to the user’s needs.

The day and night interfaces in the GARMIN Zumo 660 adapt the interface color so the user isn’t blinded with a bright light.
Adaptive design is about listening to the environment and learning user patterns. Combining smart device sensor data, network connectivity and analysis of user behavior is the secret sauce behind creating an adaptive experience. By combining these capabilities, we not only understand the context of use, we can also anticipate what the user needs at a particular moment.
Google Now is an interesting example of an adaptive application that gives users answer to questions they’ve thought rather than typed. Through a series of smart cards that appear throughout the day on the user’s mobile phone, Google Now tells you the day’s weather before you start your day, how much traffic to expect before you leave for work, when the next train will arrive as you’re standing on the platform or your favorite team’s score while they’re playing. It does this by recording and analyzing your preferences while you’re using your phone. For example, updates on your favorite sports team are based on your Web browsing and search history. And by analyzing your current location, previous locations and Web history, Google Now presents a card with traffic conditions on route to your next likely destination.
As UX professionals, we understand that some mobile users do not like to use the virtual keyboard and we try to avoid that necessity as much as possible. By utilizing the user’s personal behavior as a sensor together with smart device capabilities and enabling voice commands (similar to iOS’s Siri), Google Now creates an adaptive experience that helps users avoid using the virtual keyboard, thus further adapting to the mobile user’s needs and helping users quickly get the information they require on the go.
Adaptive systems are not only limited to mobile devices. Ubiquitous computing (ubicomp) is the idea of being surrounded by smart devices and networked digital objects that are carefully tuned to offer us unobtrusive assistance as we navigate through our work and personal lives. Similarly, ambient intelligence (AmI) refers to digital environments that are sensitive and responsive to the presence of people.

Nest uses sensors to adapt the temperature to activity in the home.
Nest, The Learning Thermostat, is a great example of an adaptive system integrated to home environments. Using a variety of sensors for temperature, humidity, touch, near-field activity, far-field activity and even ambient light, it can detect whether there are people home and how active the home is at any time. By adjusting the temperature to adapt to this information, it can automatically cut up to 20% off a home’s heating and cooling bills.
When no one is around, Nest learns to turn the heat down. When you come home from work, it knows that the heat should go back up. After the first few weeks, it learns when you come home from work and can turn the heat up before you arrive so that you come home to a warm house.
In 1991 Mark Weiser, widely considered to be the father of ubiquitous computing, wrote:

“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.”

Nest is a great example of ubicomp and how technology can disappear into our surroundings until only the user interface remains perceivable to users.
These devices create contexts of sensor and user data that provide a superior user experience by anticipating what the user might need before the need is expressed. This is the future of UX design.

Adaptive Thinking

In contrast to traditional desktop systems, mobile devices are normally used in many different situations. However, mobile applications nowadays do not often take advantage of information about the context of their use, and hence are only usable for very specific purposes. For example, an application with city maps for local businesses can be used in different contexts: walking through town or at home, and with or without network connectivity.
Today’s users can customize their device’s system through preferences and settings, and by choosing what applications work best for their needs. Even after the implementation of user-centered design processes that assure a certain degree of user acceptance and yield a richer understanding of the context, it is impossible to anticipate the requirements of all users and map those requirements to a single best or optimal system configuration.
“Adaptive thinking” is a mindset that provides the tools necessary to significantly improve the user experience and enhance the intended purpose of the product by utilizing the technology that is readily available in every pocket. It is about learning the environment and the user and adapting to their current needs and situation. Therefore, designers should first design for the context of use and then design the set of functions that are triggered in relevant situations.
Here is an instructive case where adaptive thinking was used to create a mobile application for a bike sharing program. Bicycle sharing systems, also known as bike rental, are becoming more and more common in major cities around the world. Bicycle sharing helps reduce traffic congestion and air pollution, and encourages local residents to maintain a healthy lifestyle.
A user who wants to rent a bike can use a mobile application to look for the nearest bike rental station that has bikes available to rent. If the user is unfamiliar with the city, they can use the application to get directions to the rental station; this is the core functionality of the application.
An adaptive system will realize when the user has arrived at the bike rental station and automatically offer additional options, i.e., adapt to the current situation. For example, it may offer the user a quick way to rent a bike, a feature that was not available in the application before arriving at the rental station. During the rental period, the system will anticipate the user’s needs and offer nearby bike rental stations with available parking spots where the bike can be returned, and show the user the current balance for the rental time.

A bicycle sharing application can adapt to show the user different options depending on location, and whether the user is currently renting a bike.
By using the assisted GPS device capabilities, using the network connectivity and understanding the user’s story at any given time through the product lifecycle, adaptive design will provide users of the mobile application a reliable extension to the bike rental program.

Adaptive and Responsive Design

An adaptive system is one that adapts automatically to its users according to changing conditions. Responsive design (or adaptive layout) is a subset of adaptive design, an approach to Web design in which a website is crafted to provide an optimal viewing experience across a wide range of devices. In my UX magazine article The Multiscreen Ecosystem, I discuss how responsive websites can also be adaptive by understanding the context of using a mobile device and by designing contextual paths.

Context For Adaptivity

I quote below from the 2007 book The Adaptive Web, which talks about the importance of context for adaptive mobile guides. It explains adaptivity in the scope of mobile systems as context-aware computing, i.e., the ability to use information in the current context to adapt the user interaction and the presentation of information to the current situations of the user.

“Understanding the context is an important prerequisite for the adaption process. Context is not just the location, but encompasses also information like the ambient noise or lighting level, the network connectivity or bandwidth, and even the social circumstances of the user. Furthermore, systems have to anticipate the user’s goals and intentions, which might be inferred from their actions or from physiological sensors and appropriate environmental sensors (e.g. light, pressure and noise sensors).
One prerequisite for adaptive systems is the proper assessment of the user’s situation. For this purpose, systems need to rely on a representation of relevant situations. Depending on the supported task, situations can be characterized by many different attributes. Therefore, designers of suitable adaptation for mobile devices need to look at a variety of spatial, temporal, physical and activity related attributes to provide effective assistance.
For example, a mobile application that assists users in a shop needs to know about the current spatial environment of the users (e.g. which products are nearby), the temporal constraints of the user (e.g. how much time is available for shopping), the general interests of the users and their preferences (e.g. if the user prefers red or white wine with tuna), details about the shopping task itself (e.g. which items are on the shopping list and for which purpose the products are needed) and maybe even about the physiological and the emotional state of users (e.g. whether users are enjoying the shopping or not).”

That said, understanding the locational context and the user story is now easier than ever before. We can take advantage of the fact that we carry our phones wherever we go. A smartphone is packed with technology and with information about the user that designers can use to understand context. The highly sophisticated advanced technology in a user’s pocket not only allows designers to analyze if the user is walking, standing or in a loud or quiet environment, but also can help designers understand the precise location of a person within a department store, such as a specific aisle.

An application can analyze the user’s precise location within a store to provide information that is adapted to the current content.
AislePhone, an Israeli start-up currently in the beta stage, is developing a platform for precise in-store positioning that can determine the exact position of a person down to the specific aisle. With this technology, shopping with your mobile phone in hand will be a common experience, as mobile apps for supermarkets and other large retail stores will use locational and user data to enhance the shopping experience, much like a personal shopping assistant in your pocket.
Google Indoor Maps allows users to view and navigate floor plans of several kinds of commercial locations such as airports, department stores or malls, all within Google Maps.
This technology not only knows your indoor location, but also what floor you’re on in a building. Depending on what data is available, the map shows notable places in the building you’re currently in, such as stores, restrooms or the nearest food court.
With this type of technology, “you are here” directory maps will no longer be needed in malls or department stores. You will be able to determine your location and orient yourself using a smartphone, and this experience will adapt to your specific needs. For example, apps will offer you relevant discounts as you walk through the mall or highlight shops based on your gender and age.

Designing An Adaptive System

Adaptive design integrates both subtle and obvious features. Often, adaptive qualities can be very subtle and unobtrusive: sometimes a seemingly small adaptive feature can greatly improve the overall experience. For example, did you ever notice that Google Search can read your mind? When you start typing, Google Instant, using autocomplete, knows what you’re thinking even when you enter only three letters of a search term. It does this because Google Search considers and records all search queries within a session in order to have a better understanding of the user’s intent.
When a user searches for “The Beatles,” Google understands this as part of a research session and will help you quickly discover Ringo Starr or Paul McCartney as you enter the first three letters of their name; it understands the context of your search and compares it with other similar popular relevant results.

Google Instant understands the context of your search.
Another example of a subtle feature that helps enhance the user experience is a testing system for students that adjusts the difficulty of test questions according to whether prior questions were answered correctly. Or a music discovery application that looks into your current play list and adapts to your taste, helping you discover additional music you may like.
Although the experience should always be unobtrusive, adaptive interfaces need to be obvious so users understand the context for the adaptation and always feel in control. For a better experience, applications should also allow users to manage adaptive features. For example, if at nighttime the interface changes to a darker night mode (like in navigational devices), the user should always be able to change it back manually. Or, if entering a shopping mall triggers a different experience, the user must understand the context for this adaptivity and want to embrace the added functionality.
Charles Darwin wrote:

“It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change.”

As human beings, we adapt to our surroundings naturally; it is the key to our survival. As designers, we can use this inherent ability and our physical senses and the powers of the brain to analyze and design what we would do in adaptable situations. For example, to communicate in a loud environment, we adapt by raising up our voice up to be heard. Similarly, an adaptive system will raise a device’s volume.
In an even louder environment, we use hand gestures to get attention and focus our eyes on the other person’s mouth to try to read their lips. However, unlike computers that can process multiple layers of data, human beings have limited sensory resources and a limited cognitive workload.
In today’s world, a person carries in one pocket more advanced technologies than ever before possible. An intelligent device like a smartphone is embedded with highly sophisticated sensors. These sensors, together with advanced computing power and network connectivity, can help us analyze and understand the context of use. The smart device’s ability to analyze the context of use in real time, together with understanding the user’s story, allows opportunities to provide an even greater user experience by adapting to the user needs.
I will illustrate some of the key points in using these technologies.

Analyzing User Behavior

Similar to the Google Now example, analyzing user behavior and the user’s interaction with the digital world can yield a great understanding of the user’s context. Analyzing the user’s search patterns or what applications they download can tell us about their preferences and hobbies. Tracking current location and location history can give us the user’s surroundings and the physical boundaries of their life, so we can understand what subway station they take to work or where they like to eat their lunch. Note that when this is done without the knowledge of users, it may be considered a breach of browser security and illegal in many countries.
Here is a practical example of how analyzing the user’s behavior could help in creating an adaptive system. In the now famous Google Glasses video, we follow the user throughout his morning as he eats his breakfast and then leaves his house heading for the subway. Upon arriving at the subway, he receives a message that subway service is suspended and is offered a walking route. As useful as this may be, a true adaptive system will analyze the user behavior as the user gets up and will warn the user ahead of time that the subway service is suspended.

Google Glasses uses information about the user’s location to provide relevant information.
Understanding the user’s behavior (whether he takes a subway or walks to work) and connecting it with available information online allows us to understand and adapt to his needs. Most times, using one data source is not enough; combining the technologies (network connectivity, user behavior and sensor data) is the only way to understand context. For example, we can gauge the outside temperature combining the user’s current location with online weather information, and then use this data to offer phone numbers for nearby cab companies in addition to a walking route, assuming the user may not wish to walk to work in the rain.

Making Use of the User’s Story

Behavioral targeting or personalization refers to a range of technologies used by online website publishers and advertisers that allows them to increase the effectiveness of their campaigns by capturing data generated from website and landing page visitors and then adapting to their needs. Personalization technology enables the dynamic insertion, customization or suggestion of content in any format that is relevant to the individual user, based both on the user’s explicitly provided details and their implicit behavior and preferences.
Another aspect of personalization is the increasing prevalence of open data on the Web. Many companies make their data available on the Web via APIs, Web services and open data standards. For example, Pipl is a search engine designed to locate people’s information across the web. Pipl uses identity resolution algorithms to aggregate information and cross-link various sources before delivering an online profile containing a summary report of everything that’s publicly available for each individual. Pipl offers all that wealth of information to developers via an API. One useful application for this would be running an API request for an email address; one can determine the user’s gender, age, location and interests and provide an adaptive experience based on the individual user.

Pipl Search aggregates information that’s publicly available on any individual.
Understanding the user story is possible with a network connection. However, network connectivity is not only important to understand the user and his online record, it is a vital instrument that connects all other technologies together — cloud computing, understanding local weather, traffic conditions or even the type of connection itself (Wi-Fi or G3) can help us understand context. Ultimately, the possibilities inherent in understanding and designing to the user’s story — their context — are possibilities built upon the collection of sensor data and user data via the network.

Sensor Data

A sensor for adaptive systems is any technology that allows a device to understand and evaluate context. It includes a built-in accelerometer in smart devices, a camera, a clock or even a microphone. We can use the various sensors embedded in smart devices to better understand the user’s environment. For example, the built-in accelerometer can be used to gauge if a user is walking or running.
There are two main scenarios for using sensors: everyday objects transmitting data like temperature or noise level to other devices, for example, iGrill, a cooking thermometer and application that communicates with smart devices via a secure, long-range Bluetooth connection. Or smart device applications, utilizing the built-in sensors to receive, process and output data to the user. By using these sensors and mixing other technologies discussed above, we can often obtain powerful information on the context of use and use it to create adaptive systems.

iGrill Cooking Thermometer.
Sensors can be a powerful design tool of the future. For example, with the aid of sensors, e-commerce checkout will be as easy as logging into a bank account with no password. Here is an example of how using four layers of sensor data to secure the user’s identity with a degree of certainty to create passwordless banking that would present the user a “light” version of his bank account, so he quickly could check his account balance. Imagine a user is at home surfing to his bank account through his tablet computer.
The first layer of security is the username associated with the tablet. Second is the location sensor, which will give us a greater degree of certainty that the user is in his home vicinity, cross-checked with his registered address with the bank. The third layer is the Wi-Fi connection (its MAC address, a unique identifier assigned to a network) the user is surfing on. For the fourth layer, we can check for other nearby Wi-Fi connections (The neighbors are sure to have a unique Wi-Fi MAC address) that can also be used as a security verification. If these bits of data are consistent across several password logins, the system can adapt and allow the user to enter without any password.
To learn more about adaptive design and how to get from sensors to context, I highly recommended you read this paper by Albrecht Schmidt about Context-Aware Computing (Interaction Design Foundation Encyclopedia).

Conclusion

Today, we’re just starting to see the potential of using sensors and technology to connect between devices and people. The term “Internet of things” refers to uniquely identifiable objects that are network-connected. For example, a smart flowerpot that sends a signal when it’s time to water the flowers. There is no doubt that adaptive design will play a key role in making future devices and functional user interfaces that give users an intuitive control over their environments in any situation or context.