Not long ago, I followed a thread on Twitter that started with the statement, “On a touch screen, buttons are a hack.” It was followed by a few heated responses such as my favorite, “You’ll pry my buttons out of my cold, dead hands.” I leaned back in my chair, rested my hands on my keyboard, and pondered my 140 character reply. “Buttons are sometimes a lazy touch designer’s easy way out,” I thought, “but sometimes they’re a necessity.” Not having a pithy answer, I scrolled past the slew of posts and carried on with my day. Weeks later, with the thread still lingering in the back of my mind, I decided to explore the topic in greater depth. In this post, I try to explain when gestures are more appropriate than buttons or other GUI (graphical user interface) elements. Although the recommendations can be applied to a variety of touch screens—kiosks, tables, walls—for the sake of brevity, the examples will focus on applications for multi-touch mobile devices and tablets.
If you’d like to create an application with an immersive or skeuomorphic design, slapping buttons all over the user interface will certainly undermine your goals. Apps that fall into this category may include games, books, musical instruments, and more. Instead of standard GUI elements, these apps tend to combine gestures and interface metaphors (e.g., drag finger to turn virtual page). This doesn’t mean these applications have to be entirely devoid of buttons. For example, most e-reader applications will hide buttons when the user is in reading mode but liberally use them when the user wishes to change settings (Figure 1). This strategy makes sense—imagine having to memorize gestures for all of the iBook or Kindle settings!
Emulating natural interactions
Certain applications can benefit from gestures that emulate natural interactions with the physical world. Using gestures in this manner may encourage experimentation and may be easier to learn since users already have a mental model of the interactions. For example, diagramming applications like Freeform allow users to rotate images using a two-finger gesture, just like a child would spin a small ball. Again, this doesn’t mean the entire application has to be gesture-based, only the interactions where gestures enhance the user experience. As you can see in the screen below, a large number of Freeform’s interactions still require buttons (Figure 2).
Streamlining complex interactions
Designers may choose gesture-based interactions if they are more efficient than the GUI alternative. For example, reordering items with a GUI may require users to 1) enter numbers in form fields and 2) save the new order via a form submit button. With a gesture-based interaction, the user can simply use their finger to drag-and-drop the items into the desired order; the feedback is immediate. Not only is this interaction faster, but the task requires less cognitive effort. Another example that streamlines app interactions is Yahoo! Sketch-a-Search (Figure 3). Instead of tapping or pinching into the desired search area, users simply circle the area with their finger. In this case, one gesture (circle) is more efficient and precise than multiple gestures (taps or pinches.)
Shortcuts for frequent tasks
Frequent tasks may also be strong candidates for gesture-based interactions. If the gesture-based interaction is even slightly faster, it may make the user feel more productive and efficient as they use your app. Over time, the gesture may become second-nature, alleviating usability concerns related to gesture recall. To illustrate, consider the iOS Mail application. Users may delete messages from the message view but they can also swipe across a row to reveal the delete button (Figure 4). The only downside with this particular implementation is the discoverability—from what I’ve observed, users tend to find this feature accidentally or hear about it from a friend. Ideally, apps should educate users if important gestures are not immediately evident (game apps are an exception.)
Leveraging familiar gesture patterns
If you’re not sure where to begin with gestures, start by leveraging the built-in gestures supported by your platform (see Gesture Reference Guide). Since most users will have had sufficient exposure to the built-in gestures (e.g., swipe to view photos within a slideshow) they will often discover the intended interaction in absence of tips or tutorials. On the other hand, if you don’t provide built-in gestures when users expect them, you might confuse users. For example, iOS users are accustomed to using the Pinch Open/Close gesture within the Maps app. If nothing happens when users try to pinch a map within your app, they may think your app is broken.
Take a look around the App Store and you’ll see that very few applications completely remove traditional GUI elements like the sturdy, old button. Notable exceptions are games and creative applications with a limited set of tasks. But for most genres, particularly productivity apps like email, GUI elements are still a necessity. If every single interaction was entirely gesture-based it would be difficult for users to keep track of each gesture. Lastly, keep in mind that it’s usually not one or the other. More often than not, gestures will co-exist with standard GUI elements and other input methods such as voice. It looks like buttons might be here to stay.
Thanks to Kevin Arthur @karthur for providing feedback on this article.