Physical and Frictionless Input

Lately, we’ve been thinking about ways to improve online collaboration and group participation. One aspect of collaboration that we really want to explore is input – in particular, physically linking the input experience closer to output. Imagine manipulating a real-world replica of a virtual object and synchronising both, or in a more real example, being able to check-in to a location through Foursquare by physically touching it – an experience profoundly more unified than scrolling through a list of locations based on your GPS coordinates. For our initial exploration, we decided to build a simple, frictionless way of manipulating an object and broadcasting the input to multiple clients.

We wanted to do with something with 3D, and when you’re looking for something simple to start with, that only means one thing: cubes. A colour cube was particularly appealing because of its simple physical construction and interaction – rotation to reveal colours. For the sake of experimentation, we chose the cube to represent only the colour property of the virtual plus symbol. That’s how we came up with two separate modes for the demo.

Reading data off needed to be frictionless and easy. NFC provided the perfect bridge between object and device as it requires no handshaking, no pairing, and no user setup. With this method, we could simply tap the device with the cube to change colours. The bouncing balls you see in the demo were only meant to re-enforce the fact that you’re manipulating a complex environment with simple input.

The cube itself is internally tagged with six separate NFC stickers on each face. The stickers are read/writable Mifare Classic’s that can hold 1k of data. We wrote each tag with plain text JSON that contained a single colour value. In order to read the tags, we built an Android app for the Nexus S that listened for NFC tag discovery and established a socket connection to a node.js server on EC2 once the NDEF (the standard NFC data format) was parsed. It was straightforward to work with node-websocket-server (socket.io was problematic with Android) to connect to a standard HTML5 WebSocket in the client browser. Finally, we used the three.js to render the plus symbol and 3D environment in WebGL.

One interesting result from the experiment was that the passive mode (laying the device flat on the table and tapping the cube) felt like a much better interaction. The device became invisible and we were left solely manipulating the colour cube. The link between physical input and virtual output felt stronger. Handling the cube with both hands gave us illusion of full control – regardless of the fact that the device was doing all the work. It’s also worth noting that this type of approach is easier to implement on a large scale as the availability of NFC-enabled devices is generally low (though Google is trying hard to fix that).

It’s clear that simplifying and physically tailoring input to output with tech like NFC is a good way to enhance user experience and can lead to some unique interactions. We plan on taking this idea further by experimenting with other platforms and input devices.