Designing for Mobile Interaction with Augmented Objects.

Interaction design, graphic design, prototyping, and user study setup for a paper submitted and accepted into the proceedings of the 2012 International Symposium on Pervasive Displays.

In collaboration with Martin Tomitsch and Hanley Weng.


Martin Tomitsch, Mark C. Mitchell, and Hanley Weng. 2012. Designing for mobile interaction with augmented objects. In Proceedings of the 2012 International Symposium on Pervasive Displays(PerDis '12). ACM, New York, NY, USA, , Article 5 , 6 pages. DOI=10.1145/2307798.2307803


Handheld projectors represent an emerging type of pervasive display that can provide situated, personalised access to digital information. Beyond the projection of content such as photos or websites they can be used to augment physical everyday objects with digital information. This approach has several advantages compared to other augmented reality (AR) technologies. For example handheld projectors eliminate the need to wear head-mounted displays and provide a true integration of the virtual overlay with the physical environment. Yet little is known regarding user interaction with projection-based AR interfaces. The work described in this paper aims to fill this gap through an explorative comparison of four interaction techniques. The techniques were evaluated with two iterations of a projection-based interface that augments physical books with digital information. The interface was prototyped with a fixed projector and a depth-camera sensor to simulate handheld projection. The paper provides a discussion of the techniques along with insights regarding the interface design of projection-based AR interfaces.

Interaction technique illustrations

Initial design mockups

Final design