PenLight: Combining a Mobile Projector and a Digital Pen for Dynamic Visual Overlay

Comments

coming soon...

Summary

PenLight: Combining a Mobile Projector and a Digital Pen for Dynamic Visual Overlay, a collaboration between Autodesk Research, Cornell University, and the University of Maryland, describes a digital pen with lots of extra functionality. The researchers were looking to resolve one of the important design points made by Norman in The Design of Everyday Things: the importance of feedback. Current digital pens can capture your writing using motion and tiny cameras when written on special paper, and later converted to text or saved on the computer for easy use. But, besides the physical ink that is produced by the pen, there is little useful visual feedback about what you have written. There is almost no feedback at all (except for some audio) for navigating through the menus of the pen, which can be frustrating when you're trying to make sure your work gets saved. The researchers came up with a prototype idea for the "PenLight", which would use a mobile projector (currently being added to cell phones and small devices) to project useful images about the menu and about your drawings and writings on the paper. The small projector would give great feedback so you always knew what you were doing.

The main ideas for the PenLight are:

1. Three different layers in 3D space that can be written on: the surface, hovering right above the page, and higher (the spatial layer). Different menus can be shown for the different layers, and the distance from the pen to the paper is used to determine the current layer that is being viewed.
2. The user can immediately see the menu options and choose them with appropriate visual feedback.
3. Moving the pen around vertically and horizontally in 2D space parallel with the plane of the paper will show you different parts of the image stored in the pen (as if viewing through a movable window).
4. Easy input with lots of applications.

The prototype they made consisted of an existing digital pen, a magnetic 3D tracking device attached to the pen, and (as mobile projectors are yet to be completely developed) an overhead projector simulating the projection from the pen. This system was able to completely emulate the PenLight's features as explained above.

The main application that the researchers gave was to help the architectural design field, as they still primarily work with physical schematics and drawings. Features for this program included:

1. Virtual Ink: you can write on layers within the pen's memory (directly overlaid on the paper) virtually without writing on the paper physically.
2. You can trace over an object on the paper and then move the object to other locations.
3. The pen can project a virtual guide for tracing to help with drawings.
4. The pen can overlay different content depending on the object on the paper; for example, it could overlay the electrical wires and water pipes over the drawing of a building.
5. You can overlay computations of distance, etc. on top of the schematics.
6. You can initiate a 3D walkthrough of the building by drawing a 2D path.
7. Copy/Paste from the physical drawing.
8. Searching through the physical drawing for an object or number.
9. Displaying the 3D building and cutting out 2D cross sections.

The current prototype that they creating can already do all of the above and more.

Discussion

I thought this was one of the coolest papers I have ever read. The ability to see the menus projected on the paper in front of you is already pretty cool, but also being able to draw a walkthrough on the paper and have it go through it in 3D? That's pretty amazing. Just the ability to leave extra notes about the content in separate layers without marking all over the paper is an amazing feature, and being able to wirelessly sync your new annotation to another architect's desk is great. I think this device (if they can develop it soon, and the projector, etc. can be developed and mounted on a pen) will reinvent the modeling and computational architecture field, and make designing buildings a much easier task.