Last year MIT’s Tangible Media Group went viral because of its real time shape-copying machine. But one of its latest efforts blows that machine out of the water. Called THAW, it’s a software system that allows smartphones and personal computers to communicate graphically through their respective displays. The best part? It already works with today’s devices.
THAW lets you transfer or add content to your personal computer by moving your smartphone over a computer display. For example, in the image above, the player made a platform for the video game character in the laptop by placing his smartphone right where he wanted the platform to be. That’s just one of the things that you can do with THAW. You can use it to physically drag and drop files from a computer to a smartphone (and possibly vice versa), copy the content or website displayed on a monitor just by putting your phone over it, and use multitouch to modify or interact with graphics on a computer.
So how does it work? Smartphones orient themselves using a colored pattern that’s displayed on the personal computer’s screen at all times. This pattern can be adjusted so that it’s invisible to human eyes but still detectable by a smartphone’s rear-facing camera. As you can see in the image above the colors are different for any given area of the screen, so it serves as the phone’s indication of where it is with respect to the display. Throw in a phone’s gyroscope or accelerometers and THAW can also figure out the phone’s distance and orientation in 3D with respect to the display.
This would be a great way to interact with table-style computers, like the original Microsoft Surface. This could even lead to a new class of devices – small tokens that only have the required hardware needed for THAW – and peripherals. If only OS X Yosemite’s Continuity would work like this.