Touchstone Final Wrap-Up

Touchstone

The Touchstone is a device by John Finley, Angela Huang and Jeff Kirsch that allows users to exchange a touch at a distance. In its current iteration, it uses light to communicate touch across a room. In further iterations, the Touchstone will use an internet relay to allow a pair of devices to communicate from anywhere in the world. The story of its development to date follows.

Origins and Ideation

In brainstorming projects for the final, we began by identifying a number of broad themes of interest, including:

• Action at a distance
• Visualizing the invisible
• Multiple small parts acting as one
• Bringing the distant closer
• Ambient communication

From there, we brainstormed a variety of projects that touched on one or more of these points. One in particular emerged as interesting to the entire group, and seemingly plausible within the timeframe of the project.

Whether just across town, or somewhere on the other side of the world, everyone in the studio has someone they care about deeply but see considerably less than they’d like. A number of mechanisms exist for sending composed messages to these people, in phone calls, emails, IMs, text messages, etc. But what about simply communicating a “Thinking about you,” without the added overhead of words? The deceptively simple way a touch communicates that sentiment was the genesis of our Distant Touch concept, which evolved into the Touchstone.

Original Concept

As originally conceived, the Touchstone system would consist of a pair of desktop devices. When touched, the local device would sense the heat from the user’s hand and send a signal to its distant partner, causing the remote device to heat up. The more a Touchstone was touched, the hotter the remote device would become, to a maximum heat. Touchstones would “cool” much faster than they’d heat, which meant that communication of a touch was asynchronous; a remote stone would be hot to the touch if its partner was touched recently, or frequently. On Rob’s advice, we attempted to consider a bit more of the complexity of relationships, allowing the Touchstone to become actively cold if neglected.

There were a number of points we found compelling in this original conception. The idea of translating the heat of a touch into an electronic signal, and then back to heat on the other side seemed a novel analog to the translation of voice, images and text that we’re already quite familiar with. Additionally, the fact that the device responds to touch, and its output (heat) can only be sensed through touching meant there’d be no way to determine the state of the “conversation” without effecting it.

From a technical perspective, we quickly realized that attempting to sense the heat of a touch in a device that heats up would likely lead to a variety of headaches in the form of feedback loops, we decided to instead use a capacitive sensor as a digital touch trigger, simply counting the length of a touch, and incrementing the programatic “heat” accordingly.

We also realized that getting a Peltier Junction in our hands and working would be difficult for the first iteration, and instead simulated heat through an LED that went from blue (cold) to red (hot.)

Differentiation of Labor

In order to best use each person’s skills, we decided to split out the various tasks involved in developing the prototype. Angela, with her background in Industrial Design, dove into formal studies for the shape of the device, and essential element in building something that afforded touching and could present a “neutral” posture from which to express presence or absence. Finley, one of two people in our class who could properly be called programmers, concentrated on the software while I worked in tandem building out the hardware. In reality, we both moved between both of these roles, troubleshooting each other’s code and wiring at three in the morning. As much as possible, we were all involved in the bigger picture interaction questions, and I took something of a role of keeping an eye on where the various pieces of the project were and creating informal landmarks for development.

First Iteration and Feedback

The first iteration of the Touchstone brought together a number of in-progress bits to give the class a general idea of where the project was going. It incorporated a blue to red heat-simulating LED, in addition to capacitive proximity sensor, all hidden under a hollow styrofoam orb. As there was only one device, we rigged the prototype to “heat” itself up, so that when touched, it actually displayed the state of the imaginary remote stone. The untouched device sat on the table glowing blue; when touched the color would become increasingly red, until a point where the orb glowed purely red. When the touching hand was removed, the device would slowly “cool” back to blue.

We gained some useful insight from this simple demonstration. First, we learned that although we intended the LEDs to serve only as a stand-in for heat, the class found the lights to be very compelling. Second, when we explained that the response they saw was in fact that of the remote device, our classmates raised their concern about the lack of a local verification of interaction, and indication of the state of the remote device. We took both of these into consideration as we moved forward with prototype.

Faking It (Second Iteration)

From the beginning, we realized that we wouldn’t really know much about how the interaction worked until people tried it, so we set about trying to make as “real” of a prototype as we could as quickly as possible. Unfortunately, technical progress came slower than we’d hoped, and by the second demonstration, we’d just barely gotten two Touchstone mockups to communicate over a wired serial connection. Rob encouraged us to consider building a simple man-behind-the-curtain prototype to allow people to better experience and respond to a demonstrated interaction. We quickly decided to fork the development once again, with Angela continuing work on the form of the device and Finley taking on primary responsibility for the software and hardware of what would become the eventual for-presentation prototype, while I quickly built and coded a potentiometer-controlled, two-LED interaction model.

The benefits were immediate and obvious. Before even getting the device into classmate’s hands, seeing the “remote” and “local” heat indicator lights both fully “hot” suggested the need for a special state when both devices were touched simultaneously; I quickly added a “heartbeat” pulse on both devices to indicate as much. This, in turn, helped us realize that the interaction was not so much about heating or cooling the Touchstone over the longterm, but instead about a synchronous communication of presence. Heartbeat state enabled, we let people play with it.

Two-color Touchstone interaction test from Jeff Kirsch on Vimeo.

Again, a number of insights immediately came of the first hands-on tests. Parts of the interaction seemed natural enough, but it was unclear what each light indicated without being able to see what was (or wasn’t) happening with the other device. Additionally, having multiple colors plus a “pulse” seemed too much for a device as simple as this. Blue didn’t map to “cold” or “neglected” as well as red mapped to “hot” or “maintained,” and the range of purples in between didn’t seem to map to anything at all. We’d been sticking too closely to our simulation of hot and cold, and realized that “off” served better than blue to indicate that the devices weren’t touched, allowing the increasingly bright red light paired with the heartbeat pulse to communicate more clearly. We also played with some labeling for the local and remote indicators on the stone.

Our success in this was confirmed when testers started to respond, unprompted, with realization and delight at the appearance of the heartbeat pulse, even in our obvious curtain-less test.

One-color Touchstone interaction test from Jeff Kirsch on Vimeo.

Somewhere in the midst of these various discoveries, we were able to get wireless communication working, and to pull elements of code from the interaction prototype into the actual device prototype. A video of the first successful test (where we somewhat confusingly use a blue LED to indicate the remote side) follows:

Two color touchstone communication test from Jeff Kirsch on Vimeo.

Technical Decisions

With our final presentation quickly approaching, we realized some decisions had to made as to which features and functionality would make it into the final prototype. We’d done some testing with a Peltier Junction, but the difficulties of thermodynamics, paired with the success of our LED-based mock-ups led us to drop the heat angle without much remorse. We’d also been hoping to make all the electronics for the final product fit within the shell of the Touchstone itself (by this time, beautifully rendered in acrylic by Angela) but realized after a short feasibility meeting that the benefits of spending the required time and energy to attempt that goal would be much less than spending that time ensuring that we had two identical, reliably communicating Touchstones exhibiting the desired interaction model. A similar calculation caused us to, with somewhat more remorse, abandon our attempts as making the capacitive sensing work (replacing it instead with a force sensor, which left the interaction identical from the user’s perspective) and of making it possible for the devices to communicate at a range beyond what could be achieved with the xBee radios (a more problematic concession, and one we plan to address in future iterations.)

Final Prototype

For our final presentation, we built two electronically and programmatically identical devices, each with a pair of red LEDs, a force sensor and an xBee radio to communicate with their partner device.

Programmatically, the stones still use a “heating” metaphor. Each time through the code, we check for any pressure on the device. If pressure is sensed, we increment a counter by one (to a maximum of 255), and transmit the value of that counter over the radio, while simultaneously lighting the local LED with a corresponding value. If no pressure is sensed, we decrement the counter and do the same, down to a level of zero.

On the receiving end, each stone simply reads its serial port, and sets a “remote heat” variable to the received value. The remote LED lights accordingly. If no good value is received, the variable remains at the last good value. This bare-bones approach (we’d initially attempted to transmit a single character to indicate touch, and to have each stone increment or decrement the remote value internally) not only greatly simplified our code, but made the system appear much smoother and more reliable to the user.

Physically, we used Angela’s finished acrylic model, as well as a foam form model. We put LEDs in each and force sensors on the bottoms, and set them up on a piece of cloth, running the umbilical cable hidden underneath to and under-table box with the rest of the hardware. The two Touchstones were set up on opposite sides of the room.

The setup worked convincingly enough to spark a critique about the nature of the interaction, rather than the fidelity of the prototype, and by that measure it was certainly a success.

A video of the final prototype, showing one side of the interaction:

Touchstone Demonstration from Jeff Kirsch on Vimeo.

Attempted Revisions

Prior to the final class meeting, we attempted three different additions to the prototype. At the suggestion of one of our guest reviewers to make the Touchstones more responsive physically, Angela incorporated memory foam into a second acrylic model. Finley and I worked to get an ideally more reliable capacitive sensor back into the equation, but quickly realized that we were once again going down the wrong path by trying to develop a makeshift solution for something that could be easily achieved with the right components, and that made little obvious difference to the user in the long run.

We did, however, make a serious attempt at adding a second level of interaction, namely, the ability to squeeze the Touchstone and have that physically manifest itself (through a concentrated vibration) on the remote device. While ultimately unsuccessful, we got far enough to know what needed to be accomplished to implement that new functionality, changing the “heat” values imperceptibly from 0-255 to 0-254, and using a value of 255 to indicate that the remote device was being squeezed. This would then cause a small vibrator motor (embedded in the memory foam) to vibrate.

Next Steps

We all learned a lot from working on the Touchstone, and by the end of class found ourselves every bit (if not more) interested in the project as we were at the beginning. We’re planning to continue development as time allows, concentrating on a few important areas.

First, we plan to find a way to make the device work from different locations. This is an essential part of the interaction that fundamentally changes the way the Touchstone is perceived, and potentially, the viability of the solutions we’ve out in place thus far. Part of this is making it capable of connecting to the internet, and another (I’m convinced) is getting it into a condensed, stable package. Neither is trivial, but they both should be well within our abilities.

Second, we’d like to continue refining the form. At present, there’s something unresolved about the texture of the device, and we need to determine if the addition of memory foam hurts that or helps. There’s also a difference of opinions about the organic shape of the device, whether it’s too much or not enough.

Both these things will help us in the key determinations we need to make about the fundamental interaction. Ultimately, is it successful? Does it compellingly provide a new means of communicating presence and care at a distance? Does it appropriately allow for the opposite to be communicated? Must it? In the end, the only way to tell is to get the devices out in the world, in the hands of users, and observe the results.

Code, as used in the final demo is below:

/*
_____ _ _
|_ _|__ _ _ ___| |__ ___| |_ ___ _ __ ___
| |/ _ \| | | |/ __| '_ \/ __| __/ _ \| '_ \ / _ \
| | (_) | |_| | (__| | | \__ \ || (_) | | | | __/
|_|\___/ \__,_|\___|_| |_|___/\__\___/|_| |_|\___|

by John P. Finley, Angela Huang, and Jeff Kirsch

December 9th, 2009 for Presentation to Rob Faludi's
Fundamentals of Physical Computing class at the
School of Visual Arts MFA Interaction Design program,
December 2009.

http://interactiondesign.sva.edu/classes/physicalcomputing/

Portions based on Heartbeat LED by elCalvoMike 12-6-2008
*/

// Variable Setup
#define localLED 9 // Local LED attached to PWM 9
#define remoteLED 10 // Remote LED attached to PWM 9

int incomingByte = 0; // variable to store the read value
int localHeat = 0; // variable to store the read value
int remoteHeat = 0; // variable to store the read value

int i = 0; // variable to store the read value
int rate = 20; // variable to store the read value

int analogPin = 0; // variable to store the read value
int val = 0; // variable to store the read value

// The setup opens serial comm and initializes our LED pins.
void setup(){
Serial.begin(9600); // open serial communication
pinMode(localLED, OUTPUT); // sets the digital pin as output
pinMode(remoteLED, OUTPUT); // sets the digital pin as output
}

// The rundown:
// First, see if the device is being touched. Modify
// the heat counters appropriately. Then write that
// value to the serial for the other device to read.
// Read the other device's heat vlaue over serial.
// Light up the two LEDs based on those values.
void loop(){
checkIfIAmTouched();
talkToOtherDevice();
listenForOtherDevice();
lightLEDs();
}

//**************************************************//
//**************************************************//

// Ask the FSR if there is anything happening.
// If we are getting a value, we are being touched,
// so increment the heat counter. If we have no
// value, we are being left alone so decrement.
// Delay to prevent doom.
void checkIfIAmTouched(){
if (analogRead(analogPin) > 0 ){
makeThingsHotter();
}
else{
makeThingsColder();
}
delay(10);
}

// Print out this devices' heat value to the other device.
void talkToOtherDevice(){
Serial.write(localHeat);
}

// Read the serial buffer. If there is something there,
// assign that value as the heat of the remote device.
// Flush the serial port to prevent doom.
void listenForOtherDevice(){
if (Serial.available() > 0){
remoteHeat = Serial.read();
Serial.flush();
}
}

// Increment this device's heat value unless it
// hits a ceiling of 255.
void makeThingsHotter(){
if (localHeat < 255){
localHeat++;
}
}

// Decrement this device's heat value unless it
// hits zero.
void makeThingsColder(){
if (localHeat > 0){
localHeat--;
}
}

// Light both the LEDs with their proper values.
// If both heat values are at maximum, we are holding
// hands; time to start the heartbeat animation.
// If no, just give the value that has been determined
// by makeThingsHotter() or Colder().
void lightLEDs(){
if(localHeat == 255 && remoteHeat == 255){
heartPulse();
}
else{
analogWrite(localLED, localHeat);
analogWrite(remoteLED, remoteHeat);
}
}

// Hearbeat simulation animiation.
// From full brightness go down slightly,
// then up and then down for a while.
// Inspired by elCalvoMike.
void heartPulse(){
for (i = 255; i > 100; i--){
analogWrite(remoteLED,i);
analogWrite(localLED,i);
delay(((60000/rate)*.1)/255);
}
for(i = 100; i < 255; i++) {
analogWrite(remoteLED,i);
analogWrite(localLED,i);
delay(((60000/rate)*.2)/255);
}
for (i = 255; i > 100; i--){
analogWrite(remoteLED,i);
analogWrite(localLED,i);
delay(((60000/rate)*.1)/255);
}
for(i = 100; i < 255; i++) {
analogWrite(remoteLED,i);
analogWrite(localLED,i);
delay(((60000/rate)*.6)/255);
}
}