Usability problem of tomorrow (Star Trek user interfaces)

In the courses I teach about human-computer interaction, I typically open each class with an example of a usability problem. I'm putting these online, in case others find them useful.

I sometimes take examples of interaction design from the entertainment industry, whether the systems involved are real or not. Star Trek: The Next Generation is a good standby.

(io9.com)

There are a number of obvious things we can't do as well today as in Star Trek TNG, even leaving aside transporter technology and focusing on user interfaces. Today's voice input systems, for example, are less idiosyncratic than Star Trek's, but they're also much less capable.

We also don't have holodecks. Not that we haven't been thinking about them. Ivan Sutherland described such a system in 1965 as "The Ultimate Display":

The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal. With appropriate programming such a display could literally be the Wonderland into which Alice walked.

But we don't have the technology yet--and I'm happy enough dealing with guns and handcuffs in purely virtual terms.

We don't have communication systems that can read our minds. This is only by implication in Star Trek, but consider:

Scene 1, on the Enterprise bridge.
Picard: Picard to Riker.
Riker (off-camera, gallivanting on the planet below): Riker here.

Scene 2, on the planet.
Picard (off-camera, via Riker's communicator): Picard to Riker.
Riker: Riker here.

Notice anything strange? There are no delays in the conversation. That is, Riker hears, "Picard to..." before the sentence is even finished. By analogy, imagine listening to your mobile phone as someone dials your entire number, from the first to the last digit. That's remarkable predictive technology. (That, or everyone hears the beginning of messages to everyone else, which would be pretty annoying.)

We have surpassed Star Trek in a few ways, though. 

Information displays. Take a look at QUANTUM IMAGING SCAN 61563, above. Sadly, mixed-case text has apparently been abandoned by the 24th century.

Uh-oh. In the second row of the second unlabeled table of the display, the third occurrence of the number 621225 in the second row down appears to be anomalous. (Fortunately, 382372127382372127532192 in the same row shows no such problem.) We should... do something. Perhaps reverse the polarity. Is the highlight color of 621225 related to the unlabeled boxes below it? Or the panel labeled 05-87865222? What is it possible to do with this display, in any case? Clean graphical design is great, but it's most effective if it convey useful information that helps us get some job done.

Personalization and context awareness. In his quarters, Captain Picard tells the replicator, "Tea. Earl Gray. Hot." He never orders tea any other way. And yet he must specify the order in detail every time--no one wants cold oolong. Picard seems to value efficiency; we can imagine that he might like to reduce a four-word command to just one. Would this work at a public replicator? Of course. Recognition of individual speakers by voice is possible now. And we can even define our own voice commands to tell the computer what to do.

Gesture. On the bridge, something faint but interesting appears on the main display. Picard: "Computer: Magnify sector 32 mark 14." The captain has apparently passed the "grid memorization for using large-scale displays" unit at Star Fleet Academy. It might be less dignified but it would definitely be more effective to point at the screen and say, "Magnify that." We've have interfaces to do this as long ago as 1979.