Tag Archives: User interface techniques

A Call for Diminished Reality

(a copy of my weekly syndicated column. Podcast from the BBC here.)

By Jeremy Wagstaff

I was walking the infant the other day, when I saw a vision of my future.  A mother and father, out with their son and dog.  The mother sat on a park bench, dog sitting obediently at her feet as she flicked absent-mindedly at her iPhone.

In the playground, the boy wove his way through a tunnel, across some ropes, down a slide–the father nearby, lost in his own iPhone. Occasionally he would waken from his 3G trance and, without looking up, point the phone at his son as if scanning him for radiation.  The resulting photo probably went straight to his Facebook page.  Ah, happy families, connected by place but detached by devices.

It’s a familiar lament.  Our devices distract so much we can’t ignore them.  We ignore our kith and kin but obey their beeps, walk into traffic or drive into pedestrians to heed their call.  And the solutions are usually less than imaginative, or practical: holidays where you check them in at the gate, where you put them in a glove compartment, or (shock), leave them at home entirely.

I have tried all these and they don’t work.  Which is why I fear I will be that family. Perhaps I already am; desperate to catch my infant’s first steps, words, or symphony, I think it more important that my cellphone camera is there, somehow, than I am. This is silly.  But I think I have found the answer in something called augmented reality.

Augmented reality is where our devices use their camera and positioning capability to add layers of information to what is in front of us: little pointers appear on the screen detailing where the nearest ATM is, or Chinese restaurant, or how far away and in what direction the nearest Twitter user is. The reality is the scene in front of us viewed through our camera, the augmented bit are these layers of extra information.

This is not new, but it’s becoming more popular.  And it’s kind of fun.  It is related to another technology that adds a layer onto what we see—so-called heads-up displays, that project information onto the windscreen of our airplane, or car, or goggles, that help us identify a target, a runway, an obstacle in the road.

Interesting, but I think they’ve got it all backwards.  Our problem is not that we need more information overlain on the world, we need to have the world overlain on the screens that command us.  We spend so little time interacting with the world now that we need technology to help us reintroduce the real world back into our lives.

I don’t think handing over our devices to well-intentioned guards at hotel gates is going to do it.  I think we need to find a way to fit the real world into our device.

Which is why, two years ago, I got very excited about an application for the iPhone called Email n Walk.  This was a simple application that overlays a simple email interface on top of whatever is in front of you.  The iPhone’s camera sees that for you, but instead of putting lots of pins about ATMs, Chinese restaurants and twitter users on the image, it puts the bare bones of whatever email you’re typing.  You can type away as you’re walking, while also seeing where you’re going.

Brilliant.  And of course, as with all brilliant things, it got lots of media attention and promptly disappeared.  The app is still there on Apple’s software shop, but the company’s home page makes no mention of it.  I tried to reach the developers but have yet to hear back.

They’re careful not to claim too much for the software. We can’t take any responsibility for your stupidity, so please don’t go walking into traffic, off of cliffs, or into the middle of gunfights while emailing, they say.  But it’s an excellent solution to our problem of not being able to drag our eyes from our screens, even to watch our son clambering over a climbing frame.

It’s not augmented reality, which purports to enrich our lives by adding information to it.  It’s a recognition that our reality is already pretty hemmed in, squeezed into a 7 by 5 cm frame, and so tries to bring a touch of the real world to that zone.  I believe that this kind of innovation should be built into every device, allowing us to at least get a glimmer of the real world.

Indeed, there are signs that we’re closer to this than we might expect. Samsung last month unveiled what may be the world’s first transparent laptop display, meaning you can see through it when it’s turned on, and when it’s turned off. I don’t pretend that it’s a good solution to the growing impoverishment of our lives, which is why I have no hesitation to call this inversion of augmented reality ‘diminished reality.’

And now, if you’ll excuse me, my daughter is making funny faces at me through the screen so I better grab a photo of it for my Facebook page.

A Call for Diminished Reality

By Jeremy Wagstaff

I was walking the infant the other day, when I saw a vision of my future.  A mother and father, out with their son and dog.  The mother sat on a park bench, dog sitting obediently at her feet as she flicked absent-mindedly at her iPhone.

In the playground, the boy wove his way through a tunnel, across some ropes, down a slide–the father nearby, lost in his own iPhone. Occasionally he would waken from his 3G trance and, without looking up, point the phone at his son as if scanning him for radiation.  The resulting photo probably went straight to his Facebook page.  Ah, happy families, connected by place but detached by devices.

It’s a familiar lament.  Our devices distract so much we can’t ignore them.  We ignore our kith and kin but obey their beeps, walk into traffic or drive into pedestrians to heed their call.  And the solutions are usually less than imaginative, or practical: holidays where you check them in at the gate, where you put them in a glove compartment, or (shock), leave them at home entirely.

I have tried all these and they don’t work.  Which is why I fear I will be that family. Perhaps I already am; desperate to catch my infant’s first steps, words, or symphony, I think it more important that my cellphone camera is there, somehow, than I am. This is silly.  But I think I have found the answer in something called augmented reality.

Augmented reality is where our devices use their camera and positioning capability to add layers of information to what is in front of us: little pointers appear on the screen detailing where the nearest ATM is, or Chinese restaurant, or how far away and in what direction the nearest Twitter user is. The reality is the scene in front of us viewed through our camera, the augmented bit are these layers of extra information.

This is not new, but it’s becoming more popular.  And it’s kind of fun.  It is related to another technology that adds a layer onto what we see—so-called heads-up displays, that project information onto the windscreen of our airplane, or car, or goggles, that help us identify a target, a runway, an obstacle in the road.

Interesting, but I think they’ve got it all backwards.  Our problem is not that we need more information overlain on the world, we need to have the world overlain on the screens that command us.  We spend so little time interacting with the world now that we need technology to help us reintroduce the real world back into our lives.

I don’t think handing over our devices to well-intentioned guards at hotel gates is going to do it.  I think we need to find a way to fit the real world into our device.

Which is why, two years ago, I got very excited about an application for the iPhone called Email n Walk.  This was a simple application that overlays a simple email interface on top of whatever is in front of you.  The iPhone’s camera sees that for you, but instead of putting lots of pins about ATMs, Chinese restaurants and twitter users on the image, it puts the bare bones of whatever email you’re typing.  You can type away as you’re walking, while also seeing where you’re going.

Brilliant.  And of course, as with all brilliant things, it got lots of media attention and promptly disappeared.  The app is still there on Apple’s software shop, but the company’s home page makes no mention of it.  I tried to reach the developers but have yet to hear back.

They’re careful not to claim too much for the software. We can’t take any responsibility for your stupidity, so please don’t go walking into traffic, off of cliffs, or into the middle of gunfights while emailing, they say.  But it’s an excellent solution to our problem of not being able to drag our eyes from our screens, even to watch our son clambering over a climbing frame.

It’s not augmented reality, which purports to enrich our lives by adding information to it.  It’s a recognition that our reality is already pretty hemmed in, squeezed into a 7 by 5 cm frame, and so tries to bring a touch of the real world to that zone.  I believe that this kind of innovation should be built into every device, allowing us to at least get a glimmer of the real world.

Indeed, there are signs that we’re closer to this than we might expect. Samsung last month unveiled what may be the world’s first transparent laptop display, meaning you can see through it when it’s turned on, and when it’s turned off. I don’t pretend that it’s a good solution to the growing impoverishment of our lives, which is why I have no hesitation to call this inversion of augmented reality ‘diminished reality.’

And now, if you’ll excuse me, my daughter is making funny faces at me through the screen so I better grab a photo of it for my Facebook page.

The Shape of Things to Come

This is from my weekly newspaper column, hence the lack of links.

By Jeremy Wagstaff

We’re all touch typists now.

Of course, the definition of touch type has had to change a little, since most of us don’t actually learn touch typing as we’re supposed to. Watch people tapping away at a keyboard and you’ll see all sorts of cobbled-together methods that would make the office secretary of yesteryear blanch.

But for now keyboards are going to be with us for a while as the main way to get our thoughts into a computer, so some sort of touch typing is necessary.

But the mobile phone is different. After ten years most of us have gotten used to entering text using the predictive, or T9, method, where the phone figures out you’re trying to say “hello” rather than “gekko” when you tap the 4,3,5,5,6 keys.

Texting has gotten faster—Portugal’s Pedro Matias, 27, set a new world record in January by typing a 264-character text in less than 2 minutes, shaving 23 seconds off the previous record—but that’s still slower than your average touch typist, who manages 120 words-say 480 characters—in the same amount of time.

Blackberry uses have their QWERTY keyboards, each key the size of a pixie’s fingernail, and while some people seem to be quite happy with these things, I’m not.

And the iPhone has given us, or given back to us, the idea of little virtual keyboards on our screen. I’ll be honest: I’m not a big fan of these either.

The arrival of the Android phone hasn’t really helped matters: The keyboard is usually virtual (some of the earlier phones had physical keyboards, but most have dropped them in favor of onscreen ones) and I really didn’t enjoy typing on them.

To the point that my wife complained that she could tell when I was using the Android phone over my trusty old Nokia because she didn’t feel I was “so reachable.” By which she means my monosyllabic answers weren’t as reassuring as my long rambling Nokia, predictive text ones.

But that has changed with the arrival of software called ShapeWriter. ShapeWriter is software that provides the same virtual keyboard, but lets you swipe your words on it by dragging your fingers over the keys to, well, form a shape.

Typing “hello,” for example, is done by starting your finger on “h”, dragging it northwest to “e”, then to the far east of “l”, lingering there a second, then north a notch to “o.” No lifting of the finger off the keyboard. Your finger instead leaves a red slug-like trail on the keyboard, and, in theory, when you lift your finger off the keys that trail will be converted to the word “Hello.”

And, surprise, surprise, it actually works. Well, unless you’re demonstrating it to a skeptical spouse, in which case instead of “hello” it types “gremio” or “hemp.”

Now this isn’t the first time I’ve used ShapeWriter. It has been around a while—it was first developed by IBM Labs in the early 2000s. It’s gone through quite a few changes in the meantime, not least in the theory behind it.

But the main bit of thinking is the same as that with predictive text (and speech recognition): what is called the redundancy of language. Taking, for example, the whole body of emails written by Enron employees, the most frequent email sender wrote nearly 9,000 emails in two years, totalling about 400,000 words.

That’s a lot of words. But in fact the number of actual words was about 2.5% of that: That email sender only used 10,858 unique words.

Now of course, Enron employees might not be representative of the wider population, but researchers have to work with data, and the Enron case threw up lots of data. The Enron Email Dataset is a 400 megabyte file of about 500,000 emails from about 150 users, mostly senior management of Enron. Making it a goldmine for researchers of language, machine learning and the like.

Learning from the words used—though presumably not their morals—researchers are able to figure out what words we use and what we don’t. Thus, ShapeWriter, and T9, and speech recognition, are able to tune out all the white noise by only having to worry about a small subset of words a user is typing, or saying. Most words we either don’t use because our vocabularies aren’t that great, or because we haven’t invented those words yet.

ShapeWriter has 50,000 words in its lexicon, but it gives preference to those 10,000 or so words it considers most common (presumably

In ShapeWriter’s case, they produce a template of the shape of each word they decide to store in the software, so the shape you’re drawing—left-far right, up, down, along—is recognised.

In its latest incarnation it actually works surprisingly well, and I’d recommend anyone with an Android phone to check it out. (It’s free.) There’s a version for the iPhone too, as well as Windows Mobile and the Windows Tablet PC. Only downside: For now, at least, only five–European–languages are supported.

I am not convinced this kind of thing is going to replace the real keyboard, but it’s the first decent application I’ve come across that has gotten me back into actually enjoying tapping out messages on my device.

My wife, for one, is happy.

links for 2008-09-11

  • Avego.com is where travelers cooperate to make the whole transport system more efficient, saving us all money, wasted time and reducing pollution.

    A 5-seat car traveling with only a driver is inherently inefficient, and yet 85% of the time, that’s how cars travel in much of the world. With our iPhone GPS technology, web services and your participation, we can fill up those empty seats.

  • Did I get enough exercise today? How many calories did I burn? Am I getting good quality sleep? How many steps and miles did I walk today? The Fitbit Tracker helps you answer these questions.

  • Swype was developed by founders Cliff Kushler and Randy Marsden, along with a very talented team of software programmers and linguists.

    Cliff is the co-inventor of T9, the standard predictive text-entry solution used on over 2.4 billion mobile phones worldwide. He is the named inventor on multiple patents related to alternative text entry.

    Randy is the developer of the onscreen keyboard included in Windows, with an installed base of over a half a billion units. He is a recognized leader in the field of assistive technology and alternative computer input.

    Together, their experience is unmatched in developing onscreen keyboard-based text input solutions for mobile touch-screen devices.

  • ShiftSpace (pronounced: §) is an open source browser plugin for collaboratively annotating, editing and shifting the web.

  • # Create and track invoices you issue to clients.
    # Determine what you’re owed, by whom, and when it’s due.
    # Keep track of timesheets for yourself and your employees.
    # Notify your clients of new invoices.
    # Create interesting reports and analyze payment history
    # Save time & collect your money.

The Holes We Slip Through

This interesting tool, tho too nerdy for me, highlights one of the stupid holes that make technology a still frustrating exercise for most of us. You’re heading out the door for a meeting, you have the person’s phone number and pant size (don’t ask) in your computer but not in your phone, and yet there’s no simple way to get these bits of information from one to another without some laborious syncing, or emailing. In most cases I find myself printing out a whole page, or scrawling down the phone number on the back of a condom. Why do these little holes in our technology world still exist, and why can’t we fix them?

clipped from lifehacker.com

Windows only: Zap that phone number, grocery list or set of driving directions directly from your PC desktop to your cell phone with Clipboard2Phone.

 

Clipboard2Phone is a simple script that emails the contents of your clipboard to your cell phone with a key combination you define. Especially useful for sending phone numbers to your mobile – which will most likely make that number callable in one click – Clipboard2Phone comes in handy for quickly transferring any kind of information you need on the go, like a todo list, shopping list, driving directions or just a reminder to your future self.

Some PaperPort Tips

Further to my column in this week’s WSJ.com/AWSJ on PaperPort Pro and PaperMaster Pro, here are some tips on getting the most out of the former from Bob Anderson (ScanSoft Regional Director Asia Pacific and Japan):

Tip 1 – Using PaperPort PageViewer as a separate application and displaying PaperPort across two monitors. By default when you double click on a file it opens PageViewer as an integrated window over the PaperPort Desktop. You can change this behaviour under PaperPort Options: Desktop: Page View. Select “Display items in: PageViewer Application”. Now the PaperPort PageViewer “floats” over the PaperPort Desktop and can be moved independently. Unfortunately a single monitor does offer enough space to make this productive. To really get the benefit of the feature I have an external monitor connected to my laptop. You can also put an additional video card in a desktop computer. The Display control panel in Microsoft Windows will allow you to extend you Windows desktop across two monitors. I keep PaperPort opened on my right monitor at all times and when I double click on a file they appear in the PageViewer on my left monitor. Once you start working this way you will never go back to a single monitor again.

I can now instantly “zoom” onto a document without covering the PaperPort Desktop and with the new Page Thumbnails I can click and drag pages from the PaperPort PageViewer onto the PaperPort Desktop and vice-versa. This is essential when working with documents that are all text, like a contract, where one page thumbnail looks like the next because there are no distinguishing graphics or pictures.

Tip 2 – Using Page Names to quickly identify the content of individual pages.

On multi-page PDF documents I can quickly identify a particular page by sight, (such as the signature page of a contract or the beginning of a topic), by using Page Names. Click on a document and select Page Thumbnails. Click on the Page Thumbnail you wish to name. Click on the Item Properties button on the command bar. The Item Properties will appear in the Function Pane to the left. There will be a Page (x) of (x) field where you can type a page name. Enter a relevant page name and click the Save button.

If you need to see the full page to properly identify what the page name should be you can also add page names using the PaperPort PageViewer. Double click to open the document in the PaperPort PageViewer. Click on the page you wish to name. Select File: Item Properties under the file menu. The item properties will appear as a floating dialog. Enter the page name in the Page (x) of (x) field and click the Save button.

Tip 3 – Quickly finding an important page within a document. (Searching for the exact location of an annotation within a PDF document)

If I need to find a particular page at a later date I will mark it with a standard annotation so I can search for it later. I use annotations that make sense to me such as “please review”, “please call” and “important”.

I use the “Note” tool in the PaperPort PageViewer to mark the pages with these annotations. I then add the file to the All-in-One search index. When I need to go to that page again I do a search using the All-in-One search to locate the file by using “Name, author, keyword…” search with only “Annotations” checked and “Use All-in-One index” unchecked. Even though I initially added the entire file to the All-in-One index I only want PaperPort to find my note annotations and not all my other files containing the same words as my notes. This will limit my searches to quickly find the specific files I am trying to locate.

Once the file is located, if the annotation is buried on a page deep within a long document, I double-click it to open it in the PaperPort PageViewer. I then select Tools: Find in the PageViewer menu and search the document for annotations only. This immediately jumps me to the page that I had been trying to locate.

Exploring The Phaeton Site Map

I’m sure this isn’t new, but I just saw it and thought it was worth noting: The VW Phaeton’s UK site has an interesting 3D Flash sitemap where the pages are viewed in slices, with different coloured dots representing different kinds of content (in this case factory or car):

Phaeton

Clicking on a particular page will highlight it; moving the mouse over a blob will bring up a particular item which you can then access by double clicking on it.

Strictly speaking this layout is too fancy, and the content too specific, for general use, but it’s intuitive enough to be a great way to show navigational information in three dimensional form. It might be a nice way to navigate back through old blog material, for example, with different colours for different categories?

Or does this go against the idea of trying to improve content and reduce complexity in design and layout?

This week’s column – The Mouse

This week’s Loose Wire column:

THIS COLUMN was going to be about how to get more out of your computer mouse. You know, clicking, dragging, double-clicking, dropping, all that kind of stuff. I was all fired up about it until I consulted the guy who had a lot to do with getting the mouse onto every desktop. That’s when I learned about how the mouse makes us endure more than we should. Here’s why.

Jef Raskin is a technology guru who was in the thick of it when the personal-computer revolution started. His home page features a picture of him, bearded and bespectacled, wearing what I assume is an eye computer. It lists his accomplishments concisely and modestly: creator of Apple’s Macintosh, the Canon Cat, click-and-drag selection . . . coined the term and the concept of “information appliances” . . . also well-known as an expert on the aerodynamics of miniature aircraft. More importantly, Raskin reckons the mouse is a waste of your time. “A message for your readers is that the overuse of the mouse is costing them about 25% of their time,” he wrote in response to my e-mailed questions.

Full text at the Far Eastern Economic Review (subscription required, trial available) or at WSJ.com (subscription required). Old columns at feer.com here.