[The presentation starts on an OpenOffice Presenter title slide.]

"Linux Screen Reader ... is not just a screen reader." A presentation for the 2006 GNOME Accessibility Summit in Boston, MA. Narrated by Peter Parente.

[The screen switches to slide 2.]

Linux Screen Reader is an extensible platform for creating alternative or supplemental user interfaces. Extensions packaged with the LSR platform provide a screen reader experience for users with visual impairments. New extensions can be written, however, to create entirely different user interfaces.

During the next ten minutes, I will demonstrate three distinct use cases for the LSR platform. The goal is not to cover all of the features of each interface, but to demonstrate the flexibility of LSR.

[The screen switches to slide #3.]

Use case #1: screen reading. Roughly ten million people in the U.S. alone have a visual impairment. Modern desktop environments are inaccessible to many of these users without an alternative interface. 

The screen reader scripts for LSR currently provide a speech display with plans for supporting screen magnification and Braille in the future. Here is a quick demonstration of some of the screen reading features in LSR 0.3.0.

[The screen switches to a GNOME desktop. The visible windows are gnome-terminal, gedit, Rhythmbox, and a Gaim chat.]

I'll start LSR using the top panel.

[User runs LSR from the gnome-panel menu bar. The screen reader starts with the Festival speech synthesizer, announces its version, and starts to speak its revision date.]

And I stopped the introduction prematurely.

I can active the settings dialog to configure scripts, devices, system-wide settings, and the current profile. 

[The user presses the hotkeys to show the settings dialog. The screen reader announces all GUI interaction until the end of this segment.]

Let me set a different preference for my speech synthesizer in this profile.

[The user navigates the settings dialog and changes the speech synthesizer to Clique Audio.]

When I hit apply, the changes take effect immediately.

[The user activates the apply button. A new synthesized voice speaks.]

Now I can configure my synthesizer.

[The user navigates to the synthesizer settings panel and twiddles some of the controls.]

Perks, or script extensions that control interaction with the user, can have their own settings too. 

[The user navigates to the Perk settings tab.]

The Default Perk defines the basic screen reader interface. The Gaim Perk does some extra work to make Gaim more usable. Perks for other applications exist, but only these two define user configurable settings at present. I will enable background message reading in the gaim Perk before I close the settings.

[The user turns off and then on the background message reading option for Gaim.]

And now I'll OK the dialog. After OKing it, I'll take a walk around some desktop applications.

[The settings dialog closes. The gedit window gains focus automatically. The screen reader starts to read it before the user stops it.]

First I'll switch to the Rhythmbox music player.

[The user switches to Rhythmbox using Alt-Tab and navigates through two available songs.]

And I'll start this track playing.

[The user starts playing Beethoven's Symphony No. 8.]

Now I'll switch back to gedit.

[The user switches to gedit and moves the caret over a few lines.]

Navigation by line, word, and character work as you would expect.

[The user navigates by words and characters.]

Listen to what happens when I get a new IM, even when I'm still in the gedit document.

[A new instant message arrives from John Doe. The screen reader announces it.]

It's read aloud because I've enabled reading of background messages in the Gaim perk. When I switch to Gaim, the incoming messages are still announced even when I'm busy responding. 

[The user switches to Gaim and begins composing a response. While typing, a new message from John arrives. The screen reader reads the new message in parallel with announcing the characters the user is typing into his message.]

The two voice streams give me the choice of stopping to listen to a new message or continuing to type my own without one blocking the other.

[The user completes and sends his message.]

LSR 0.3 also has initial support for browsing web pages in Firefox 3. Here is the GNOME Live! headpage. [sic]

[The user runs Minefield using a desktop icon. The GNOME Live! wiki homepage loads first.]

I can use the review keys just like I can in other applications to browse the document.

[The user navigates through the page header and the first paragraph.]

And I can give a command to have LSR finish reading the rest of the page.

[LSR continuously reads the page.]

Let's stop here and continue with our other user cases.

[The screen switches to slide #4.]

Use case #2: memory aids. Two million adults in the U.S. have Alzheimer's, some form of senility, or dementia. Four million have some form of a learning or mental disability. 

Some of these people can benefit from automatic reminders about their daily tasks. For instance, a care giver might visit an older adult in the morning and create a todo list with times to take perscriptions, call the doctor, and perform household duties. But without prompting to check the list on a regular basis, the person might forget about important tasks.

A special purpose audio reminder application could solve this problem. But we can more easily write a LSR extension that turns any text area into a todo list with audible reminders.

[The screen switches to another GNOME desktop. A gnome-terminal window is visible and a gedit window is minimized.]

I'm now running LSR under a different user profile. Unlike the default profile, this profile contains none of the screen reader extensions, only those for showing the LSR configuration dialogs.

[The enters the command in the terminal to start LSR under the demo1 profile.]

I can bring up a Tomboy note which has a list of my daily tasks and times when I should start them. 

[The user shows a Tomboy note called Pete's schedule].

I can now ask LSR to show the Perk dialog. In this dialog, I can choose to load the Reminder Perk for this particular window. 

[The user shows the Perk chooser dialog.]

I can go through the list of Perks. There it is at the bottom. And I select it to load and OK the dialog.

[The user completes the dialog.]

There's nothing special about Tomboy. gedit, OpenOffice Writer, an email in Evolution would work equally well. I can also set the Reminder Perk to load automatically when I start LSR with a particular user profile. For now, though, loading it manually this one time will suffice.

I can bring up the LSR setting dialog and configure the Reminder Perk.

[The user shows the setting dialog.]

I see in the Tomboy note that my next task doesn't start for about another half hour minutes. Let me change the settings so that the reminder Perk announces Tasks at least 45 minutes ahead of time. I'll leave the notification repeat at every 1 minute so we don't have to wait too long.

[The user changes the upcoming reminder option to 45 minutes instead of 10. Then he completes the dialog.]

While we wait for our reminder, let's quickly look at the code for the reminder extension.

[The user restores the gedit window. At the same time, LSR plays a sound while announcing the names of the two upcoming tasks and how many minutes remain until they start.]

And there's the reminder.

The first class defines the user configurable settings for reminders. Notice there's no GUI code, the settings dialog generates the interface automatically based on the kinds of settings. The next class initializes a timed interval for checking the schedule. The last class has methods for parsing the todo list and playing sounds and speech as announcements. It's less than 200 lines with comments, five user configurable settings, and support for speech and sound output.

[The screen switches to slide #5.]

Use case #3: Reading Aids. 800 million adults world wide are considered illiterate. 4.6 million children in the U.S. have a form of a learn disability. People in both groups might benefit from visual aids during reading. 

Again, we could write a special purpose document viewer that pops up images when the user selects unknown words. But we can write an extension for LSR that does the same thing and works with any application that supports text selection.

[The screen switches to another GNOME desktop. A gnome-terminal and a gnome-dictionary window are visible. A gedit window is minimized.]

I'm now running LSR under yet another user profile. This demo2 profile automatically loads a Perk for all applications. The Perk watches for a hotkey press, grabs the selected text in the active application, queries the Flickr image sharing service for relevant images, and shows thumbnail results to the user. 

[The user starts LSR from the gnome-terminal under the demo2 profile. The user switches to the gnome-dictionary, enters the word "marsupial", and highlights that same word in the results.]

Here in the GNOME dictionary, I can search for the word "marsupial", select that word in the results, and press the hotkey.

[A window named "Marsupial" appears with an image of a marsupial skeleton.]

In a moment, a little window appears showing me an image of a marsupial. I can use the little arrows to flip through other possibly relevant images.

[The user presses the next area to show the picture of a kangaroo in a cave, a man with a dog hanging out of his jacket pouch, a statue of a marsupial mascot in a city, a close up of a small marsupial, and a kangaroo standing behind a stone column.] 

This same Perk works equally well in other apps. Here in gedit, I can select some words in a story, press the hotkey, and see the results.

[The user switches to gedit. The story of Jack and the Beanstalk is loaded as a text document. The user selects the word "market" in the text.]

Here is a market.

[A window with the title "market" appears showing a picture of an Asian market. The user presses next to show another picture of the same market.]

If I select the word "butcher" and do the search, some pictures of butcher shops come up.

[The user selects the word "butcher." A window named "butcher" appears momentarily. The user flips through pictures of the inside of a butcher shop and a picture of a brick building some white bricks spelling Wright Butcher.]

The word "beans" since this is a story of Jack and the Beanstalk.

[The user selects the word "beans." A window named "beans" appears showing an image of bean pods growing appears.]

Of course, querying Flickr doesn't guarantee relevant images. But one could extend the code in the Perk to improve the search parameters, look at open clip art sites, or even use a local image repository provided by the user, a teacher, a care giver, etc.

[The screen switches to slide #6.]

Some final thoughts. 

While our efforts are focused on improving the screen reader experience for LSR, we encourage interested developers to explore new uses cases. For instance, the memory and reading aids are merely prototypes, but a little love from the community could take them a long way.

Finally, ideas for extensions can go beyond accessibility. For instance, spoken todo list reminders might prove attractive to Evolution users. As another example, audible alerts for certain events in a web server log might be useful for server administrators.

[The screen switches to slide #7.]

For more information about the LSR project, please visit http://live.gnome.org/LSR.

Thank you for your time.

[The screen switches to slide #8. The slide contains the following Creative Commons attribution information for all of the images and sounds used in the demonstration.]

Sampling+ 1.0
chimes by psakion, claxon by han1

Attribution-ShareAlike 2.0
Marsupial: ryarwood, orinrobertjohn, romych, tastefulTN
market: gerrypopples
butcher: hugovk
beans: cobalt123

Attribution-NoDerivs 2.0
Marsupial: microbophile
butcher: Mr.mt

Attribution 2.0
Marsupial: Drake LeLane