Emacs, C-SPC, dead keys, and ibus

Some time before or during the GTK+ hackfest, Emacs on my laptop started acting strange. C-SPC wasn't working (a major catastrophe), and when using a Latin American keymap, it was spewing nonsense like <dead-acute> is undefined.

Today I finally figured it out, but just by comparing things between my machine-where-emacs-works and my laptop. It turns out that ibus had gotten installed on my laptop, and it was screwing things up.

TL;DR Emacs stopped handling C-SPC and dead keys; I uninstalled the ibus package and everything works again.

I have no idea of how Emacs users manage to live when they actually need to use ibus for input methods.

This is amusing and bitter at the same time.

Report from the GTK+ Hackfest

Last week we had the GTK+ Hackfest in the OLPC office in Cambridge.

My intentions for the hackfest were to finish the merge of GtkPlacesSidebar into the master master branch — it's a new public widget that will show up in GTK+ 3.10.

Over the past months I have worked on finishing the details of the sidebar: merging all the code from Nautilus, polishing the API with the help of the GTK+ team, writing reference documentation. During the hackfest I worked on some of the last user-visible details in the sidebar, particularly the way drag-and-drop feedback gets shown.

After I pushed the merge into GTK+, Cosimo gave me the green light for merging this into Nautilus, and so the thing is done now! It feels good to have shared code between Nautilus and GtkFileChooser at last.

Some things are pending:

• In the file chooser, when you click on the Trash item in the sidebar, you get an error message saying that "trash:/// is not a local file system". Even though files in the trash live in your normal Unix file system, GIO says that trash:/// is not native and doesn't have a local path associated to it. We have to see how to get GIO/GVFS to mount the trash into a FUSE mount automatically.

• There is still an ugly gtk_places_sidebar_set_show_desktop() API. I'll remove it and turn that into a GtkSetting, so that the sidebar can automatically adjust to the surrounding environment's policy about whether to show the Desktop folder or not (e.g. show it for XFCE, don't show it for Gnome).

• Add a GtkSetting for showing XDG directories (Music, Photos, etc.) or not. XDG doesn't want to show them by default, and prefers to let users bookmark those folders directly; Gnome does.

• Once GTK+ gets an API for notifications, the places sidebar should be able to notify you when a volume is being mounted (and is taking a long time to do so). Right now that code is disabled; it came from Nautilus, and it pretty much uses libnotify directly.

Meanwhile in Cambridge

For me going to the Boston area is a treat. It's a lovely city, with good public transportation, good restaurants, and with a manageable size.

I had a chance to meet Steve Branam, of the fantastic Close Grain blog — a fellow woodworker and software developer. Steve started a hand-tool woodworking school a while ago, so if you are in the Boston area and are interested in learning, shoot him an email.

Also, thanks to my friends Alán and Dori, for letting me crash at their house and keeping me well fed :)

Dmitry Orlov summarizes Mats Alvesson and André Spicer's A Stupidity-Based Theory of Organizations. A good afternoon read with lots of food for thought.

Functional stupidity is organizationally-supported lack of reflexivity, substantive reasoning, and justification. It entails a refusal to use intellectual resources outside a narrow and “safe” terrain. It can provide a sense of certainty that allows organizations to function smoothly. This can save the organization and its members from the frictions provoked by doubt and reflection. Functional stupidity contributes to maintaining and strengthening organizational order. It can also motivate people, help them to cultivate their careers, and subordinate them to socially acceptable forms of management and leadership. Such positive outcomes can further reinforce functional stupidity.

Have you donated to Yorba's crowdfunding campaign to develop Geary, a new, free software email client? I have.

As the Yorba people explained in their last keynote at GUADEC, they have been working on experimenting with funding models for free software. This crowdfunding campaign is the latest experiment, and I urge you to participate. Give them a little money. You just can't have people writing software full-time without them also having a way to make a living, and this crowdfunding campaign is the way they intend to make their living while they write the first versions of Geary.

Is this undermining Evolution? I don't think so. Evolution is relatively old software. It is really good, and also pretty bad at times — the results of lots of technical debt. Normally I would say, Yorba is crazy for writing an email client from scratch, and they should just give their love to Evolution. But they have proven themselves to be competent to write something like Shotwell, the photo manager I use every day, so this is good evidence that they can write Geary.

So, go to Yorba's crowdfunding campaign page and donate to Geary!

How I added a test suite for GtkFileChooserButton

GtkFileChooserButton is a quirky little beast. It is the widget you use when you have something like a dialog box to configure options, and you need a way to ask the user for a file or a folder.

Jim Cape wrote the original version of GtkFileChooserButton a long time ago, as a reasonably simple wrapper around GtkFileChooserDialog. You clicked on something like a GtkButton, which would bring up a GtkFileChooserDialog, and when the dialog was dismissed by the user, the button would then show the name of the selected file. Simple.

And then, things got more complicated. The underlying file chooser dialog became asynchronous. We replaced Gnome-VFS with GIO. GTK3 happened.

While I've kept GtkFileChooserDialog mostly in check (a debatable statement, given the number of bugs it has...), I've never really paid much attention to GtkFileChooserButton. It's a wrapper, so it should just keep working, I thought. But bugs appeared as the underlying infrastructure changed, and every time I fixed one, other new bugs would appear.

I never did a good job of testing all the subtleties of GtkFileChooserButton. So, this happened:

Resurrecting the tests

A long time ago I had a litle, ad-hoc, automated test suite for GtkFileChooserDialog in general. This was before GTestUtils got included in GLib. The code for the file chooser's tests got adapted to use GTestUtils as a testing framework, but since the tests failed, that code was left in the gtk+ source tree but never built.

A few weeks ago, when I felt shark teeth in my buttocks the extent of the brokenness of GtkFileChooserButton started to become clear, I started resurrecting the old tests.

My home-grown testing program initially looked like this:

static gboolean
test_filechooser_open (void)
{
    ...
    return passed;
}

static gboolean
test_filechooser_save (void)
{
    ...
    return passed;
}

static gboolean
test_filechooser (void)
{
    passed = test_file_chooser_open ();
    passed = passed && test_file_chooser_save ();
    passed = passed && test_file_chooser_select_folder ();
    passed = passed && ...;

    return passed;
}

int
main (int argc, char **argv)
{
    gtk_init (&argc, &argv);

    if (test_filechooser ())
        return 0;
    else
        return 1;
}
	    

This had several shortcomings. First, since the tests were hardcoded as a chain of "&&" operations, all the tests had to be run every time. You couldn't select each test individually. It was hard to see which test failed when one did.

When the tests were intially converted to use GTestUtils, it looked more or less like this:

static void
test_filechooser_open (void)
{
    ...
    g_assert (passed);
}

static void
test_filechooser_save (void)
{
    ...
    g_assert (passed);
}

int
main (int argc, char **argv)
{
    gtk_test_init (&argc, &argv);
    ...
    g_test_add_func ("/GtkFileChooser/open", test_filechooser_open);
    g_test_add_func ("/GtkFileChooser/save", test_filechooser_save);
    ...
}
	    

So, all the "return passed" became "g_assert (passed)" and each test got a human-readable name. With this you can pass an argument like "-p /GtkFileChooser/save" to the test program, and GTestUtils will just run that test. Also, when a test fails, you get its name printed on the console. GTestUtils is not the nicest testing framework out there, but it's much better than my old setup.

Tests for GtkFileChooserButton

My original tests weren't very useful for GtkFileChooserButton, so I added new ones. After writing some tests by hand with similar code between each other, I figured out that they are all a slightly different subset of these:

1. Create a GtkFileChooserButton in either OPEN or SELECT_FOLDER mode.
2. Set a starting filename or folder.
3. Invoke the underlying GtkFileChooserDialog.
4. Change the filename or folder of the GtkFileChooserButton while the dialog is active.
5. Confirm the dialog or cancel it.
6. Look at the final state of the button.

I replaced the hand-written tests with a generic "test everything" function, that checks the externally-visible state of GtkFileChooserButton after each of those key stages. The tests run many variations of the steps above. Some don't set up a starting filename, and others do. Some use the underlying dialog, some don't. Some confirm the dialog as if you pressed "OK"; some cancel the dialog, and thus GtkFileChooserButton needs to go back to its initial state.

The tests failed at first, of course, due to bugs in GtkFileChooserButton. But it was having the tests that forced me to understand exactly why the widget was failing in subtle ways. Having a way to test each assumption and every change made me get a deep understanding of how GtkFileChooserButton works very quickly. I now regret not having written those tests since the beginning.

What was hard to test

GtkFileChooserButton can work in two modes: one for GTK_FILE_CHOOSER_ACTION_OPEN, where it lets you select a single file, and one for GTK_FILE_CHOOSER_ACTION_SELECT_FOLDER, where it lets you select a single folder.

In OPEN mode, the button looks like an actual GtkButton that shows the basename of the currently-selected file. When clicked, it brings up a proper GtkFileChooserDialog. When the user dismisses the dialog, the button updates its contents to show the newly-selected filename.

However, in SELECT_FOLDER mode, the button actually uses a GtkComboBox. When clicked, it brings up a combo-box menu of Places (standard folders plus bookmarks). The menu also has an "Other..." item, which you can select to get a proper GtkFileChooserDialog to select an arbitrary folder. When the user dismisses the dialog, the combo box must be updated to show the newly-selected folder.

I wanted to test some things:

• When you call GtkFileChooser's methods, for example to select a file, does the GUI update accordingly? While you can easily call gtk_file_chooser_select_file(), there is no direct way of asking, "what text is displayed in this widget?".

• When the user presses on the button in OPEN mode, or in the combo box in SELECT_FOLDER mode, does the selection change accordingly? Testing the selection is a simple matter of calling gtk_file_chooser_get_filename(), but the first part of the question requires digging in the widget hierarchy and telling widgets to do things as if the user were driving them.

GUI testing projects like LDTP can more or less simulate a user driving the GUI by using accessibility interfaces. When a test wants to ask, "what text is this widget displaying?", it wants just to get the string, without caring whether the widget in question is a button, a combo box, an entry, or a label — and the a11y interfaces make this easy. Similarly, they make it easy to say, "activate this combo box, and select the Nth item", something which would be really cumbersome (and fragile) to do by simulating mouse clicks and grabs.

Since I had never used the accessibility interfaces to do anything, I ran Accerciser to see how to do operations on the GUI "by hand" with the accessibility APIs. For example, once you have a widget you want to query, you can get its accessible representation with gtk_widget_get_accessible() and then do atk_object_get_name() to see the text that the widget is displaying. Bingo! There is no need to see if the widget in question is a GtkButton, or a GtkLabel, or anything else.

So, currently, the tests for GtkFileChooserButton call ATK by hand and are written in C, as is the rest of the GTK+ source tree. I'd much rather have something higher-level like LDTP in place, which already has utilities to dig for widgets and activate them and dig for strings, all in various high-level languages.

Advancing tests based on catching signals instead of sleeping

The file chooser in GTK+ is highly asynchronous. When you call gtk_file_chooser_select_file(), that function will return quickly and the file chooser will start selecting that file asynchronously: it will load the directory, select the file, figure out its icon...

Regretfully, one anti-pattern in the file chooser's API is that while it is easy to know when an async operation starts, it is hard to know when it actually finishes. Although internally GIO has the right kind of async API (you start the async operation, you pass a callback to be called when the operation finishes, the callback gets called with a success or error value, you can cancel a running operation), the file chooser was originally not designed to be async. It had a synchronous, blocking API. When it was converted to be async, the original signals like selection-changed remained there, but we made no provision for those signals to return error values ("the file you wanted to select was not found"), or for operations to be cancelled.

Normally this is not a problem, and you can consider gtk_file_chooser_select_file() to be fire-and-forget — you call it when you are initializing your file chooser, and you only query the final filename when the user closes the file chooser. You never query it in between, and if the file chooser was not able to select the file your program wanted, it's not a big deal.

However, querying things in between is exactly what GtkFileChooserButton's test suite wants to do. Since my testing code could not know when the various operations were finished, it just had things like a sleep_in_main_loop() function that would wait for a fraction of a second before continuing. In theory, this would let the file chooser catch up. This is both fragile and slow: it is fragile if the tests are being run on a slow machine (or a slow VM), and it is slow because there is so much sleeping being done in dozens of tests.

To compound that, GtkFileChooserButton wasn't always proxying the correct signals from the underlying GtkFileChooserDialog. So, after doing an operation in the test code, all the code could do was wait for a bit before continuing.

I replaced the "sleep here" functions with a few others that actually check if certain signals have been emitted by the GtkFileChooserButton — signals that indicate that the file chooser has really loaded the file in question and that the async operations are finished. In the tests, there are calls to a function like signal_watcher_expect (watcher, "selection-changed"). The signal watcher notices if the signal has been emitted yet, and if not, waits for up to a predefined maximum amount of time before deciding that the signal is just not being emitted at all.

After doing that all the tests failed, of course — the signals weren't being emitted at the right times! So I set to fixing that. This made me re-understand the code of GtkFileChooserButton, and let me clean it up substantially.

After fixing things, the tests are able to run as fast as the file chooser button can confirm that things have happened. There is no sleeping anymore.

The tests took 71 seconds with the sleeps; they take only 21 seconds with signals only. Big win! And now GtkFileChooserButton's signals are actually reliable, and tests can be reliably run on any kind of machine. I hope.

Postscript, and help wanted

Finally, I would like to apologize to the people for whom GtkFileChooserButton has been so broken in the past. I hope that the tests will help keep things in check from now on.

The tests run out of the box in "make check" in GTK+ master, because accessibility is always on in Gnome 3. However, the test code does not work at all in the gtk-2-24 branch. I don't know how to enable a11y for the test program, or even to enable it for all GTK2 apps running in the desktop. If anyone knows how to do this, I would be very thankful to know.

Gnome-emacs-utils news

Dirk-Jan C. Binnema sent a couple of items to let Yasnippet (screencast) expand g_return_*_if_fail() when you type.

Alex Murray pointed out that DevHelp ships devhelp.el, by Richard Hult, to let you open the reference docs for a library function with a single keystroke.

Thanks, and keep the ideas coming, people!

git clone https://github.com/federicomenaquintero/gnome-emacs-utils

Introducing Gnome-emacs-utils

Today I was working on the Overview of the Gnome Platform, which is written in Mallard. I am a die-hard Emacs user, but a lazy one, and so I haven't kept up with all the relatively new things in Emacs: nxml-mode, the incredible Emacs Wiki, elisp package systems...

I became tired of Mallard documents opening up in plain nxml-mode, with no schema. So I learned how to teach nxml-mode about Mallard files and its RELAX NG schema. And I got to thinking why we don't have a well-known set of Elisp utilities for Gnome developers.

Back in the Gnome 1.x days, we had a wonderful little snippet of Elisp by Michael Zucchi, which would let you press a hotkey to add inline documentation comments for gtk-doc. If you had code like this and the point was at [*]:

GtkSandwich *
gtk_foo_make_me_a_sandwich (GtkFoo *foo, GList *ingredients, GError **error)
{
    [*]

Then you could hit the hotkey and this would be inserted before the function's prototype:

/**
 * gtk_foo_make_me_a_sandwich:
 * foo: 
 * ingredients:
 * error:
 *
 * Put the description here.
 *
 * Return value: 
 **/
GtkSandwich *
gtk_foo_make_me_a_sandwich (GtkFoo *foo, GList *ingredients, GError **error)
{

This elisp doesn't work anymore, sadly.

And so, I've started a little collection of Emacs stuff for Gnome developers. Let's tweak it to perfection.

git clone https://github.com/federicomenaquintero/gnome-emacs-utils

For the software-development-as-urbanism crowd: "When Tokyo was a slum" is an excellent article. It doesn't talk about software, but the process of construction of cities on a piecemeal basis is very illuminating.

When the war ended, Tokyo’s municipal government, bankrupt and in crisis mode, was in no condition to launch a citywide reconstruction effort. So, without ever stating it explicitly, it nevertheless made one thing clear: The citizens would rebuild the city. Government would provide the infrastructure, but beyond that, the residents would be free to build what they needed on the footprint of the city that once was, neighborhood by neighborhood.

[...]

The central part of the city is the historical core of Edo, which became Tokyo in the late 19th century. But the periphery grew largely without planning. Tokyo swallowed up the surrounding villages as it sprawled outward [...].

[...]

Throughout the 20th century, as surrounding villages were absorbed into the expanding city and rural villages became urban neighborhoods, their inhabitants preserved some of their customs and social organization. But the central government never saw the autonomy of these neighborhoods as a threat.

Does Gnome operate somewhat like that, and if so, what can we learn from the process? Is the center (the core desktop) traditionally the "capital", and has it swallowed little villages in the periphery (apps and libraries that became part of the core, or otherwise became tightly integrated with the core)? How much autonomy (let them be) vs. how much central planning (let's tackle horizontal issues)?

How do we remain civil as the city grows, when it is no longer possible to know everyone who lives there in person?

Jane Jacobs, in The Death and Life of Great American Cities, makes a point that one virtue of a great city is that it allows complete strangers to live very close to each other. A great city gets made through explicitly allowing and nourishing diversity: diversity of uses, so you don't have business neighborhoods which go dead at night; diversity of people, so any minority's needs don't go unmet; diversity of age in buildings, so people can afford a cheap shop or apartment if they need one, or a big and new building if they can.

Jacobs also talks about the "self-destruction of diversity":

[...] the tendency for outstandingly sucessful diversity in cities to destroy itself; the tendency for massive single elements in cities (many of which are necessary and otherwise desirable) to cast a deadening influence; the tendency for population instability to counter the growth of diversity; and the tendency for both public and private money either to glut or to starve development and change.

[...]

The purpose of recognizing and understanding [these forces] is to try to combat them or — better yet — convert them into constructive forces. Besides influencing the growth of diversity itself, these forces also sometimes affect the ease or difficulty with which the basic conditions for generating diversity can be introduced. Leaving them out of account, even the best planning for vitality would fall a step back for every two steps forward.

There is a lot of food for thought in all of this. See also, Photoshop is a city for everyone: how Adobe endlessly rebuilds its classic app for a tangible example of a large, old software project, that everyone seems to love and hate. This article also portrays its subject as a world-class city!

The Produce Savant, a wonderful blog about what to do with "unpopular" vegetables. By Sally of the amazing Tools for Working Wood.

2013 Developer Experience Hackfest, Brussels

I'm late to the blogging party about the Developer Experience Hackfest, which happened right before FOSDEM, but here it goes!

The Hackfest

We split into four teams: tooling and IDEs, application bundling and sandboxing, development platform, and documentation. I was part of the documentation team.

In the docs team, Allan focused on developer.gnome.org, our main web site for developers, while Meg worked on an introduction to the Gnome libraries. Fred, Thomas, Aleksander, and Jasper worked on documentation tools — DevHelp and documentation generation. I worked on updating our historic Overview of the Gnome Platform document, based on Tuğçe Şirin's analysis of what it lacks or what could be made more clear in it.

That document has a little history that is worth noting. Shaun McCance wrote it on comission from the Gnome Foundation in 2006. It is, to the best of my knowledge, the first time that the Foundation contracted someone for pay to do something related to development of Gnome. Shaun did an excellent job of summarizing the libraries we had available as part of the core development platform.

Over time, however, that document has gotten a bit outdated. People have kept it mostly up-to-date with respect to the libraries, but the document suffered from a bit too technical language at times, and from being a catalog of stuff rather than an introduction on how things work together.

I've been making some changes to the Overview of the Gnome Platform to add the missing platform libraries, to make the descriptions easier to understand, and to add a bit of a better structure to the document.

JavaScript

Practically any high-level, garbage-collected language is much better than plain C, in terms of productivity, once you get used to the idiosyncrasies of the bindings to the Gnome libraries.

So, I'm very happy that we have a high-level language accepted as officially supported for Gnome. I was kind of surprised that Python, our de-facto and historically "best supported binding" wasn't chosen, but if this means that another high-level language can get as much work put into it, then all the better.

Also, for JavaScript we have outstanding tutorials in Gnome Developer Platform Demos mega-document. Taryn Fox, Tiffany Antopolski, and other people who have worked on this have done an excellent job. See also the coordination for this in the wiki page for the Gnome Application Developer's Guide.

My hope is that we will get good tutorials for beginners, and that people will later take care of making sure that the examples are available in all our "good" language bindings, and that the text makes sense regardless of programming language. Gnome is a polyglot project, fortunately, and people will program in whatever language they find the most conveninent — which is perfectly okay.

Brussels in winter

Cold. Wind. Rain. Wind. Rain, and cold. The streets are littered with the corpses of umbrellas. My own didn't last one minute after taking it out of the backpack.

Still, it's worth it once you get to the pretty architecture, which permeates the center of the city. And a waffle warms you up. This is Juan Pablo Ugarte, from the Glade / IDEs team, a marvelous guide and conversation partner from Argentina, recharging his batteries.

Also, churros in Brussels? For 5€? What is this how can they I don't even.

Thanks

The hackfest took place at the Betagroup Coworking Space. This is a nice place! They have desks and chairs, coffee for which you pay based on an honor system, a kitchen, good networking, beanbag chairs, and it all leads to a very nice atmosphere. This is the first time I visit a coworking space, and it was very nice.

Thanks to the Gnome Foundation for sponsoring my flight/hotel!

Composting is awesome.

Humanure is awesome.

Community projects for composting are awesome.

But to fundraise on Kickstarter for a calendar with "Ladies of Manure 2012" is completely tasteless.

After a long delay, I've finished adding my speaker's notes to my talk from GUADEC, "Gnome and the Systems of Free Infrastructure". You can also download the ODP file.

The Battle for the Life and Beauty of the Earth: A Struggle Between Two World-Systems, Christopher Alexander's newest book, is just out. It describes the process of planning, designing, and building the Eishin School campus in Tokyo, while dealing with the Real World. I'm eagerly awaiting my copy of the book.

From the veritable "In the beginning was the command line":

Applications create possibilities for millions of credulous users, whereas OSes impose limitations on thousands of grumpy coders, and so OS-makers will forever be on the shit-list of anyone who counts for anything in the high-tech world.

And that was written in 1999. Plus ça change, plus c'est la même chose.