Go forward in time to December 2006.
Thanks to the people who replied to my question about turning DocBook into PDF. The standard procedure seems to use the XSL stylesheets to turn DocBook into xsl-fo, and then something like fop to render the final document.
Dear Documentation Hackers of the Lazyweb,
A while ago someone posted a link to some beautiful PDFs generated from the GNOME manuals. I have chunks of DocBook which I want to turn into similarly beautiful PDFs. How do I do that?
On my laptop with GNOME 2.12 (SLED 10), gnome-settings-daemon starts up with a heap of 1132 KB, of which 1004 KB are resident. On my desktop with GNOME 2.16 (openSUSE 10.2), it starts with a heap of 1300 KB of which 1176 KB are resident. That's a difference of 1173 - 1004 = 169 KB.
In turn, gnome-settings-daemon-2.12 has 1968 KB of dirty memory, while 2.16 has 2308 KB. That's a difference of 340 KB.
After I leave my laptop running for a while, some of gnome-settings-daemon's heap gets swapped out. It ends up with 276 KB of resident heap: all the rest is not part of the working set and it remains swapped out. What is all that crap doing in the heap of gnome-settings-daemon? Why don't we free it as soon as possible?
Argh, I'm an idiot. Paolo pointed out that is_local() thing is already fixed on CVS. Note to self: when you find a bug, check the latest code first!
SMB slowness and is_local() roundtrips in Nautilus
At Novell we have customers with huge Windows networks using Active Directory. They invariably complain about how Nautilus gets really slow when browsing SMB shares in the network. There are two parts to the problem.
First, libsmbclient from Samba provides you with a SMBCCTX object, which is the main way in which you access functions in libsmbclient — "SMBC" stands for "SMB Client", so SMBCCTX is a SMB Client Context. You create a SMBCCTX object, and then do "handle = ctx->open (ctx, "smb://...", ...);" or "num_read = ctx->read (ctx, handle, buffer, 42);". However, if you have multiple threads, only one thread at a time may call into libsmbclient: the library is not MT-safe. Fixing this shouldn't be too difficult if you want to make SMBCCTX objects MT-safe: you can simply hang all the state you need from the SMBCCTX object, instead of using global variables. It's just a lot of grunt work in the Samba code.
The second part of the problem is that gnome-vfs uses a single mutex around all calls to libsmbclient. It must do this because libsmbclient can't be called from multiple threads at once.
Regardless of that, I wanted to see how much contention there is on that mutex, i.e. on average, how long Nautilus has to wait for one SMB operation to complete before the next one is processed. Like all graphical file managers, Nautilus is very "chatty" with the file system: it has to fetch directory lists, stat() each file to get its metadata, sniff some files to figure out their MIME type, etc. Add that chattiness to the fact that we are accessing SMB files over the network and that we are constrained by a single big mutex, and you can appreciate the extent of the problem.
In gnome-vfs, the code that calls into libsmbclient lives in gnome-vfs-daemon. So, I added code to the gnome-vfs library so that it will give me timestamps when calls into the daemon are initiated and finished (gnome-vfs2-log-daemon-client-calls.diff). I was hoping that I would see a pattern like this:
operation 1 issued operation 2 issued operation 3 issued operation 4 issued ... big gap ... operation 1 finished ... gap ... operation 2 finished ... gap ... operation 3 finished ... gap ... operation 4 finished
That is, I would see that Nautilus send several asynchronous requests to the daemon, and since these have to be processed serially, there would be big gaps of time between each result.
Instead, I got a big surprise:
thread--- timestamp--------------- operation------ serial- URI------------------------- 0x80fb2a0 2006/11/14 15:34:43.6797 (GLog): IS_LOCAL issued (1166): smb://champignon/documents 0x80fb2a0 2006/11/14 15:34:43.6819 (GLog): IS_LOCAL replied (1166) 0x80fb2a0 2006/11/14 15:34:43.6824 (GLog): IS_LOCAL issued (1167): smb://champignon/documents 0x80fb2a0 2006/11/14 15:34:43.6827 (GLog): IS_LOCAL replied (1167) 0x80fb2a0 2006/11/14 15:34:43.6827 (GLog): IS_LOCAL issued (1168): smb://champignon/documents 0x80fb2a0 2006/11/14 15:34:43.6829 (GLog): IS_LOCAL replied (1168) 0x80fb2a0 2006/11/14 15:34:43.6835 (GLog): IS_LOCAL issued (1169): smb://champignon/documents 0x80fb2a0 2006/11/14 15:34:43.6838 (GLog): IS_LOCAL replied (1169) ...
There is a *ton* of calls to IS_LOCAL, and all for the same URI! And if add the differences between the issue/reply, I get that we spend 0.31 seconds doing roundtrips. Now, this is not a lot in the grand scheme of things, but the whole network directory took about 2 seconds to load. So, we spent about 15% of the time computing the same thing over and over again. What is going on?
Is_local() is a very important operation in gnome-vfs. Roughly, you use that function to know whether a file lives in a local file system or in the network. Nautilus uses this function to follow the "create thumbnails for remote files" preference: thumbnailing remote files is slow, so we need to know whether a file is local or remote.
I set a breakpoint in gnome_vfs_uri_is_local() to get stack traces when it gets called. This would let me see where Nautilus calls the function, and more importantly, why it was querying the same URI so many times. The traces look like this:
#0 gnome_vfs_uri_is_local (uri=0x8307fc0) at gnome-vfs-uri.c:1283
#1 0xb7ef3f51 in nautilus_directory_is_local (directory=0x8724748) at nautilus-directory.c:545
#2 0xb7f07d22 in nautilus_file_is_local (file=0x87488e8) at nautilus-file.c:1411
#3 0xb7f2e56d in should_show_thumbnail (file=0x87488e8, mime_type=0x82f8080 "text/plain") at nautilus-icon-factory.c:797
#4 0xb7f2e817 in nautilus_icon_factory_get_icon_for_file (file=0x87488e8, embedd_text=0) at nautilus-icon-factory.c:880
#5 0xb7f2fa2e in nautilus_icon_factory_get_pixbuf_for_file_internal (file=0x87488e8, modifier=0x0, size_in_pixels=24,
force_size=1) at nautilus-icon-factory.c:1565
#6 0xb7f2faed in nautilus_icon_factory_get_pixbuf_for_file_force_size (file=0x87488e8, modifier=0x0, size_in_pixels=24)
at nautilus-icon-factory.c:1597
#7 0x080d0e7d in fm_list_model_get_value (tree_model=0x82f7e58, iter=0xbfdfa6cc, column=3, value=0xbfdfa5cc)
at fm-list-model.c:306
#8 0xb79c1985 in gtk_tree_model_get_value () from #/opt/gnome/lib/libgtk-x11-2.0.so.0
NautilusFile is the main abstraction for a file in Nautilus, and nautilus_file_is_local() is the way you ask a file object whether it is local or remote. That function is implemented like this (details omitted):
gboolean
nautilus_file_is_local (NautilusFile *file)
{
return nautilus_directory_is_local (file->details->directory);
}
Okay. So, a file is local if its parent directory is local. Let's look at that one (details omitted):
gboolean
nautilus_directory_is_local (NautilusDirectory *directory)
{
return gnome_vfs_uri_is_local (directory->details->vfs_uri);
}
Aha! There we go. The fix is simple enough: add a boolean field to the NautilusDirectory object so that we can cache the result of gnome_vfs_uri_is_local() (nautilus-directory-cache-is-local.diff). This works since a directory never changes its "local-ness", so we need to call is_local() just once.
This is not the biggest part of the "SMB is slow" problem. But it's nice to see that there is low-hanging fruit around it. The next step is to actually fix Samba's libsmbclient and gnome-vfs's smb-method.c.
For unfathomable reasons, sweet lemons are rather hard to get in Mexico for most of the year except during November/December. This is when they get used as filling for piñatas. Lemons are also a basic ingredient in sopa de lima, from the cuisine of the state of Yucatán.
Yesterday we decided to make a good Yucatecan dinner. First we made sopa de lima, which is chicken broth with tomatoes, shredded chicken, strips of fried tortilla, and lots of lemon slices. You add red onions with habanero peppers for spice.
Then we made papadzules. These are tacos filled with hard-boiled egg, and then dipped in a sauce made with pumpkin seeds and epazote herb. Then you make another sauce with onions, garlic, tomatoes, and habanero peppers, and use it for decoration and flavor.
Finally, we made fish in poc chuc style. This is basically any kind of meat marinated in achiote sauce, with salt, pepper, orange juice, and vinegar. Then you grill it along with orange slices and red onions.
Animations: From Cartoons to the User Interface. It has some nice ideas on how to make transitions and movements in user interfaces seem less surprising/drastic/machine-like.
Background
In pretty much all the elections for the Board of Directors of the GNOME Foundation, it has been inevitable that the "rock stars" who nominate themselves are eventually elected. The rock stars are the most prominent hackers who contribute to GNOME. They know that their contributions are valuable, and they wish to give even more of their valuable time to further GNOME's goals: this is why they nominate themselves for the Foundation's Board of Directors. And since everyone in the Foundation knows who they are, they get elected.
I have been a Board member of the GNOME Foundation for three or four non-consecutive years. I am also a pretty visible member of the GNOME project, and a busy hacker at Novell. My personal experience in being a Board member is that I never have time to do all the little administrative things that are part of being a Board member, and I end up feeling terrible about myself.
This is not a problem only for myself. Other Board members, if they happen to be busy hackers, also suffer from this problem. They never have time to do all the little things that one must do for the Board, and the Board is seen as a nebulous entity that accomplishes nothing.
Would you be a good member of the Board?
If you are a rock star hacker (or a busy non-rock star hacker at work), you will not be a good Board member. Don't think that you can squeeze in a couple hours each week; you won't be able to. In the Board you have to do little tasks like answer mails, take minutes, send minutes to the public, be in contact with the companies in the Advisory Board, make plans, etc. If you wouldn't normally have time to participate in a volunteer organization where you do paperwork, the Board is not fit for you.
If you are a hacker, your GUADEC experience will be destroyed. GUADEC is about meeting fellow contributors and talking to them in person. It is about having lunch with them, discussing the technology or your favorite brand of beer. If you are a Board member, during GUADEC you will be in meetings literally all day and you will not be able to talk to fellow hackers. Forget about the parties; you will be too tired to go to them if the meetings are even over by then.
You need 1 hour every two weeks for the regular meetings. You need about 1-2 hours each week to deal with minutiae: your assigned actions, writing mails to people, figuring things out. But this is not 1-2 hours of well-defined work. It's a big context switch from your regular activities, and if you don't like interruptions for paperwork, you'll dread having to do it. If you are the kind of person that forgets what you talked about two weeks ago, you won't be a good Board member.
The Foundation will not reimburse you for the conference calls. The calls are invariable hosted in the USA, so you may have to do long-distance calls every two weeks. We haven't done any work to make the calls through VOIP or anything.
So who would be a good member of the Board?
People who have had experience in running other volunteer organizations.
People who have had experience in administering other people's money. I.e. you could be the Treasurer for the Foundation.
People whose daily jobs don't extend beyond "normal work hours": hackers tend to keep working on the stuff from their job even after they go home.
People who are good at paperwork, following up in communications, and who don't mind frequent meetings.
But don't let me discourage you
I am not trying to discourage all hackers from nominating themselves for the Board elections. I am just pointing out why being a Board member is not the best idea for everyone.
Remember that the last day to submit your candidacy is November 16!
More fodder for the discussion, this time from the Microsoft side (via Kurt Pfeifle).
Dear Lazyweb,
I'm writing a little library with Mono. Does anyone have a Makefile.am skeleton to integrate NUnit tests into the build process?
Emmanuele Bassi is a hacking machine who is fast becoming one of the GTK+ superheroes. Not only did he wrote the GtkRecent set of classes, used to manage the list of recently-used files; he's now hacking on GtkUnique, a way to have single-instance applications at the GTK+ level.
The Global Technology Revolution 2020, In-Depth Analyses: Bio/Nano/Materials/Information Trends, Drivers, Barriers, and Social Implications. Report from the RAND Corporation.
Same as always: the countries which actually invest significant money in science will be able to take full advantage of new technologies. Countries like Mexico and Brazil will only adopt a few interesting scraps. Africa will remain fucked.
(So the USA government pays RAND to think and make plans. I wonder if anyone in Mexico is paid to think about the future, or if we are simply coasting along...)
Go backward in time to October 2006.
Federico Mena-Quintero <federico@gnome.org> Sat 2006/Nov/04 18:40:00 CST