• Still feeling dead, heavy breathing, lem-sipped; checked mail, fixed an OpenSUSE / OO.o crash-on-start related to some GIO / GVFS confusion. Irritated by under-performing calc save I did some callgrinding, and found what looks like ~10% of the time taken up in chasing (non-existent) symlinks in the 'store' code - what a crock, usually back-to-back in batches of 4 duplicates, and of course in the huge IMPLEMENTATIONS/ directory with the hundreds of components this is not good. Knocked up a patch.
• Call with Greg, lunch, still feeling awful. Poked at Jon's gdm mail - wow, he wrote the patch for me - nice, tested it - saves me ~10seconds on autologin, great work. Discovered that the nice interaction diagrams I needed to understand gdm are in fact already drawn and in the wiki. Chat with Jono. Cold moving to the chest at speed.
• Charl came over to go out to the pub, but so wet, and ill that we just slugged by the fire instead; really good to talk with him.

• To work; mildly amused by the Ubuntu hype-machine's success; I read somewhere (with no hint of irony)

  [Mark's] ... goal is to push the Ubuntu GUI up to the level of Apple's ... It's possible that sentiment has never been voiced in the history of GNU.

  Strange, I thought I heard this aspiration regularly, throughout GNOME development, and even by RMS back in the day and wow, reading that controversy - I yearn for the old days: "In short, I'm a huge believer in an open meritocracy that encourages bottom-up development instead of forcing top-down "standards." Like CDE." - there's still perhaps a lesson for OO.o there somewhere.
• Claire came to visit from Coventry with baby Jared. Dead pleased to hear her ask So what's a PC ? in relation to the recent Microsoft advertising campaign - seems like they have a cliff-face of work ahead there; perhaps a bring-back-Jerry campaign is in order. Lunch, discovered not having rsync installed stopped me up-loading my blog for a while, doh.
• Call with JP, poked at odd shell-scriptage around login, ugh. Rather woozy with flu, some amusing fading in and out. Prodded MA about our tennant. Read analyst reports.
• Dinner with Sharon, Luke & Cassy, Claire & Jadon, and our lot - fun. Put most of them to bed. Stayed up chatting happily to Sharon by the fire.

It seems that MS are entering the hosted Office space with Office Live, Q&A, Video (hover over the downloads link). Here are some of my initial thoughts, at least crystallising my dislike of some of the 'cloud' computing hype.

Architecture

This appears to involve putting a full-blown instance of MS Office on the server, per user - this is (presumably) one of the reasons they are on a vast server farm building spree: and who can blame them - running at least one full copy of office on the server per user is going to be incredibly expensive.

This at least lets them expose the full feature set of MS Office, and makes it something I'd want to use - the down side being things like those Excel no more limits features mean a casual user can swallow gigabytes of server RAM fairly trivially:

Total amount of PC memory that Excel can use
Old Limit: 1GB
New Limit: Maximum allowed by Windows

I hope that works out well for whomever happened to get dropped into sharing the same server as the heavy spreadsheet user.

What about the web technologies ? - apparently this is not Silverlight only, though there seems to be an acknowledgement that if you want a version that performs well, you will need that, otherwise (reading between the lines) I suspect it's a lot of bitmaps, lag and server side rendering.

The exact technology mix doesn't seem to be public yet, but the MS Office code-base is mythically large and twisty. Having said that they seem to have a Model/View separation clean-up underway, unless the live collaboration is some grisly hack. I strongly suspect, where claims of perfect visual fidelity (6:05 into the video) are made, that this is a StarPortal model, or perhaps even more basic - with EMF+ or even an RDP-like protocol being used with a -very- thin client, ie. a model akin to Ulteo's embedded-Java/VNC style setup.

Of course, this type of architecture is really great for getting apps onto the web fast, and sharing that code-base with the fat client, but ultimately can never allow dis-connected operation. Then again, for large complex applications I've never believed the "re-write everything in JavaScript with Gears" mantra (after all, we've not yet finished the re-write of everything into Java yet), and it seems (to me) an expensively lame solution to the simpler deployment problems the web is supposed to solve.

Normal Web problems

This of course ties into the problem of payment - sure, in this world people can save money by buying some trivial piece of hardware, and running just Firefox on it; but sadly - unless money can be made simply by competing to give things away faster than Google, or by advertisements: someone has to pay for 10k new machines per month, and worse the electricity to run them. Is there some corresponding, functioning micro-payment / metering scheme to make a business model fly here ? and how does the transition to that work ?

Then of course, service level agreements tend to be an issue - particularly in the presence of the known pathological resource sharing problems. Of course, service wise - there needs to be some really good sand-boxing to isolate everyone from the next MS Office binary filter vulnerability, and no doubt there remain many of them to be discovered.

To overcome some of these problems, and the potential confidentiality / compliance issues people can run this on their own Sharepoint server(s), then, if you're not careful this begins to look like some of the lamer desktop 'virtualisation' solutions that essentially are just an exercise in PC movement: all the PCs move into the data-centre: but you have just as many of them, perhaps at greater expense, but at least they sit somewhere else. Still - that should make hardware manufacturers salivate I guess.

Of course, in the browser world, as I understand it, there are a host of evil problems with missing open standards and ergonomics around interacting with local devices, and exposing those back to the server: USB keys, printers, printer-quirks-and-settings, 3D acceleration etc. Perhaps many of these can be overcome with Silverlight, but no doubt eventually deploying and updating increasingly complex fat-client technologies starts to eat into the 'reduced deployment' rational for all this work in the first place. It reminds me of our old XMS system (written in Java), that only worked (or was certified / supported) with Java 1.2.3.4-5.6, and only on Windows, and only in IE, such that everyone had to use Citix to access it on a Windows cluster in Provo.

Win32 for the Web ?

If my guess is correct: that this is some very thin RDP-like layer to a fat client on a server - and indeed for eg. VBA macros, Excel analytics plugins etc. to work well this is basically necessary (since they can use any part of the Win32 API) then life is interesting. It would suggest a possibility to extend the life of the Win32 APIs to become a 'web' application framework: if so, surely ~all windows desktop apps can be "Webbed" in a similar way. If my guess is not correct, then at least some degree of embarrassing retraction wrt. the functionality available in the web version will be due soon, and/or some wholesale feature axing for Office 14.

Cross Platform

Does this mean Microsoft is finally shipping Office for GNU/Linux ? lets see how it performs there, I guess this is at least better than nothing for the last few percent scared of the GNU/Linux desktop and OpenOffice, and it'll be interesting to see if MS bothers sustaining their increasingly creaky Mac version: apparently the Web version on Mac will be more feature complete, albeit less 'beautiful'.

Summary

This is a smartish move by Microsoft. It will make, thanks to Miguel's prescience, MS Office available on the GNU/Linux desktop. However it will cost Microsoft a fortune in server hardware and electricity, and there are formidable problems around metering and managing the live service, particularly against a leaner, simpler free (beer) rival in Google. Of course, as soon as the half of computer users with laptops go dis-connected, or catch a flight (when can we expect high-bandwidth, low latency transatlantic internet services ? or even ubiquitous in-seat power ?) - they will have to use OpenOffice anyway, oh, and their portable hardware will need to be beefy enough to be capable of running that, so - why not author it there in the first place ?

This is a problem for Free software - traditionally it has been hard to fund even simple and lightweight shared services (eg. freenode) - never mind server computing on this scale. This is an architecture only giants can play with, as such there is much hope that it will come horribly and expensively un-stuck. It is yet more of a problem as RMS has pointed out because people have no control over their software - they surrender all their freedoms (if it is even implemented with Free software) to some all-knowing hosted provider.

A great Free-software response in my view is to work on adding collaboration features using existing protocols, I'm a fan of bootstrapping from existing IM services with Telepathy tubes; and sticking with the fat clients. That keeps your data local, and shares it only transiently with people you trust, and it also requires little-to-no server load, just bandwidth. We should also keep adding features that (so far) are not going to work well in the 2D fat-web space: eg. get some more sexy 3D transitions going, and better native Mac integration.

Finally - poor ideas don't die, they come back to haunt us; the idea that there would be a few large powerful computers with lots of terminals is a perennial, and goes in cycles. In the past it has foundered on the rocks of Moore's law - it turned out to be cheaper, easier and more reliable to put almost all the software, data, and processing on the local device. Enabling dis-connected operation will still mandate beefy thick clients. I'm optimistic that the trend will continue - eventually even for Mobile devices, last time I looked Google develops and deploys some fat clients.

I look forward to trying the tech. preview at the beginning of next year to corroborate my suspicions, or the final product in Office 14 in 2010 ?.

• Up early; prodded mail, architects call. Submitted Ricardo's latest set of yast2-gtk fixes after persuading osc to submit to me. Fun packed morning of expense filing. Beta customer call, helped isolate the Intel / 3D problem based on Stefan's inspiration. Charl and San Marie around for dinner.

• To work, poked at mail, quick but interesting call with Pavel M. Dug at the go-oo website a bit, call with Jeff, Clarity. Poked at some split build issues, clobbered a daft OO.o warning. Interrupted by the 'Peatle' (M.) curious as to whether I was talking on the 'Hello-phone' - luckily not.
• Committed an apply tweak to accelerate split build patching pleasantly.

• Up in the night with exhasperating lack of sleeping daughters. Day off - breakfast, out swimming in the local pool, while E. slept. Lunch. Out to Aldeburgh's playround in the afternoon - lots of swinging & sliding fun.

• Up early, J. lie-in, packed everything into the car - remembered the hour change; off to NCC, DT spoke on Judges: Deborah / Barak. Caught up with Jon, Troy & Leeann.
• Drove to Bruce & Anne's in the rain, appreciating the faster / upgraded A14. Played with babes indoors much of the afternoon.

• Up late, lunch, baby-sat while J. took H. to a party, then took N. to her first one on her own (great excitement in advance, and apprehension while there). Picked up by J. - N. managed to wedge herself in the boot in the seconds she was away from the car.
• Home, frantic cleaning, the massed Webbs & Bancrofts arrived - really great to see them, broke out the apricot wine, and stayed up talking to Hannah & Nick.

• Took H. to school, back to mail. Bashed out a LXF column, filed bugs variously. Please by the BBC's Michael Sparks's Kamaelia project for making good use of threads in Python - seems like a good approach.
• Up-loaded my slides from Linux Expo Live. Lunch. Back for more mail catchup, bug bits, and boot profiling. Tried to get my needinfo problem under control, generated lots of info.
• Out for a Sad Dads drinking evening in town, good fun, but we're all lightweights it seems; back rather too late.

• Up all to early & off to Linux Expo Live - remarkably fewer exhibitors and attendees than expected. Caught up with Roger & Malcolm. Had an interesting panel discussion, and gave a quick Linux Desktop update.
• Annoyingly compiz and metacity seem to rather like trying to run at the same time - which gives a somewhat surreal double window decoration experience; must get bug filing.
• Enjoyed part of Ulrich's paper (pdf) What every programmer should know about memory - whatever his attitude to glibc contributions, the man is a masterly writer of papers.

• Brief mail touch, on to conference prep; frantic slide drawing much of the morning and afternoon; interspersed with OO.o team and customer conf-calls. Dinner, worked late on slides.
• Apparently it's a surprise to some people that employees file patents; I happen think that's a wise precaution, particularly since Novell uses them only defensively - prefering to compete on technical merit. Of course, ultimately software patents are bad news, my short lobbying experience against them (in Europe) was highly interesting - particularly inasmuch that some of the UK Patent Office appear to be Linux users.
• Dug out the USB ID for my ultra-cheap Alba mp3 player, fiddled with a .fdi file and lo - it works with Banshee just as nicely as the IPod, filed patch; apparently lsusb sees it as a SigmaTel device.

• Up early, shunted mail; prodded my OO.o build, it seems the gcj build option (I was using by accident) has bit-rotted in some unfortunate way (to do with imports and ridljar); switched to openjdk for some more joy. Poked at a Beta3 pre-build live-CD, filed a few bugs.
• Apparently some people mis-understood the nn vs. kohei graph perhaps due to the small scale, the time period (as labeled) is since 1st Jan 2006 - ie. 30+ months. Some others moan about the focus on code which of course excludes translation; clearly if you include those changes we might see lots of people from all over the world contributing l10n, (which of course is great in itself). Why exclude l10n ? Primarily because, though it is immensely valuable work, without improved software to translate, the task eventually runs into a wall of 100% completion (which must be nice). L10n is a derivative task of developing the underlying software: if the l10n change rate is dropping, is that because we are nearing complete translation ? or is it that there are fewer people contributing ? hard to say & potentially misleading. Secondly - the necessity of a fair comparison, in this case with Linux: which has little-to-no l10n component. Finally, as I understand the mechanics: l10n is often proxied via release engineering into the source tree, meaning the data as to whose hard work each string was is lost. Clearly the process of getting a reasonable view of the data includes winnowing out various classes of things: eg. auto-generated source files, or HTML web-pages (which tend to get checked into CVS), and so on - clearly that is no surprise.
• Played with multi-head some more; discovered ~/.config/monitors.xml is worth backing up after getting it right (with xorg.conf).

• To work, still pretty groggy. Sexy new SanDisk SSD arrived to play with, nice ! Re-read SUSE bug definitions - perhaps my bugs are not all so critical. Chewed mail over.
• More work prepping for Linux Expo Live later this week. There is -so- much work going into the Linux desktop, it's really relatively hard to summarise. Lunch, clarity. Chat with Guy.

• Up early, dealt with babes. Off to NCC, manned the creche with Ruth. Back for lunch with Martin; felt terrible most of the afternoon, nasty cold, tried to sleep on the sofa. Bed early.

• Up late, held the fort with the babes, while J. went to shop, delayed by racing traffic and the ongoing clock-tower traffic stupidity. The new dig up the roads scheme for reducing congesting traffic.
• Lunch, children played in the garden. Watched Rattatoille on the wall with them; Tea, and packed them off to bed. Out to DT & Zoe's for a birthday party - lots of fun.

• Really pleased to see Peter's blog about AEGIS, and good to see Cambridge involved too; good to see some more millions poured into making our a11y even more effective - clearly some great people involved there.
• Prodded mail; spoke to 'Fan' at the Hotel in Beijing to get a reservation, and confirmation number; finally got and printed that. Churned through older E-mail, cycled off to post my passport to CIBT.
• Interesting to see Thorsten's patch statistics for up-stream OO.o. These were not included in my contribution numbers, and are interesting enough. I'm not convinced the apparent recent decrease in submitted patches is statistically significant either, indeed the metric itself is pretty weak. Having said that, it's great to see Caolan (RedHat)'s heroic effort across the board (and in particular removing unused code) recognised with 1st place by miles; if there were ~550 patches filed so far in 2008; Caolan carried nearly 30% of the work by himself, with another 10% from the Novell guys in the top 10 [ pmladek, kohei, thb, kendy ] - and a great showing from Pavel (l10n++) & Tono (on the mingw port I guess). It's a shame not to post the raw data / spreadsheet for further mining though. Having said that - while 550 patches sounds a lot, is it really ? hard to say without a comparison to another live project.
• Discovered my calc is using a layout'ed "find" dialog; spent some time tweaking the XML for that to make it prettier and knocking up an E-mail for Janneke. Chased, reproduced & filed fontconfig related crasher.
• Lunch; Phoned Morris Armitage - apparently their new terms of business they sent are not those that they'll be using for us; funny people. Phoned Furniture Village's Cyrill - replacement for beetle infested sofa, removed ~10 weeks ago still pending.
• Dug at on-screen display bug, apparently down to some odd defaults. More gdm poking, gave up - dug at OpenJDK briefly instead, pinged Andrew. Dinner with babes, J. out to a girls night.
• Poked at UML - no matter how many hackers there are on Linux, it seems UML can't keep up, which is a shame - it should be dead useful. Chat with Miguel.

• Wow, another day of mail; how dull. Flora finally got my VISA approval thing from China, nice work! Now to persuade the printer to print it. Call with JP. Kelli's staff.
• Tried to get evince to render my Chinese VISA PDF no chinese fonts - filed a bug, tried Windows XP instead - wow, amazingly more broken (no doubt a version issue), but nice. It looks like evince might be a great target for some PackageKit hooks via a. "you are missing all the interesting fonts, click here ..." type feature, assuming of course that it's not there already - interestingly the latest Acrobat has this on Win32.

• To work, pondered mail much of the day, chat with Caolan & Thorsten. Listened to half the ops conf-call, customer conf call, chat with Patrick.
• For some odd reason my Dell / (mostly) intel machine is wedging itself quite frequently with Beta2, seems X related - annoying to chase. Up late, call with Guy - plugged away at Empathy.

Bad, bad cutlery !

Apparently people are trying to stick 'forks' into each other (and in particular me) these days cf. Simon & Roy's noted enthusiasm for this. Apparently (particularly according to medieval, western, knife manufacturers everywhere) the fork is hell-spawn, not to mention economically damaging, particularly when bundled with rice. But, what is a fork ? Wikipedia's write-up is helpful but extremely broad.

Historical antecedents

Back in the day - I used Slackware Linux (cue nostalgia for antique hardware etc.) but now I read up on it, I seem to have been duped: Slackware seems to have been based on a fork of SLS! - I was cheated out of using the real thing. Worse than that, apparently SLS was forked again by that Ian Murdock guy into something called Debian. Sadly, since then, SLS appears to have gone the way of the "Save the Stegosaurus" campaign.

Of course - there are now hundreds of Linux Distributions, each with different choices built in, many with different packages. Of course, there is some nuance here - is Mandriva a fork of RedHat ? is Ubuntu a fork of Debian ? is Mepis a further fork of Ubuntu ? are these cycles of creative destruction: branching, tweaking, trail-blazing new directions, including new features, and/or failing and disappearing bad ? should we have a central planning committee wielding powerful legal tools to pro-actively stamp them out ? or are they a positive side-effect of open-ness: allowing bad ideas to be weeded out by survival-of-the-fittest ?

Waffling and yardsticks

What makes a fork ? Can a budding etymologist confirm the relation to the unix 'fork' system call ? If you use that, in a miracle of copy-on-write-ness; post fork, anything you change is not shared with the parent process ?

Why are linux distributions not generally seen as forks ? I hypothesise that the answer is primarily that development is (for the most part) not independent - distros work on the same underlying code, and (hopefully) contribute their changes back to the common underlying code-pool. Of course, they tend to put different branding on top, have different configuration settings, and often include packages specific to their own distribution, some of them (sadly) not Free software.

What about branching ? is cvs tag -b gnome-2-22 a fork ? do projects go forking themselves regularly as wikipedia suggests ? again, usually people don't call this a fork - it's a branch, and code changes go into both sides of the branch. What then about adding components to a package ? is writing a new out-of-tree kernel module a fork ? what if you give it to some of your friends ? Does this apply more broadly ? is open-sourcing Solaris creating a fork of the GNU/Linux stack - by replacing a key component with a duplicate ? or is that just duplication ? or is Linux itself a fork/ duplication of PrimevalNix and thus intrinsically bad ? does licensing matter ? is duplication for licensing reasons acceptable ? how about purely for ownership ?

David A. Wheeler's forking appendix (to his interesting paper) postulates intent here; is the intent to become a competing or replacement project ? Well, certainly I'd like go-oo to become, as long as Sun pointlessly excludes our features, a replacement distribution (NB. not fork) of OO.o to the one Sun controls: simply because I'd like to get this injustice addressed, but of course, I'd far prefer to have a single up-stream community controlled source for OO.o.

Re-applying this to the world of OO.o, lets look first at StarOffice - was that a fork of OO.o ? in the past when it included proprietary closed source pieces and custom tweaks, and clearly was intended to compete with OO.o, did Sun fork it's 'own' project ? or was that a commercial re-distribution of OO.o with bits added ? How about ooo-build / go-oo ? that includes plenty of patches that for one reason & another (mostly the pain threshold) are not yet up-stream, the snapshot looking far worse than it is, since many patches have already gone up-stream over the years. Of course go-oo also includes some pieces (mostly well separated) that Sun simply refuses to accept up-stream because it wants to own them. Does that make it a fork ? if so, who is to blame for any negative effect ?

Conclusions

Some issues are great for endless charged debate - to quote Yes Minister, The problem is all my facts are statistics, and all your statistics are facts. I (of course) pass on useful and relevant information; it is other people that gossip, I take a lively interest in my surroundings; it's the other people that are nosy etc. People can call go-oo a fork certainly, it's a convenient way of putting it in a box marked 'evil' without mature consideration - but perhaps for consistency they need to call rather a number of other things forks, putting us in good company. Personally, I think forking is sometimes justified, but is go-oo really a fork ? you decide. Whatever your answer, it is totally facile to insinuate that because we encourage people to work on go-oo, that none of their work will get into Sun's OO.o - some of it will (we hope), and we'd love Sun to take the components too, under the copy-left license of their choice.

• Prodded mail, specced up some work for a new OO.o hacker / volunteer, good stuff. After much digging, once again managed to find paman - the pulseaudio thingit that lets you actually set the output volume level to a sane level (thus saving neck injury while trying to get the ear near the speakers). For some reason I keep forgetting it's name. Why the underlying device likes to have a really low level in the first place is beyond me.
• Prodded a fun samba autotools issue, soon fixed. Lunch, with Laura & Benjy. Voted for the OpenSUSE board suitably. Call with Frank, Holger, Guy & Peter.

• Up earlyish, packed people off to school. Poked at yast2-gtk, fixed up the regression tests, created a new version with more nice fixes from Ricardo. Recalled I need to commit to the osc branch I created.
• Noticed some of the on-line interest in my article last Friday;
  • LWN's excerpt of my conclusion is perhaps more stark when dis-connected from my definition of an 'active developer' (and the several caveats around that); obviously the real number of active developers (in either the Linux kernel or OO.o) is unlikely to actually dip around release time, yet it's clearly an interesting metric.
  • The high-bloodpressure-brigade apparently think I'm a hijacker (clearly an under-emotive & balanced view; After all, I have a beard; what more proof should be necessary !). Admittedly I would love to see some sensible changes in control & direction for OO.o, but I happen to think these would be good for Sun as well as OO.o. As for the rampant factual inaccuracies the most amusing is the apparent lack of understanding that Sun's OO.o 3.0 ships with built in MS/OOX import too, oh and that Sun have a patent cease-fire in-place with Microsoft.
  • Of course, it's right to be frustrated that OO.o is not getting enough developers - join the club. This guy clearly has the right attitude, apparently I should get actively involved in OO.o (the sad truth being that I have less time than ever before on it currently - with lots of other Desktoppy things to look at). In general though, that's excellent advice, please do mail me if you are a developer and want to get actively involved, we can help out.
• Poked at espeak vs. festival, some Evolution bits - seemingly performance is substantially improving wrt. the sqlite re-work. Lunch, Clarity, FATE, hopefully, eventually back to prodding gdm.
• Interested that Roy published Simon's (apparently private) E-mail: verbatim; quick call with Simon. Fascinating to hear Simon holding forth on behalf of the OO.o engineering community, while in the same breath accusing me of spin; cue reality check. In other sections we start with a reasonable summary: "he doesn't trust Sun to use the aggregated copyright wisely." before moving on to something I happen to be sure is a rather egregious falsehood: "Note that the aggregated copyright has been used twice in the last 4 years, once to drop SISSL as a dual license (an act that got IBM to start contributing) and once to adopt LGPLv3 instead of LGPLv2.". If this is so, and the ownership rights are just used to simplify really occasional re-licensing between free-software licenses (and they're not needed to drop a second license), it should be trivial to proscribe that use in the SCA, and I (for one) would be thrilled; sadly it is not so. Much more spin than substance there, but I'll go easy on Simon since apparently it was not for publication & thus not dissection proof, or apparently well researched.

• Mini lie-in; walked to All-Saints - Hannah singing in a harvest festival thingit - complete with carboard combine harvester animated by several fun-sized people. Good stuff.
• Left H. for lunch with Naomi B. and headed home, dinner, cycling practice with N. in the road - nine steps without falling over. Went to pick H. up, who griped about tucking her dress into her knickers to ride back; un-tucked it as soon as the Father's back was turned, and managed to wreck her favorite dress in the chain - it's a (own) goal ! poor dear.
• Piano practice, and put babes to bed. Out for a drink with Chris, up rather late.

• Up early, let J. sleep. Sawed extraneous chair in half from the front room - lots of cotton wadding, and horse-hair bits in there. Off to the tip, amusinglyrecognised from my hackergotchi by some member of the cognoscenti. Home, cleaned perishing mould from the windows: easier than expected, thank God the moulds havn't worked out how to eat my tasty plastic windows yet.
• Off to Thetford Forest's High Lodge for picnic and play with the girls. Discovered the new, and unfeasibly large Pyramid Slide rather a triumph of fun loving over the health & safety nuts. Lifted small people up inside at length.

• Up early, experimented cycling H. to school before punting that to J. hmm. Baby-sat a bit, and onto compiz - Beta3 should be in rather good shape, osc down so filed bugs variously. Poked at yast2.
• Prodded at boot charts; it seems on my machine, Jon's sexy new gdm - complete with nice a11y features, speech and so on, appears to start an entire session before noticing it is in auto-login mode & tearing it down again, poked at that.
• Lunch, sidetracked by an interesting OO.o crasher. Poked at gdm again; halfline pointed me at daemon/INTERNALS which is somewhat helpful, but these async, multi-process, intensive IPC designs are still hard to grok; bother.

What is success?

Is success measured in downloads, or up-loads? are bugs filed as good as bugs fixed? are volunteer marketers as valuable as volunteer developers? If we have lots of bugs filed and lots of volunteer management material is that success? is the pace of change important? Does successful QA exist to create process to slow and reject changes, or by accelerating inclusion of fixes improve quality? Is success having complete, up-to-date and detailed specifications for every feature? Is success getting everyone to slavishly obey laborious multi-step processes, before every commit? Alternatively does success come through attracting and empowering developers, who have such fun writing the code that they volunteer their life, allegiance and dreams to improve it?

I encourage people to download & use OpenOffice.org in one of it's derivatives. I'm pleased when people file bugs, help with the QA burden, promote the projet etc. However, in a Free Software project the primary production is developing and improving the software - ie. hacking. So the question is: how is OpenOffice.org doing in this area? Are we a success in attracting and retaining hackers? Is the project sufficiently fun to be involved in that lots of people actually want to be involved?

As we are finally on the brink of switching away from the creaking (22 years old) CVS (provided by Collab.net), to an improved Sun hosted Subversion (sadly not a DRCS) - Kohei and I created a set of scripts to crunch the raw RCS files as they go obsolete. They reveal an interesting picture.

Caveats

As with any measurement task, we believe these numbers are fairly reasonable; and we try to make them meaningful. On the other hand perhaps there is some horrendous thinko in the analysis, bug reports appreciated. It'd also be nice to see if the internal Sun statistics match these.

Firstly - the data is dirty; since we're analysing RCS files; so - when files are moved to the binfilter, or even renamed they have been simply re-committed - causing huge commit spikes. Similarly license changes, header guard removals and various other automated clean-ups, or check-ins of external projects cause massive signal swamping spikes. We have made some (incomplete) attempts to eliminate a few of these. In recentish times all work happens on a CVS branch, which is later merged release engineers (who appear to have done ~50% of the commits themselves), so we filter their (invaluable) contribution out by account name (cf. rt's oloh score).

Secondly - another distorting factor is that we chart only lines added: in fact when you change a line it is flagged as an add and a remove; so the number is more correctly lines added or changed. This of course fails to capture some of the best hacking that is done: removing bloat, which should be a prioirity. In the Linux kernel case this metric also gives extra credit to bad citizens that dump large drivers packed with duplicated functionality, and worse it rewards cut & paste coding. I don't often agree with Bill Gates but:

Measuring programming progress by lines of code is like measuring aircraft building progress by weight.

still at least the 'lines changed' facet should be helpful.

Thirdly - release cycles cause changes in contribution patterns, clearly frantic activity during feature development lapses into more bug-fixing later in the cycle. Thus we expect to see some sort of saw-shape effect.

Fourthly, working on OO.o is infernally difficult, getting code up-stream is extremely and unnecessarily painful - this results in many contributors leaving their code in patches attached to bugs in the issue tracker, and we make no account for these; these changes (if they are committed at all) would appear to be Sun commits. Thus it is possible that there is at least somewhat wider contribution than shown. Clearly we would hope that full-time contributors would tend to commit directly to CVS themselves.

Magnitude of contributions

This graph is more meaningless than it might first appear, the raw data still shows noise like individuals committing obvious sillies copying chunks of OO.o to the binfilter eg. To some extent it is further distorted by us trying to clean this up for the past couple of years before giving up:

So the data is not that useful. Is it more useful to look at an individual to see if they are contributing something? If we threshold the data we can at least approximate an activity metric / boolean. The graph below shows two developers - the Sun developer Niklas Nebel, and the Novell hacker Kohei Yoshida. Both work primarily on calc, and you can see the large bar when Kohei committed his solver to a branch at the end of 2006.

It seems clear that we can at least approximate activity with some thresholding. More interesting than this though, we can see a most curious thing. Despite Calc (apparently) being the relative weakness of OO.o, and Niklas being the maintainer of the calc core engine, and the calc "Project Lead" (with special voting privileges for the 'community' council), in fact he hasn't committed any real amount of code recently. That jumps out in the comparison with (vote-less) Kohei in the last six months. It is very sad indeed to all but loose Niklas from the project, though at least we'll see him at OOoCon. Verifying this counter-intuitive result with bonsai reveals the same picture.

Activity graphs

Extending this metric to the entire project we see perhaps a more interesting picture. By thresholding contributions at one hundred lines of code added/changed per month, we can get a picture of the number of individuals committing code to OO.o. Why one hundred? why not? it's at least a sane floor. Clearly we get a metric that is very easy to game, but luckily that's hard to do retrospectively.

It is clear that the number of active contributors Sun brings to the project is continuing to shrink, which would be fine if this was being made up for by a matched increase in external contributors, sadly that seems not to be so. Splitting out just the external contributors we see some increase, but not enough:

Novell's up-stream contribution appears small in comparison with the fifteen engineers we have working on OO.o. This has perhaps two explanations: of course we continue to work on features that are apparently not welcome in Sun's build cf. the rejection of Kohei's solver late in 2007, and much of the rest of our work happens in ooo-build, personal git repositories, and is subsequently filed as patches in IZ.

A comparison

So, it should be clear that OO.o is a profoundly sick project, and worse one that doesn't appear to be improving with age. But what does a real project look like that is alive? By patching Jonathon Corbet's gitdm I generated some similar activity statistics for the Linux kernel, another project of equivalent code size, and arguably complexity:

Graph showing number and affiliation of active kernel developers (contributing more than 100 lines per month).
Quick affiliation key, from bottom up: Unknown, No-Affiliation, IBM, RedHat, Novell, Intel ...

There are a number of points of comparison with the data pilot of active developers aggregated by affiliation for OO.o.

Similarities: both graphs show the release cycle. Spikes of activity at the start reducing to release. Linux' cycle is a loose 3 months, vs. OO.o's 6 months.

Differences: most obviously, magnitude and trend: OO.o peaked at around 70 active developers in late 2004 and is trending downwards, the Linux kernel is nearer 300 active developers and trending upwards. Time range - this is drastically reduced for the Linux kernel - down to the sheer volume of changes: eighteen months of Linux' changes bust calc's row limit, where OO.o hit only 15k rows thus far. Diversity: the linux graph omits an in-chart legend, this is a result of the 300+ organisations that actively contribute to Linux; interestingly, a good third of contribution to Linux comes from external (or un-affiliated) developers, but the rest comes from corporates. What is stopping corporations investing similarly in OO.o?

Conclusions

Crude as they are - the statistics show a picture of slow disengagement by Sun, combined with a spectacular lack of growth in the developer community. In a healthy project we would expect to see a large number of volunteer developers involved, in addition - we would expect to see a large number of peer companies contributing to the common code pool; we do not see this in OpenOffice.org. Indeed, quite the opposite we appear to have the lowest number of active developers on OO.o since records began: 24, this contrasts negatively with Linux's recent low of 160+. Even spun in the most positive way, OO.o is at best stagnating from a development perspective.

Does this matter? Of course, hugely ! Everyone that wants Free software to succeed on the desktop, needs to care about the true success of OpenOffice.org: it is a key piece here. Leaving the project to a single vendor to resource & carry will never bring us the gorgeous office suite that we need.

What can be done? I would argue that in order to kick-start the project, there is broadly a two step remedy:

• Kill the ossified, paralysed and gerrymandered political system in OO.o. Instead put the developers (all of them), and those actively contributing into the driving seat. This in turn should help to kill the many horribly demotivating and dysfunctional process steps currently used to stop code from getting included, and should help to attract volunteers. Once they are attracted and active, listen to them without patronizing.
• Distance the project from Sun: perhaps less branding, certainly less top-down control, reduce the requirement to 'share' all your rights over to Sun before you can contribute to the project. Better still, share ownership of the code with a non-profit foundation to guarantee stability and an independent future for the code-base.

Unfortunately, the chances of either of these points being addressed in full seem fairly remote - though, perhaps there will continue to be some grudging movement in these directions.

A half-hearted open-source strategy (or execution) that is not truly 'Open' runs a real risk of capturing the perceived business negatives of Free software: that people can copy your product for free, without capturing many of the advantages: that people help you develop it, and in doing so build a fantastic support and services market you can dominate. It's certainly possible to cruise along talking about all the marketing advantages of end-user communities, but in the end-game, without a focus on developers, and making OO.o truly fair and fun to contribute to - any amount of spin will not end up selling a dying horse.

Postscript

Why is my bug not fixed? why is the UI still so unpleasant? why is performance still poor? why does it consume more memory than necessary? why is it getting slower to start? why? why? - the answer lies with developers: Will you help us make OpenOffice.org better? if so, probably the best place to get started is by playing with go-oo.org and getting in touch, please mail us.

Finally - we invite you to repeat the analysis, the raw spreadsheet data (for data-miners) is here: ooo-stats.ods linux-stats.ods and the RCS parsing scripts parse_rcs.py with dependants in that same directory.

• To work; turns out my USB key, while having reasonable burst read speeds makes for a thoroughly pathetic root file-system; and while excellent at seeky reads (Alt-F2 doesn't kill the machine grokking .desktop files), is overall slower than a turgid slugnoodle.
• Greg Swift reports, that instead of editing my ~/.ssh/known_hosts file I should be doing ssh-keygen -R , which is a good point, though hopefully something I'll have to do a lot less of now.
• Started to poke at an interesting bug Thomas Biege threw up. Since my 11.0 OO.o build is broken on 11.1 Beta2, decided to use the new split builds we have in 11.1 - grabbed the OpenOffice_org-libs-gui.src.rpm package (11Mb), and zypper got me another 46Mb of -devel package (we're working on shrinking that too). Realised I need to tweak /usr/lib/ooo3/solver/LinuxIntelEnv.Set.sh to add an /opt/icecream/bin to CC & CXX, then: sudo bash -l -c "/usr/bin/rpmbuild --eval '%define jobs 10' -ba SPECS/OpenOffice_org-libs-gui.spec" and bingo, it built for 10 minutes and died with some Java problem; bother (missing some package dep somehow). Luckily I had enough built to debug, and committed some deferred dependency generation speedup before hitting the gdb-is-not-working issue: filed that instead.
• Lunch with Laura, back - prodded text; tried to get compiz working on my hardware - eventually it did: a beautiful accelerated experience on top of radeonhd, prodded multi-monitor support too. Prodded wall plugin crasher with David.

• An exciting morning of bug filing, and investigation. Finally set too with the evolution snapshots which avoided at least one already-fixed bug being filed again; nice.
• Poked at the session bus problem, which (it seems) doesn't like root connecting to it, causing grief for yast. Discovered my setting for "what happens when the suspend button is pressed" was "Do nothing" - which explains the absence of action on Fn-F4: with users like this who needs real bugs ?
• Prodded the build service - Darix kindly helped me create a home project, such that osc branch doesn't complain no permission to create project ..., which is great. Apparently the latest osc needs --no-verify to build things, and flies more gracefully with su-wrapper = sudo in ~/.oscrc.
• Lunch, looong analyst call. Did a live-install to a USB key - wow, it's a really nice installation workflow, albeit the write-rate to the key is impossibly bad: 1Mb / second x 4+Gb == 67+ minutes, fun - almost makes sense to write it to an on-spinning-disk ext3 image, then blit it en-masse across.

• Poked at bugs; downgraded to xorg B1 to test things. Since I regularly re-install machines locally, I'm ~always getting irritating messages of the form "REMOTE HOST IDENTIFICATION HAS CHANGED" and having to edit ~/.ssh/known_hosts - which is hyper-irritating. Dear lazyweb, is there a good way to turn off such checking but only for a local sub-net ?
• lejo read the manual for me; and dug out:

  Host 192.168.0.*
  	UserKnownHostsFile=no
  	StrictHostKeyChecking=no

  thanks.
• Signed up for the UKUUG conference finally, only to realise it overlaps with OOoCon - bother. Managed to catch rsync faliing nastily - another variant of an out-of-space problem; this time, not having enough space to duplicate my 700+Mb Sent mail folder, before updating it: --inplace fixed that.
• Lunch, backed up, installed Beta2 on the laptop - looking interesting, filed bugs left & right, and worked on one or two. Pleased with RadeonHD - I seem to have a beautiful X server with DRI, and working hot-plug multi-head bits out of the box: nice. If only compiz wasn't broken. Hesitantly switched to the latest Evolution, and filed bugs.
• More paperwork, mail sending, helped bathe the babes: Elisabeth is known as the 'sitting duck' in honour of the various (regularly un-successful) attempts to love her by her sisters in creative ways; as in "my hair has all been loved off" etc.

• Poked mail, estimated travel budget for the next financial year, call with Henne; Clarity. Poked at yast2-gtk a little wrt a11y. Nice to see Mike's ORBit2 / root stuff working nicely. Nailed hideous yast2 interaction bug - whereby 'log' assumes it can hang about to the const char *s you pass to it forever.

• Lie in, to NCC; Tony spoke on an overview of Judges. Mike & Leanne back for lunch, tidied up, some Violin practice with N., put the kids to bed, slept.

• Dealt with babes to give J. a lie in. J cycled into town to get the market shop done. Poked at yet more architectural drawings trying to fit stairs into here and there, printed and read the fire regulations, inner rooms on floor three are not kosha: bother.
• Lunch; took H. to the Funky Fun House for Loren's party, lots of playing, some 'disco' thing afterwards, it never ceases to amaze me that the Barbie Girl song is thought to be at all suitable for young children. Took a tired Naomi Brighty home early with H.

• Prodded mail, somehow a mass of tiny administrative irritations has accumulated over the last days; started on them somewhat reluctantly. Prodded Evolution gingerly with Sankar, then split-build image issues.
• Phoned Abbey's Zoe Burnett to be re-assured that my flexible mortgage is in fact not structured as a savings account, hence the tax-free interest, and as such is neither covered by government guarentee, and would not be lost in the case of a collapse; good.
• OO.o call, call with JP, call with Kelli, I'm at other people's call it seems.

• Mail, poked Tor wrt. DNS lookups on Win32, wrote up PDX summary finally, chat with Olaf, backed up laptop, poked at Evolution some more with Sankar.
• Lunch with Mary Rogers, back to work, Kelli's staff, sync with Roger. M. waking people up multiply in the night.

• Back to work, poked mail. Set off a build of the latest layout work from Janneke, annoyed by helpex performance (building 8-way with icecream it bites). Seems to do 3 system calls per line read from a 300k line file, a million sys-calls here & there - but where is it slow ? Made the code 40% faster but realised that the real system hit is from the well-known ext3 / page write-back / sync evilness; and the fact we generate 100's of Mb of output file, bother.
• Lunch, Nick came over, chat with Guy, OO.o team meeting, call with Florian, call with Guy. Out to Alpha in the evening.