Stuff Michael Meeks is doing

This is my (in)activity log. You might like to visit my employer SUSE which is an amazing company, and also Dell who in days of yore provided me with a free laptop for Gnome development / conferences. Also if you have the time to read this sort of stuff you could enlighten yourself by going to Unraveling Wittgenstein's net or if you are feeling objectionable perhaps here.

Older items: 2012: ( J F ), 2011: ( J F M A M J J A S O N D ), 2010: ( J F M A M J J A S O N D ), 2009, 2008, 2007, 2006, 2005, 2004, 2003, 2002, 2001, 2000, 1999, legacy html


2012-05-21: Monday

2012-05-20: Sunday

2012-05-19: Saturday

2012-05-18: Friday

IBM's Symphony code contribution

Today IBM seems about to deliver on their promise of opening up the Symphony codebase. That is a good thing. It represents an important way-point, in the middle of a long process.

A long journey

I recall well meeting Don Harbison at the OpenOffice conference in Koper in 2005, and a memorable party during which I no doubt bored him to death by re-iterating the importance of working with the community, in the open and contributing your code. Then around April 2006, IBM Workplace 2.6 arrived: a proprietary product based on a version of the OpenOffice.org 1.x code-base. That was enabled by the non-copy-left SISSL license variant the code was under at the time. Fast fowarding to September 2007, Lotus Symphony appeared in beta, complete with an interview "IBM joins OpenOffice to widen it's reach" with Doug Heintzman, promising:

"IBM will dedicate a core team of 35 programmers in China to the OpenOffice project, but more people will be added as needed around the world, he said."

Around this time, we got some contributions of parts of the Symphony feature-set thrown-over-the wall. Sadly these were mostly vs. an obsolete code base, and were mostly not maintained or forward ported (though LibreOffice's current Lotus Word Pro filter was rescued from that dump). At the time I confess I was eager for IBM not to contribute anything towards propping up the fundamentally unjustly managed and structured OpenOffice.org project, with which I'd become utterly disillusioned.

As time passed, the waiting and suspense continued to build, in November 2008 at OOoCon Beijing I had the pleasure of meeting Michael Karasick, whose (keynote) gave an apologetic score-card for this contribution, and promised "we will be contributing". More time passed. By July 2011, the donation of the code was announced in a press release "IBM Donates Lotus Symphony Source Code to the Apache OpenOffice Project", and still no code.

Then, this week Don Harbison announced that IBM have signed a software grant agreement to the Apache project for the code, which is planned to appear in svn as a single, flat, code dump. At last ! the code will be read and the valuations independently assessed. I have fond memories of working together with Doug, Michael & Don, and I'm certain their commitments were sincerely given and meant on each occasion. I suspect the primary cause of the delay is degrees of embarrassing frustration inflicted by part of a corporate machine fearful of, and unused to the transition costs of open, community based development.

Every day, open and engaged ...

Of course, it is great when code that has been proprietary and closed is finally opened, and licensed in a way that LibreOffice can include it. While there is some sad level of duplication vs. work done in LibreOffice, there are also some nice sounding features that should be useable for our next release as/when we have re-licensed.

On the other hand, one of the real pleasures of working in LibreOffice is the collegial, interaction and co-operation with my much-appreciated colleages from Red Hat, SUSE, Canonical, Lanedo, Google and the hundreds of developers and other supporters that have contributed to the project since we started ~eighteen months ago. The credit these guys deserve, for their outstanding effort defies praise. In my book what looks like the boring, every-day, long-haul work of open interaction to achieve shared goals is worth immeasurably more. It may take time to hammer out results that I don't always agree with, but it is good.

Playing well with the community seems to me to mean a lot more than a one-off code-dump. It also means being willing to compromise and work constructively with others of differing ideological viewpoints, encouraging and motivating people to work together.

It means that companies should not horde their changes, to try to be first to the market. There should be direct contribution of the totality of bug fixes and improvements, as they are made, to an appropriate branch. Unfortunately, licensing is a factor here too. The Apache license, by providing you with the choice of when to release your code: "now ? later ? never ?" creates an economic incentive to horde and create a saleable, proprietary feature-edge. That in turn creates a disincentive for others to share. In contrast, a weak copy-left license pushes people inevitably towards sharing, working together, and a service & support based business model.

Hoping for good corporate citizens

So, what does this mean, if anything ? if this move signals a genuine change of heart, towards working collaboratively with the developer community in a sustained and non-proprietary fashion - releasing code changes as they are made and working fully in the open, that is good news. Of course, the most convincing way to make such a commitment to work well as peers with the community, is to join with the existing majority of the developers around the code-base, who are eager to work with IBM as part of LibreOffice. Indeed, more than that - I (and I suspect others) are anxious to make room for our friends from IBM: Peace, Love, LibreOffice ! However, that will inevitably mean making a few real compromises, working in community always requires that. One would be formalizing that intention to contribute well in the most convincing way: using the form of a source-code-license like the MPLv2 or LGPLv3+ which codify such good behaviour. That helps to ensure timely, collaborative code contribution from all players, protecting everyone independent of scale. Is it hard to make such a binding commitment ?

Failing that the option of competing with that community, while trying to build a track record from scratch as an enthusiastic believer in open development, collaboration, compromise, working as an equal, etc. may prove problematic. One canary here may be how this substantial code dump is treated. Will it be split up into individual, digestible features & commits, which can be merged individually into the existing Apache OpenOffice codebase. Or will a single, big, un-documented code commit be attempted ?

A conclusion or two

It looks like IBM will release six+ years of work by their development team; that is a good thing, it will be interesting to see what their sharp team has been up to over that time. Opening previously proprietary software is almost always a good thing, and it may provide our users with some improvements in due course.

In this historical context actions speak much louder than words, but are much harder to extrapolate into the future. Will we see a positive, constructive engagement moving forwards ? the best sign of that would be positive interaction with, compromise and re-unification with the vast majority of developers. An ongoing sadness for me is the lack of even contemplation of that.

Still, in the meantime the LibreOffice community is having fun preparing for it's 3.6 freeze with steady hacking progress; a prototype new feature page is in the process of getting built, though I suspect completing that will need to wait for some last minute features to get pushed. It's a great place to have fun, make a difference and get involved with Free Software, why not try an Easy Hack today ? every little helps.

2012-05-17: Thursday

2012-05-16: Wednesday

2012-05-15: Tuesday

2012-05-14: Monday

2012-05-13: Sunday

2012-05-12: Saturday

2012-05-11: Friday

2012-05-10: Thursday

2012-05-09: Wednesday

2012-05-08: Tuesday

2012-05-07: Monday

2012-05-06: Sunday

2012-05-05: Saturday

2012-05-04: Friday

2012-05-03: Thursday

2012-05-02: Wednesday

2012-05-01: Tuesday

2012-04-30: Monday

2012-04-29: Sunday

2012-04-28: Saturday

2012-04-27: Friday

2012-04-26: Thursday

A LibreOffice/Apache OpenOffice Comparison

As the date of the Apache OpenOffice release approaches, and the final release candidate wends its way through a couple of rounds of approval / voting, I thought it might help clarify the current situation to have a side-by-side summary of what is in each suite. I'll update this entry in response to feedback, please do mail me with corrections if I've got things wrong.

Let me say, straight off, that I think the 'removal of copy-left' code (or at least its replacement) has been done reasonably well. Potentially rather a confusing description though: there are still great big gobs of copy-left code as hard requirements for a useful Apache OpenOffice but these are category b copy-left, instead of the category x licenses: (including the LGPL) that Apache excludes. The functionality loss from this removal is modest, as new versions of dependencies have been selected or system dependencies added, with even some rule-bending around shipping GPL dictionaries.

On the other hand, thus far, there are rather few really new features in the release that did not come from Oracle's existing work; that is outside of some pleasant drawing improvements, which we hope to merge into LibreOffice for our next major release.

A (very) potted history

So - what has been going on in these projects:

Of course, that misses a huge amount of detail out, a huge team of developers, translators, QA guys, infrastructure / sysadmin, conference and hack-fest work that people have done on the LibreOffice side: a simply staggering job, building a near-complete infrastructure and community that relentlessly execute against our time-based release plan. I can't begin to itemize or call out everything that has been achieved by so many.

On the other side work eventually started at Apache OpenOffice Incubating. Here - there were a few star developers from the Oracle Office BU that, despite having just been fired, continued to help Apache fix up what was left post Beta-1 by that hasty exit. They did so, not because they were required by their employer, but because they felt bound by duty to the project and the codebase. Prominent among them were Mathias Bauer, Eike Rathke and Michael Stahl, the last two of whom we enjoy the company of as active LibreOffice contributors today.

Rob Weir of the Apache project also produced an infographic on the project's progress which is worth checking out.

A side-by side comparison

I spent a little while building a side-by-side comparison matrix to make it reasonably easy to see what is going on and what is in each of the three versions involved. I collected my data from the Apache OpenOffice 3.4 new features page which is helpfully split into those from the beta, and the new ones. I merged some subset of the distinctive main features from LibreOffice: 3.3, 3.4, 3.5 until I gave up.

Perhaps the biggest single chunk of the work we've done is hidden behind just a couple of bullet-points: MS Office 2007+ import / export. While all right-thinking people demand ODF, there is a sad reality of people using Microsoft's formats that Free Software users need to interoperate with. Having said that, the LibreOffice team has done some amazing work building other file filters to migrate legacy formats into the ODF future. Other features mentioned are a bit passé, SVG import has been shipping to GNU/Linux users, and many others since around 2008. Anyhow - here it is (also as an odg):

Some thoughts

One of the most curious things about the OpenOffice.org brand, is the loyalty that users have to it, despite the 3.3 feature freeze being twenty-two months ago, having lost much of its development community, and having had no new release since January 2011 - users are still downloading this increasingly old and creaky release at top speed. Getting the LibreOffice message out: about the new, exciting, much more featureful, and fun suite is important - and much appreciated. Existing profoundly clueful GNU/Linux communities already know about the best free office suite ever others don't yet.

So - re-merging the projects; what is the sticking point ? In large part this comes down to licensing; do you believe that large companies will contribute their changes in a timely and constructive fashion without the encouragement of the license requiring that ? do contributors 'just want to see their code used' ?, or do they really 'just want to work together with others towards a common goal' ? who can say; only time will tell. Interestingly, there is perhaps a middle ground in the category b license. This type of copy-left license can allow binaries to be distributed under eg. an Apache license - but only after all of the source code is contributed back.

Interestingly, Apache OpenOffice Incubating already has a heavy dependence on code under this license; LibreOffice plans to move wholesale to the category-b Mozilla Public License (MPLv2). For me, going further to the Apache license would break faith with that significant proportion of our large developer community who see enabling non-contribution of core fixes and improvements as anathema, encouraging proprietary software creation. LibreOffice's weak-copy-left licensing is a compromise, enabling all sorts of awful proprietary things to be created and sold alongside, and embedded into LibreOffice, yet it requires contribution to humanity of changes to its core. That is good for interoperability, and community dynamics. Sadly, most end-users care remarkably little about licenses - clicking through them without even reading; that is something we should try to fix.

Another possible development, is the ongoing hope that IBM's commitment (of nine months ago) to donate whatever improvements Lotus Symphony has made to the code-base will finally materialize; which may help to close the growing feature gap to LibreOffice somewhat in places.

Ultimately, I would strongly prefer to have all of us working happily together in a community chosen by the active developers, and governed by a license that defines and regulates our co-operation. I'd like to think we have a good and growing track-record of being a flourishing, dynamic and stable foundation in The Document Foundation today. Certainly, we can't compete for length of track-record with the Apache pedigree, but in terms of attracting and inspiring developers to have fun together and produce a predictably great product I think we're excelling.

The Document Foundation is a diverse meritocracy, open to all contributors with plenty of room for new parties. We will share the same category-b licensing style that Mozilla and Eclipse have. There is no reason for more companies not to get involved. I'm happy with where LibreOffice is, but we can always be better - why not get involved today and become a key part of the future.

2012-04-25: Wednesday

2012-04-24: Tuesday

2012-04-23: Monday

2012-04-22: Sunday

2012-04-21: Saturday

2012-04-20: Friday

2012-04-19: Thursday

2012-04-18: Wednesday

2012-04-17: Tuesday

2012-04-16: Monday

2012-04-15: Sunday

2012-04-14: Saturday

2012-04-13: Friday

2012-04-12: Thursday

2012-04-11: Wednesday

2012-04-10: Tuesday

2012-04-09: Monday

2012-04-08: Sunday

2012-04-07: Saturday

2012-04-06: Friday

2012-04-05: Thursday

2012-04-04: Wednesday

2012-04-03: Tuesday

2012-04-02: Monday

2012-04-01: Sunday

2012-03-31: Saturday

2012-03-30: Friday

2012-03-29: Thursday

2012-03-28: Wednesday

2012-03-27: Tuesday

LibreOffice Collaborative Editing Prototype

One of the last, big missing features in LibreOffice is collaborative editing, to fill this hole Eike Rathke (of RedHat) has been working for some weeks getting Telepathy code hooked into the LibreOffice core. This was chosen to allow us to setup a channel for multi-way communication over existing Instant Messaging (IM) protocols, without requiring any form of server.

A mini hack-fest

With the lovely Google Summer of Code coming up and this Collaborative Editing task on the menu, we thought that it would be good to break the back of the problem, so we had a basic design agreed, some of the heavy-lifting done, and an awareness of the problem areas, so that we'd be good mentors for that task (though please apply for other tasks too, since we suspect this will be rather a popular one, there are a load of other great things to get stuck into in the project list).

Using some quick funding from The Document Foundation, it was possible to get Eike to Cambridge, with Collabora sponsoring us providing office space and Will; getting all the brain-power we needed into one place: Will, Eike, Myself & Rob.


Outline / progress

The kernel of the collaborative approach chosen here is to realise that, as long as every user of the suite applies every change in exactly the same order, it doesn't much matter what the result is; users will get used to resolving occasional conflicts, as long as they can deterministically see the results, and know that this is what everyone sees. Naturally, more complicated designs are possible, but this at least provides a simple, initial approach while we tease the code apart to extract real controller objects.

Telepathy provides a powerful instant-messaging framework that allows the creation of abstract 'tubes' that tunnel invisibly over the IM protocol - allowing arbitrary new protocols to piggyback on your existing IM chat. Perhaps, even better than that, the Jabber protocol provides multi-user chat-rooms, where the order of messages is defined and consistent - meeting our collaboration requirement. Extending that to one-to-one connections we can select a master to ensure ordering, and for local LAN / auto-discovery we have low enough latency that we can do likewise.

So after some initial design overview from Will, we hacked away happily - I spent much of my time re-immersing myself in the calc core code. It is easy to forget how heroic the hackers that work on the top of LibreOffice are - after all, the lower layers of the system are shared, and have more use and polish. I dug at the view code, and ensuring that the ScDocFunc object, was used - splitting large operations (such as cell entry) into a series of smaller, simpler operations in an undo transaction on the model.

Eike meanwhile worked to get our unit tests working with various main-loop integration horrors under control, and the basic communication channels created and sending messages from A to B. Then he connected that to my lame demonstration protocol work to get messages and co-editing flowing.

Will - providing invaluable design input, hacking help, debugging assistence, API work, an 'Approver' to handle incoming connection requests from mission control, chased down obscure g_signal and C++ exception mis-interactions, provided non-stop enthusiasm, and was an exemplary host.

We spent much of the week, Monday to Friday, from early until late at night closeted in a remarkably pleasant sunlight room hacking away, heads down - with occasional frustrated noises, shaking of the head at code in dire need of discipline and so on.

What does it look like


(awful image hack linked to movie above to avoid iframe/planet troubles). For those who dislike screencasts, a more shaky, real-camera and two side-by-side laptops version, where you can see rather less you can go here, failing that perhaps you'd like a much higher resolution, non-dubbed webm of the above.

Thanks

To Collabora (where dreams solidify into reality) for hosting us, and providing the invaluable advice, support and enthusiasm of Will Thompson - thank you: Rob McQueen, Philippe Kalaf, Guy Lunardi, Neil McGovern & team. To RedHat for Eike's time & of course to SUSE who pays your humble scribe.

Also to all those who supported LibreOffice financially, your generosity makes this sort of strategic travel possible, thank you.

What's next

This is really just a prototype, there is a lot of work to turn this into a product, ie. you will not be seeing this anytime soon outside of some Experimental feature in any shipping LibreOffice - so unless you actively want to help hack on it - please don't bother the developers with questions.

Next, is the Google Summer of Code, there are a lot of rough edges here for this demo, the instant-messaging contacts list needs finishing, connection setup and negotiation needs re-factoring and pushing down into the framework. Then of course, big chunks of re-work to slowly tease out a controller concept from the code, and ensure that it is interposed correctly between the Model and View / Scripting.


Another area that will need more work, is encouraging the Empathy client onto windows, and providing the telepathy framework in an easy-to-consume form there.

Of course, if you want to play with this, and/or see what new exciting things will come out of the next LibreOffice Hackfest, it would be fantastic to see you at:


For some good German beer, sausages, excellent company, and some intense co-editing of the code.

Get involved

Are you a developer ? do you want to get involved with LibreOffice ? Now is a better time to start than later. It's a good idea to get a generic build first, and do an Easy Hack to get confident that we welcome code contributions, and love to have new people involved.

Then if you want to hack on the telepathy integration - assuming that it has not been merged to master by now (if so the branch will be gone) you want to checkout the feature/tubes2 branch, build it and ask on the developer list which bit you could best work on.

2012-03-26: Monday

2012-03-25: Sunday

2012-03-24: Saturday

2012-03-23: Friday

2012-03-22: Thursday

2012-03-21: Wednesday

2012-03-20: Tuesday

2012-03-19: Monday

2012-03-18: Sunday

2012-03-17: Saturday

2012-03-16: Friday

2012-03-15: Thursday

2012-03-14: Wednesday

2012-03-13: Tuesday

2012-03-12: Monday

2012-03-11: Sunday

2012-03-10: Saturday

2012-03-09: Friday

2012-03-08: Thursday

2012-03-07: Wednesday

2012-03-06: Tuesday

2012-03-05: Monday

2012-03-04: Sunday

2012-03-03: Saturday

2012-03-02: Friday

2012-03-01: Thursday

2012-02-29: Wednesday

2012-02-28: Tuesday

2012-02-27: Monday

2012-02-26: Sunday

2012-02-25: Saturday

2012-02-24: Friday

2012-02-23: Thursday

2012-02-22: Wednesday

2012-02-21: Tuesday

2012-02-20: Monday

2012-02-19: Sunday

2012-02-18: Saturday

2012-02-17: Friday

2012-02-16: Thursday

2012-02-15: Wednesday

2012-02-14: Tuesday

2012-02-13: Monday

2012-02-12: Sunday

2012-02-11: Saturday

2012-02-10: Friday

2012-02-09: Thursday

2012-02-08: Wednesday

2012-02-07: Tuesday

2012-02-06: Monday

2012-02-05: Sunday

2012-02-04: Saturday

2012-02-03: Friday

2012-02-02: Thursday

2012-02-01: Wednesday

2012-01-31: Tuesday

2012-01-30: Monday

2012-01-29: Sunday

2012-01-28: Saturday

2012-01-27: Friday

2012-01-26: Thursday

2012-01-25: Wednesday

2012-01-24: Tuesday

2012-01-23: Monday

2012-01-22: Sunday

2012-01-21: Saturday

2012-01-20: Friday

2012-01-19: Thursday

2012-01-18: Wednesday

2012-01-17: Tuesday

2012-01-16: Monday

2012-01-15: Sunday

2012-01-14: Saturday

2012-01-13: Friday

2012-01-12: Thursday

2012-01-11: Wednesday

2012-01-10: Tuesday

Removing unused code in LibreOffice

One of the unfortunate things that LibreOffice inherited, as part of the several decades worth of unpaid technical debt, is unused code that has been left lying around indefinitely. This was particularly unhelpful when mingled with the weight and depth of the useful code we have around the place. Caolan McNamara of RedHat wrote a beautiful tool callcatcher that identified these unused methods, and in recent times in LibreOffice we've had an unusedcode.easy file in our toplevel with a list of methods that should be removed. It's pretty easy to find and expunge a method or two, with a quick git grep, and dropping a patch to the developers mailing list. To escape from a pile of administration recently, I knocked up a pretty nasty perl script to parse the git numstat output, to see how we're doing. That produces a fun graph:


It seems that over half of our unused code has now bitten the dust. Uunfortunately as we remove more, more wasteage tends to be revealed, which explains some of the upward jumps in the graph, nevertheless the trend is clearly down. One of the side benefits of the unsung heros working at the conversion of our old-style macro driven generics to modern STL is that this looses us several unused methods per class converted.

If you want to get involved with LibreOffice development, it doesn't get much easier than this - please do check out the code and have a go. For the more adventurous finding an unused destructor, without a matching unused constructor is proof of a leak that needs chasing, of which there are a handful.

Failing that, why not run Caolan's callcatcher over your project to see which nooks and crannies are surplus to requirements.

2012-01-09: Monday

2012-01-08: Sunday

2012-01-07: Saturday

2012-01-06: Friday

2012-01-05: Thursday

2012-01-04: Wednesday

2012-01-03: Tuesday

2012-01-02: Monday

2012-01-01: Sunday


My content in this blog and associated images / data under images/ and data/ directories are (usually) created by me and (unless obviously labelled otherwise) are licensed under the public domain, and/or if that doesn't float your boat a CC0 license. I encourage linking back (of course) to help people decide for themselves, in context, in the battle for ideas, and I love fixes / improvements / corrections by private mail.

In case it's not painfully obvious: the reflections reflected here are my own; mine, all mine ! and don't reflect the views of SUSE, Novell, The Document Foundation, Spaghetti Hurlers (International), or anyone else. It's also important to realise that I'm not in on the Swedish Conspiracy. Occasionally people ask for formal photos for conferences or fun.

Michael Meeks (michael.meeks@novell.com)

Made with PyBlosxom