Stuff Michael Meeks is doing

This is my (in)activity log. You might like to visit Collabora Productivity a subsidiary of Collabora focusing on LibreOffice support and services for whom I work. Also if you have the time to read this sort of stuff you could enlighten yourself by going to Unraveling Wittgenstein's net or if you are feeling objectionable perhaps here. Failing that, there are all manner of interesting things to read on the LibreOffice Planet news feed.

Older items: 2015: ( J F M ), 2014: ( J F M A M J J A S O N D ), 2013, 2012, 2011, 2010, 2009, 2009, 2008, 2007, 2006, 2005, 2004, 2003, 2002, 2001, 2000, 1999, legacy html


LibreOffice progress to 3.6.0

a

Today we released 3.6.0, checkout the new feature summary and release notes. As always a lot more has happened under the hood than can easily be addressed in the features page; so here are some of the behind-the-scenes bits and some of the unsung heros doing the vital, but more hard-to-see work.

Improving extension registration

For this release we introduced new code to improve the way extensions (and built-in components) are registered and looked up. This dungs out a very old and unpleasant piece of the UNO core, replacing it with a rather simpler and cleaner version that performs better - casual callgrind profiling shows some nice wins here.

Unfortunately, for one reason and another, despite a delay for a fourth release candidate, there are still some circumstances where an upgrade will not re-register some built-in extensions, and silently exit on first launch (just re-run it). On the down-side that can affect spell-checking dictionaries, and (for some) on the up-side disables Auto-COrrection too. This is already fixed for 3.6.1, due in three weeks, and can easily be worked around by removing the extensions/ directory from your user-profile (which will then be re-generated). Better than that as this has been fixed Stephan Bergmann has further dunged out the registration / upgrade code to stop this recurring.

The ESC decided to sticking to our release discipline and cadence to underline the importance of early QA and bug fixing. I'd love to see more people running our daily dev-builds of master and reporting bugs much higher up the food chain, and building good relationships with the grateful developers who just created them. See here for a list of other annoying issues in 3.6.0.

Upgrading the code: dead code

In other areas we've made improvements that are hard to see; one of these is eliminating dead code; here's how we're doing at that: unused methods vs. time - lower is better.

As you can see the task is ~nearly done. The remaining bits are particularly hard and involve finally completing the re-write away from the legacy, non-standard macro-generated template-alikes, to use modern C++ STL templates, that are far more accessible. Many thanks in particular to Noel Grandin for his ongoing work there in this release, and Michael Stahl for reviewing lots of patches, and Caolan McNamara for building the dead-code detection infrastructure.

Upgrading the code: German comments

The ongoing charge to translate our remaining German comments continues to bear fruit. We're around 50% done here - down to around 25k lines (as detected by Miklos' nice bin/find-german-comments script).

With many thanks for this release to Luc Castermans, Philipp Riemer, Julien Nabet, Tomaž Vajngerl, Jesso Clarence – Murugan, Florian Reisinger, Sebastian Spaeth, Philipp Weissenbacher and several others. One of our (encouraging) problems is that comment translators tend to get hooked on code cleanup, and move off to do ever more substantial fixes elsewhere - so it's a great way to get involved in LibreOffice: not unlike the typing-in of BBC BASIC games that got me into code syntax; if you can understand German, you can help out here - but please don't send automated translations, we're eager to have the highest quality.

Upgrading the code: fixing the make system

One profound unpleasantness of the OO.o legacy was the custom version of the unusual 'dmake' required to reliably build the code. Some heroic hackers have been taking on the work Bjoern started at Oracle to convert to using GNU make instead. One benefit of this is exposing massive build parallelism - we now have a global view of dependencies for nearly all of our internal code in one GNU make process allowing an incredible degree of parallelism - thousands wide at times: so if you want to actively stress-test your 1024 CPU machine, we can help you.

Many thanks to David Tardon, Matúš Kukan, David Ostrovsky, Peter Foley, Pierre-Eric Pelloux-Prayer and more for working hard at this; and many others for detecting, filing and fixing any associated bugs. It is encouraging to see David's latest test branch to complete the UNO Runtime Environment migration, nailing the last set of problems before tackling the final chunk of 'external' modules - the around 80 bits of source we download, patch, build and distribute (mostly to enable Windows builds). Hopefully we'll be fully dmake-free in six months time.

Upgrading the code: unit tests

One of the most exciting things that has been ongoing in LibreOffice up to 3.6 is the growth in unit tests. In contrast to the various unreliable automated tests that were present before, these are run very regularly. Some we run on every module build (ie. just building calc), and yet more we run on straight-through builds. We aim to spend some small proportion of our compile-time say under 5% building and executing these as widely as possible:

Many thanks to Markus Mohrhard, Artur Dorda, Daniel Bankston, Kohei Yoshida, Miklos Vjana, Caolan McNamara, Michael Stahl, and many more - who got tired of endlessly re-fixing the same sorts of bug, and did something permanent about it by writing small, fast, reliable unit tests. This is a great job to get stuck into - we have code lying around that could be turned into nice unit tests, and it would be great to have more people involved in that work.

Faster, more readable code

Another thing that can't be seen are the very long overdue improvements in our basic String classes. Traditionally we've had four string classes where one would be best - heroically Caolan McNamara finally killed the ByteString class in 3.6, leaving only three. With perhaps a greater impact Lubos Lunak spearheaded a fantastic improvement in good-taste-based-coding with some tweaks to our OUString class, used almost everywhere. You can see the improvement in this pseudo diff based on real changes:

-    if ( aFlavor.MimeType.indexOf( ::rtl::OUString(RTL_CONSTASCII_USTRINGPARAM( ";Aspect=THUMBNAIL" )) ) != -1 )
-        xPropSet->setPropertyValue(::rtl::OUString(RTL_CONSTASCII_USTRINGPARAM("HelpText")), aAny);
+    if ( aFlavor.MimeType.indexOf( ";Aspect=THUMBNAIL" ) != -1 )
+        xPropSet->setPropertyValue("HelpText", aAny);
This sort of change is repeated many, many times. Interestingly in many cases these make the code not only far more readable, but also produce smaller, faster code by avoiding intermediate allocations of strings; a great improvement. Lubos also has some interesting clang plugin in progress to automatically generate this sort of semantic code upgrade. Using that is naturally blocked by completing the on-going re-basing on the Apache licensed version of LibreOffice.

Cross-compiling code

Other major improvements in 3.6 are the ability to do cross-compiles of LibreOffice - Tor Lillqvist did a lot of work here, initially focused on ensuring that the MingW cross-compile for Windows from Linux worked well. This can be run on very fast hardware as a tinderbox to ensure that master remains buildable always. That work has expanded into cross-compilation to Android and iOS both of which continue to develop and improve in master. At the same time Eilidh McAdam for Lanedo has done some great work at cross-compiling .msi files from Linux, re-using wine code - to improve that install flow and move us towards fully Free Software tooling.

Better builds out of the box

Another nice improvement is the growth of big-machine tinderboxes to ensure that we keep master in a buildable state on many platforms at all times. You can checkout the latest build status. With many thanks to Norbert Thiebaud for his time and CPU horsepower donations, and to ByteMark for their generous donation of a very substantial machine for native Win32 building that really helps.

Easier code contribution

Yet another ongoing initiative, is that of improving our code submission process - Bjoern Michaelsen, Norbert Thiebaud, Robert Einsle, David Ostrovsky and others have been working hard to get Gerrit (an automated code submission system) setup. This lets you convert your OpenId account to git push access to a review branch without any interactions, it also has a fine code review flow for submitted patches, and makes including cherry-picking patches, and providing feedback much easier. We plan to migrate our main repository to this shortly, leaving freedesktop as a read-only mirror.

Conclusions

Well, there is much more to say - so many people have done such a lot of work to make 3.6 better; the QA team has been doing a fantastic job triaging incoming bugs and raising their priority / flagging them as Most Annoying - we couldn't work effectively without you guys. The design team have done some really excellent work in 3.6 as well, overall it's an exciting time for LibreOffice. I hope I've given you a snapshot of some of the things you can't easily see in the 3.6 release - of course, there are also a ton of things you really can see that are visible on our 3.6 new features page - I commend that to you; thanks to Marc Paré for pulling that together.

If you want to get involved in LibreOffice, there has never been a better time, there are more people spun-up to help mentor, we continue to grow our developer base, and there are still plenty of easy hacks to ease you into the code - we'd love to hear from you.


My content in this blog and associated images / data under images/ and data/ directories are (usually) created by me and (unless obviously labelled otherwise) are licensed under the public domain, and/or if that doesn't float your boat a CC0 license. I encourage linking back (of course) to help people decide for themselves, in context, in the battle for ideas, and I love fixes / improvements / corrections by private mail.

In case it's not painfully obvious: the reflections reflected here are my own; mine, all mine ! and don't reflect the views of Collabora, SUSE, Novell, The Document Foundation, Spaghetti Hurlers (International), or anyone else. It's also important to realise that I'm not in on the Swedish Conspiracy. Occasionally people ask for formal photos for conferences or fun.

Michael Meeks (michael.meeks@collabora.com)

Made with PyBlosxom