Stuff Michael Meeks is doing
Older items: 2016: ( J F M A M J J A S ), 2015: ( J F M A M J J A S O N D ), 2013, 2013, 2012, 2011, 2010, 2009, 2009, 2008, 2007, 2006, 2005, 2004, 2003, 2002, 2001, 2000, 1999, legacy html
std::condition_variableinstead - except - that works beautifully on all platforms except windows. Dug into it - and found this horror: punch line, you can only use a std::mutex in a static de-initializer if you compile with Visual Studio 2015+ and run on anything later than WinXP: nice. Encouragingly the boost guys appear quite able to make it work.
----------------------------------------- Note: We are a Microsoft Office site. Our base version is 2010. Please make sure that files you send can be read in this format.followed of course by all manner of other legal boilerplate. I guess it's a damning indictment of another products' interoperability with itself too; hey ho. It is amazing to see how government policy around ODF meets reality.
Today we release CODE 2.0 which includes Collaborative Editing. We've done a huge amount of work since CODE 1.0 - and many of these improvements have been back-ported for our customers & community, but it is perhaps well to credit the authors in one place and survey progress over the last six months.
There have been 1200 commits to 'online' and around 500 to the LibreOffice core since we branched for 1.0. While we celebrate and credit the fantastic work of the LibreOffice community without which there could be no LibreOffice Online - it is clear that Collabora is the architect and driving force behind putting LibreOffice in the Cloud - we have 14 online commits from non-collaborans. On the flip side we're eager to change that - that there is lots of low hanging fruit here, for web developers, and we love contributions. Anyhow - to the details:
$ ./loolstress --bench --iter=100 /tmp/looltrace.gz Iterations: 100 Latency best: 67 microsecs, 95th percentile: 4740 microsecs. Tile best: 52 microsecs, rendering 95th percentile: 1446 microsecs. Cached best: 34 microsecs, tile 95th percentile: 67 microsecs. Rendering power: 55.4497 MPixels/sec. Cache power: 826.327 MPixels/sec.
While each LibreOffice Kit process is fundamentally single-threaded and isolated into its own jail, we serve many hundreds of clients per WSD daemon involving a certain amount of concurrency excitement. In 2.0 Ashod Nakashian removed a per-connected-client thread count from the slave kit processes significantly simplifying things, he also did a lot of lock auditing, session cleanup and admin console state synchronisation improvements.
Meanwhile Pranav Kant ensured that users' WOPI credentials are tracked such that the right user is tagged as having hit the explicit 'save' (of course we also auto-save regularly).
Other cleanups and fixes here include switching to un-named pipes thanks to Tor Lillqvist, catching and fixing races around tile rendering (Ash) and avoiding a race when loading the same document multiple times concurrently thanks to Kendy.
Performance and interactivity is significantly improved in 2.0 with a large number of improvements. Tile requests are now prioritized such that tiles near active cursors are rendered and returned before others thanks to Ash and Kendy who also re-wrote the tile request, combining, de-combining, and re-aggregating in the core to ensure that we render the largest area possible to avoid the costs of repeated in-document image scaling. Ash also added a tile versioning scheme to ensure that we always get the latest, correct tile under heavy load and avoid duplicate painting. Kendy and Tor added a prioritisation scheme for presentation slide thumbnailing to improve interactivity.
Other improvements are to use
LD_BIND_NOW to have
all the linking performed by the forkit process once and not again,
Ash also improved the forkit child spawning and lifecycle
Pranav profiled and optimized the WOPI GetFileInfo usage to reduce it to a single call from three, some implementations have unexpectedly expensive implementations of this. Marco Cecchetti - made SSL support run-time configurable to make it easier to configure SSL off-load / acceleration in large setups and Ash added compiled-in limits to avoid misleading enterprise users that CODE is suitable for deployment at scale without support. CODE remains firmly targetted at home users, Henry - added a warning for too many users/connections.
The most obvious improvement in 2.0 is collaborative editing,
which has a lot of UX elements to enable users to see other users' cursors,
selections, with name popups when they are active, or mouse-overed, and more
thanks to Henry Castro. There are also a number of nice new elements
such as the user-list pop-up in the bottom toolbar containing the
colors of users' cursors thanks to Pranav you can also click to
jump to that user's position in the document which is neat.
Pranav also dunged out lots of the shared editing pieces from 1.0.
Meanwhile Miklos Vajna did lots of heavy-lifting around Undo & Redo
which gets significantly more complex with multiple users. Our eventual solution
is of building an infrastructure for simple de-confliction, while also
allowing a Repair Document power mode to unwind the rare
<ctrl>-a, <space> issues. This seems to work
really well under our normal collaborative editing conditions.
Various general cleanups have been implemented such as re-locating, and anti-aliasing the progess-bar and spinner thanks to Pranav, who also added git hashes to help->about dialog - with Andras adding cgit web links to make QA versioning trivial. Andras also did lots of l10n enablement work to the Admin Console, slide layouts, status bar, colour pickers, menus and context menus, while Pranav added context sensitive enablement to menubar items. Meanwhile Kendy switched us to use larger icons, and improved the toolbar look, while Andras made it easy to escape menus with that key. Meanwhile Henry added status bar items to all of the component - with much requested features like calc selection quick summing, and powerful word-count in writer.
One beautiful win for calc users is Henry's tweak to highlight row and column headers to show cursor and selection positions. He also took the time to implement lots of other nice wins around row & column headers from drag & drop re-sizing, double-click to optimial size, as well as adding context menus to show & hide rows & columns.
Meanwhile Marco fixed a pernicious range issue with page-up/down to make editing smoother while Pranav fixed some formula-bar ergonomics and Henry tackled the interactive auto-sum functionality. Henry also extended the toolbars and menus to include sorting, simple number formats, merge & center, wrap text and more. Meanwhile, for this release, Andras disabled zooming in calc - interestingly our competition don't do this either - and the non-linear co-ordinate space caused by row height rounding was causing real issues.
Meanwhile in writer Andras significantly extended the format menu allowing simple page size, line-spacing, alignment and other simple formatting. Pranav improved the context menus exposed on in-document comments, to allow replying and deleting, while Andras added foot & end-notes, page & column break insertion, as well as Table of Contents and image wrap & anchor context menu items for one of our customers.
Lots of other miscellaneous but useful wins went in such as Henry fixing <ctrl>-f to focus the 'find' toolbar and Pranav mapping <ctrl>-s to .uno:Save. A volunteer Feyza Yavuz added the insert comment toolbar button. Faruk Uzun implemented a draggable insert table toolbar button grid, and Pranav customized a nice read-only mode for users without edit permission.
Collaboration was the largest, and the most substatial part of the 2.0 work. It is hard to list everything that was done there though - it is infrastructure work, many parts of that are implementation details, it was done incrementally, and also with some trial & error on the way; really more of a research task than just plain development.
Conceptually, we wanted to reuse as much of existing code as possible - as we always do - which was luckily possible for the collaboration too. LibreOffice has a feature that allows users to open multiple views of the same document: Try Window -> New Window in your desktop LibreOffice. You will see that this already gives you multiple cursors, and multiple selections. We "just" mapped the multiple document views to the multiple users. That means, on the server, there is only one document open, but with many views - one view for one user.
Of course, the hard part was to make it all fit together: Each user (view) needs to see the cursors and selections of the other users (views) which is something that LibreOffice did not provide previously. There were many bugs when update in one view did not trigger update of the other view (which is probably because the feature itself has not been much used until now). Lots of code had to be updated and fixed. And also, lots of decision had to be made about the updates - what change has to be broadcasted to all the views ? vs. which changes apply only to the current view, and what to other views than the current one. Not to mention all the corner cases when one of the views is closed in the middle of the broadcast to all the views, etc. You get the picture.
To mention the most important parts of this work:
Of course, on top of the new features, various people have contributed a tremendous amount of bugfixes. To list the most important ones:
While 99% of the code comes from Collabora, many thanks again to everyone who contributed including the LibreOffice translation community. As announced at the LibreOffice conference - we're working hard to get a first up-stream release of LibreOffice Online included into LibreOffice 5.3 - and to make daily image builds available for testing as we move towards that.
We've done a huge amount of work here; this is just a short summary of it, and it is an amazing privilege to be able to work with such a talented set of developers at Collabora and across the wider community around LibreOffice. Why not head to and get involved with LibreOffice, or deploy the stable 1.0 version of Collabora Online in your organization - while we harden Collabora Online 2.0 for your delectation.
make uitest.uicheck. Built ESC bug stats, commercial call, customer call. Lunch. Mail turnaround, setup customer call, admin. Out in the evening with J. while M. was at cubs.
make dump-deps-pngin the evening.
Today we release LibreOffice 5.2.0, the next step in our journey, and what will become the base of the increasingly stable 5.2.x series. There is a fine suite of new features for people to enjoy - you can read and enjoy all the great news about the user visible features from many great hackers, but there are, as always, many contributors whose work is primarily behind the scenes, and a lot of work that is more technical than user-facing.
Some moons ago, the ESC decided to add some under-the-hood wiki
pages so that people could add their own credits: I encourage you to read
those here: 5.1
There are lots of good things there, and it saves me reading and summarizing
~10k commits each release, but then again - that can be fun too. This is my
very quick attempt to make up for a year of inactivity on this front, and
pluck a few of the un-mentioned bits out of 17,734 commits (that is an average
of ~50 commits every day of the year) from
A great initiative of Norbert Thiebaud has been to collect together most of the infrastructure and entry points that we have at TDF, and build an attractive list of these to help those new to the project find and use our tools and services. Checkout http://devcentral.libreoffice.org/:
LibreOffice has been able to run in a headless mode, doing its own
pixel-bashing for a long time, and this is used intensively by both LibreOffice
Online and also the Linux / gtk3 port. It has been a neverending source of
amazement to myself and others that the (unreadable) template code using
Vigra produces rather poorly performing code for all manner of cases - and
needs special case optimization in the client code.
One of the great joys of LibreOffice 5.2 is the final replacement of vigra,
and the removal of the
basebmp directory which allows us to use
native (and assembler accelerated on eg. ARM) cairo for pixel bashing. While
basebmp has served well and accelerated rendering for years, cairo also gives
us accelerated anti-aliased line rendering and more. Thanks to Caolán
McNamara (RedHat) for that.
There were a large number of OpenGL and OpenCL improvements in this time period.
cache/opengl_device.logfiles with details of your drivers. After lots of work, it was discovered that Intel GL drivers on Windows 7 were flaky beyond belief, so these were black-listed.
vcldemotest paths, improved native widget rendering, faster clipping. There was a chunk of work put into lifecycle fixing, and shutdown cleanup, and many bug fixes.
sc/source/core/opencl/here: Tomaž Vajngerl (Collabora), Tor Lillqvist (Collabora), Noel Grandin (Peralex), Stephan Bergmann (RedHat), Caolán McNamara (RedHat), Markus Mohrhard, Marco Cecchetti (Collabora)
Performance improvements are hard to screenshot; but important nonetheless. We maintain performance regression tests (running under valgrind) at http://perf.libreoffice.org/ - which really helps to pin-point and repair issues as they arise.
One big-win in 5.2 was Armin Le Grand (CIB)'s work to improve our thread-pool, and use it to accelerate the 3D software renderer - which gives a particularly noticable speedup for 3D chart re-rendering.
Markus Mohrhard has done some simply excellent work on crash reporting - to help us find the most frequent crashes on Windows. While many Linux distributions have had similar tools deployed for years, covering Windows is important too. The implementation re-uses Google's Breakpad to create minidumps which are analyzed on the server side and charted prettily at http://crashreport.libreoffice.org/stats/
Markus would love some help from someone with web development skills
to help improve the front-end, and analytic / querying / data presentation
here. Please poke the developer list
This work has already lead to several vital fixes of the top crashers that have been seen in the wild, using good data to nail the worst quality issues. Thanks to Markus Mohrhard, Caolán McNamara (RedHat), and Miklos Vajna (Collabora) who have commits referencing crashreport URLs.
Work is ongoing around code quality in many areas, with 196 or so cppcheck fixes thanks to Caolán McNamara (RedHat), Julien Nabet, Jochen Nitschke, Noel Grandin (Peralex), Michael Weghorn, Takeshi Abe, Giuseppe Castagno and others.
Caolan and the RedHat guys manage to keep the Coverity scan count, and crash-testing (loading and saving ~91k documents to many formats) counts at or around zero at all times, as well as doing fuzzing and other cleanup.
Continuing the VclPtr re-work where we added strong referenced
lifecycle to all our widgets, Dipankar Niranjan in
has cleaned out all of the old & awful 'dog-tag' work that tried to get
lifecycle right in VCL, using much cleaner references.
In more recent times Yurtoglu, Melike Ayse and
Noel Grandin have worked to abstract the
and apply this to Menus, to ensure they are correctly lifecycle managed.
Thanks too to
Jocken Nitschke for internalizing the DeletionListener
which should be removed when we eventually reference count the sal/
Other great work to exorcise manual refcounting comes from
Thomas Arnhold, Daniel Robertson, Noel Grandin, Aleksas Pantechovskis
and especially Xisco Fauli for their work on
converting lots of manually reference counted or copy-on-write structures
to using the nice
cow_wrapper template; as well as converting
pImpl pointers to use std::unique_ptr. This reduces the scope for
programmer error, and missed corner cases and allows us to make
copy-on-write ref-counting thread-safe if necessary. There are 140 cases
fixed, and about 68 refcounting instances still needing love.
We continued to build and execute more critical unit tests this year - each one stopping some suite of regressions from ever returning. Grepping for the relevant TEST and ASSERT macros we continue to grow the number of unit tests:
The LibreOfficeKit API is the foundation of the Android App, GNOME Documents, and the ongoing LibreOffice Online work, and a lot changed in the last year. The two main building blocks of the LOK API are the methods of the exposed objects, and the callback types. Since 5.0, the following new methods has been added:
lok::Office::getFilterTypes()allows obtaining an up to date list of filter name - MIME type pairs, added for GNOME documents.
lok::Office::setDocumentPassword()allows opening password-protected documents.
lok::Office::freeError()allows releasing an error string allocated by the API.
lok::Document::getPartPageRectangles()allows obtaining the size and position of pages in a Writer document.
lok::Document::getTileMode()allows writing clients that work with both the new (Cairo) and the old (Vigra) headless backend (see above).
lok::Document::initializeForRendering()has been extended to allow key-value style options while initializing, like Writer's hide whitespace mode.
lok::Document::postMouseEvent()has been extended to handle click buttons and modifiers.
lok::Document::postUnoCommand()has been extended to get a callback when the result of an executed UNO command is ready.
lok::Document::paste()has been added to handle copy and paste.
lok::Document::getCommandValues()has been added to query possible values for UNO commands (font and style names, Calc row/column header details, Calc cell cursor).
lok::Document::setVisibleArea()has been added to be able to do correct page up / page down.
lok::Document::getViews()have been added as an initial support for collaborative editing.
lok::Document::renderFont()has been added to help providing font previews.
lok::Document::getPartHash()has been added to help clients tracking reordered slides.
lok::Document::paintPartTile()has been added to allow stateless rendering of different parts of a document.
LOK_CALLBACK_DOCUMENT_SIZE_CHANGED: Size of the document changed.
LOK_CALLBACK_SET_PART: The current part number is changed.
LOK_CALLBACK_SEARCH_RESULT_SELECTION: Selection rectangles of the search result when find all is performed.
LOK_CALLBACK_UNO_COMMAND_RESULT: Result of the UNO command execution.
LOK_CALLBACK_CELL_CURSOR: The size and/or the position of the cell cursor changed.
LOK_CALLBACK_MOUSE_POINTER: The current mouse pointer style.
LOK_CALLBACK_CELL_FORMULA: The text content of the formula bar in Calc.
LOK_CALLBACK_CONTEXT_MENU: Context menu structure (after a right-click).
An outcome of our Turkish Hackfest
in Ankarah was discovering that LibreOffice would not even build in the
toupper('i') != 'I' in this locale; fun cf.
Thanks to Krishna Keshav, Gökhan Gurbetoğlu and Apurva Priyadarshi
for the fixes.
A large number of UX / usability papercuts were fixed and improved such as keyboard shortcuts, consistent display issues, mis-sized and mis-placed controls, and some regressions from UI file conversion. Thanks to Caolán McNamara (RedHat), Akshay Deep, Regina Henschel, Maxim Monastirsky, Jürgen Funk (CIB), Bubli Behrens (CIB), Samuel Mehrbrodt (CIB) & Jay Philips.
One metric we watch in the ESC call is who is in the top ten in the freedesktop Weekly bug summary. Here is a list of the people who have appeared more than ten times in the weekly list of top bug closers in order of frequency of appearance: V Stuart Foote, Adolfo Jayme, Cor Nouws, Maxim Monastirsky, Eike Rathke (RedHat), raal, m.a.riosv, Julien Nabet, Caolán McNamara (RedHat), Miklos Vajna (Collabora), Beluga, Alex Thurgood, Michael Meeks (Collabora), Buovjaga, Yousuf (Jay) Philips, Joel Madero, Samuel Mehrbrodt (CIB), Markus Mohrhard, Timur, Jean-Baptiste Faure, Xisco Faulí, Aron Budea, tommy27, Michael Stahl (RedHat), Heiko Tietze, David Tardon, Laurent BP With many thanks to everyone else that helped to close and triage so many bugs for this release.
Thanks fundamentally to Norbert Thiebaud - we now have not only
great Jenkins / CI infrastructure, with gerrit integration - but we have a
much enlarged and more reliable hardware build farm that encourages people to
use Jenkins to check their work before pushing, in turn increasing the
quality and reliability of master. This year we've seen the benefits of
running our full suite of
make check unit tests on our debugging
builds under Linux.
The work Jenkins did for just master in the six month run up to the 5.2 branch point we have 34,124 tinderbox builds covering seven configuration across linux, mac and windows. With 27,737 builds covering three platforms tested on our gerrit infrastructure. This is for the CI managed hardware, there are a number of other volunteer run Tinderbox builders. This is extremely helpful for keeping master build-able and usable, making it easy for newbies to get stuck in. With 60,000 builds for 8k commits we are doing a lot of loop test and validation of the LibreOffice code at CI time.
Some of my favorite heroes are those who open up the code so
that others can work on it ! One of the key tasks there is translating
the remaining German
Comments (a list). The last four thousand lines seem to defy
translation - why this graph should flatten like this - I have simply
no idea. All patches from German speakers who love to finish things -
much appreciated. Many thanks to those who stuck with it, with more
than 2 commits; in order of commit count: Albert Thuswaldner, Phillip Sz,
Thomas Klausner, Chris Sherlock, Philipp Weissenbacher &
We are now down to seven modules left:
include, reportdesign, sc, sfx2, stoc, svx, sw
The 5.2.x releases run well on Windows XP, but we cannot be sure how long our tooling will continue to target that platform. As such - while the ESC has no concrete plans to drop Windows XP support, we are 'deprecating' it after the 5.2.x series - meaning that having a modern C++ compiler and a modern Windows build OS base-line have a higher priority. That means that in future it is possible that future major LibreOffice releases may not run on Windows XP; you have been warned. For now, there is no change here.
I hope you get the idea that more developers continue to find a home at LibreOffice and work together to complete some rather significant work both under the hood, and also on the surface. If you want to get involved there are plenty of great people to meet and work alongside. As you can see individuals ('Assigned') make a huge impact to the diversity of LibreOffice (the colour legends on the right should be read left to right, top to bottom, which maps to top down in the chart):
These charts, made so early in the month are still missing som individuals and commits queued up in gerrit for review - hence the July numbers. In terms of diversity of code commits, we love to see the unaffiliated volunteers contribution by volume, though clearly the volume and balance changes with the season, release cycle, and volunteers vacation / business plans:
Naturally we maintain a list of small, bite-sized tasks which you can use to get involved at our Easy Hacks page, with simple build / setup instructions. It is extremely easy to build LibreOffice, each easy-hack should have code pointers and be a nicely self contained task that is easy to solve. In addition some of them are really nice-to-have features or performance improvements. Please do consider getting stuck in with something.
Another thing that really helps is running pre-release builds and reporting bugs just grab and install a pre-release and you're ready to contribute alongside the rest of the development team.
LibreOffice 5.2 is great; it is made by a set of developers having fun, working together, and building an increasingly attractive and beautiful Free Software Office suite, I hope you enjoy using it. Thanks for reading, don't forget to checkout the user visible feature page and thank you for supporting LibreOffice.
If I have missed your good work out (and this is a very incomplete view based on a few hours of analysis) - please do add yourself to the relevant wiki page and potentially mail me a unified diff to this page; thanks !
online. Prepped for kids work tomorrow.
ImplDelDatacode from the deep pre-historic darkness of VCL - from Ras-al-Ghul.
184.108.40.206.4264driver appears to happily install a
My content in this blog and associated images / data under
data/ directories are (usually)
created by me and (unless obviously labelled otherwise) are licensed under
the public domain, and/or if that doesn't float your boat a CC0
license. I encourage linking back (of course) to help people decide for
themselves, in context, in the battle for ideas, and I love fixes /
improvements / corrections by private mail.
In case it's not painfully obvious: the reflections reflected here are my own; mine, all mine ! and don't reflect the views of Collabora, SUSE, Novell, The Document Foundation, Spaghetti Hurlers (International), or anyone else. It's also important to realise that I'm not in on the Swedish Conspiracy. Occasionally people ask for formal photos for conferences or fun.Michael Meeks (email@example.com)