Stuff Michael Meeks is doing
|
|
This is my (in)activity log. You might like to visit my employer
Novell which is an amazing company, and also
Dell who in days of yore provided me with a
free laptop for Gnome development / conferences.
Also if you have the time to read this sort of stuff you could enlighten
yourself by going to Unraveling Wittgenstein's net or if
you are feeling objectionable perhaps here.
Older items:
2008: (
J
F
M
A
M
J
J
A
),
2007: (
J
F
M
A
M
J
J
A
S
O
N
D
),
2006,
2005,
2004,
2003,
2002,
2001,
2000,
1999,
legacy html
- Up early, chewed mail, more gcc tweakage, improving the
debug output to find my construction vtable issue. Bug triage day -
lots of bug churn in the mail-box. Nailed the gcc issue, continued the
OO.o build.
- Poked at libical memory management with PChenthill - most
curious, and no doubt hides all manner of evils.
- Fixed a load of sillies in pagein-common, discovered with
Sankar's work, iogrind claims that this should knock about 500ms off
cold startup time, then again it claims we should cold start in 5
seconds where I see more like 20. Anyhow, a 10% win with little work
is not so bad. The extra data read is the cost of reading more than you
need, but linearly.
| task | before | after |
|---|
| pagein | 3.4sec 93Mb | 2.6sec 73Mb |
|---|
| soffice.bin | 1.2sec 10Mb | 2.7sec 23Mb |
|---|
| total | 4.6sec 103Mb | 5.3sec 96Mb |
|---|
So in theory a 11% win. Attempted manual cold-start timings to
check - the before/after times were 4.46sec to 4.07sec - ie. an 8.5% win
close ? not at all - subsequent runs of the 'before' case yield times
(for the same flush-all-caches, run gedit, quit, start OO.o) process of
4.4, 3.1, 2.5, 2.9, 2.7 secs. ie. totally non-deterministic as expected.
Attached some iogrind photos to the issue.
- The hidden thunks OO.o build completed & runs nicely,
good, must generate some callgrind numbers on linking cost.
- Bathed babes, dinner, back to catch up. Poked at yast2-gtk
with Coolo. Poked Alex & HPJ about GVFS locking, read a little about
webdav.
- Isolated & measured the vtreloc speedup with callgrind,
only about 10% savings in instructions, L2 cache misses etc. hmm. And
just a 4% memory saving, bother. Will focus on more memory savings
instead then.
- Up early, took H. to school, finally posted Muthu's letter,
replaced Rose's battery (now charged), car started perfectly: good &
bad - why was it flat ?
- Wrote my LXF column, booked flights variously. Back-ported
the gdm xdmcp fixes to SL10.3, and filed; X crashes going from a 64bit
system though - strange, poked at X there.
- Bruce & Anne over for lunch; good to see them. Filed X
server bug. Played with sysprof and metacity - lots of time spent in
fbCompositeCopyAreammx bother, more investigation later.
- Got gcc into shape wrt. copy constructors eventually and set
off a new full build, giving a chance to profile the whole build via
sysprof.
- Cell group in the evening, more poking at gcc corner cases
afterwards, and got iogrind's valgrind snapshot synced with 3.3.0 and
building on x86_64 again.
- Prodded mail, nice - snapshot builds of the latest Evo
for SL10.3 are at
http://download.opensuse.org/repositories/home:/srinidhi:/evolution-unstable/openSUSE_10.3/i586/
and the Evo team are fixing bugs fast.
- Interested to read Joseph Assad's thought provoking piece
on the corporatisation of Free software. Poked at remote T+C performance,
filed a tcpdump trace.
- Set off an OO.o build with hidden thunks turned on;
hopefully it'll be a nice win - after a while though it stumbled over
construction vtables: missing some vtreloc goodness.
- Team meeting, call with Tor, call with Thorsten, chat with
jrb. Banged on gcc some more.
- Up early, poked at a horrendous b-a-s deadlock, exacerbated
by an un-diagnosed re-connection issue, hmm. Chewed & spewed mail,
filed bugs.
- Got the GeodeLX xorg driver to do it's bit, at least some
things are accelerated now, and a higher-res screen too.
- Poked at metacity, hacked up the client composite version
detection for cow / named windows & sent to Thomas. Poked at gdm
apparently there were a load of XDMCP fixes for horrific breakage that
we missed in SL10.3, filed bug.
- Apparently the hideous fglrx external monitor driver bugs
are un-fixable, and pessimism reigns: the joys of closed-source
drivers. Switching to radeonhd looks like an attractive option.
- Dinner, J. out for lunch, conference call with Rich, Guy
& co. call with Guy & David Reveman afterwards: an expert's
insight into compositing problems.
- NCC, Tony speaking, J. did creche. DT & Zoe & John
back for a buffet lunch: went well. Talked & caught up, it's been
a long time. Out to try to jump-start Rose's car in middle - apparently
the battery is dead, took it home to charge. Tea, Gordon sermon, bed.
- Up early, out to the market with the babes, stocked up
on food, home, baby to bed, fine lunch. Out to a park near Milton,
not such a great playground. Back for DVD, pizza for tea, then bed.
- More UVC setup, poked mail, chased a gtkhtml2 issue:
filed with a valgrind log. Battled X - my Ati FireGL flickers the
external crt1 (only) as I press keys or move the mouse - amazing:
fought it for a while, before trying to persuade another Intel
card, then another Ati card to drive the 1920x1200 beast sensibly.
- Nick over for lunch. Then trying to get
X -query foo to
work - gdm reports WARNING: gdm_xdmcp_handle_manage: Failed
to look up session id random-number - irritating in
the extreme.
- Long call with Patrick & co. Back to work in the
evening. Interestingly, metacity (once you fix the assumption that
because the local machine has the latest composite extension, the
remote X server must too) works rather better via remote X to my
Ati RageXL, than it does locally, amazing. Discovered my screen
corruption was prolly down to a (now fixed) bug in the Damage
impl. in older fglrx drivers.
- Alex wanted to read my vtreloc patches (
binutils,
glibc,
gcc)
of course, not finished yet. Managed to remove the 'stable-x86'
pieces that had mangled my system, and get back to a more
reasonable state, albeit without the latest evolution (sadly).
- Up early, J. off to the dentist, back to poking at relocations
on writer start (from LD_DEBUG=bindings output):
| variant | startup bindings |
|---|
| current | 91300 |
|---|
| -Bsymbolic-functions | 63600 |
|---|
| -Bsym + vtrelocs | 58100 |
|---|
Clearly there are a number of PLT/function bindings that
we're not saving on startup. Browsing the remaining relocations I
was rather amused to find 2400 relocations on startup (4%) are to
please a cunning 31337 'memory-saving' strategy: the thinking goes like this:
- Instead of having lots of small strings in-line in
several libraries, lets have a single copy of each string in a
common library (svtools). [ so far, so clever ].
- Lets implement that using a symbol per string eg.
extern SVT_DLLPUBLIC sal_Char const SVTOOLS_CONSTASCII_DECL( sHTML_S_aacute, "aacute" );
[ oh dear ! - this of course generates a nice const char * rodata
string - but it also generates another string in the svtools symbol
table: "sHTML_S_aacute", plus another string in -each- shared library that
refers to it ].
So - comparing to the approach of using a simple in-line string
(also uniquified by any decent compiler & put in rodata), we loose
size: ( sizeof "sHTML_S_" + sizeof (relocation) ) * ( num-references + 1 )
and we loose performance: 2400 unique named relocations, searched across 50+
shared libraries: ~0.8% of OO.o CPU time on startup. And all in the name of
efficiency. Unfortunately, the technique dates back to the dawn of
time, (in cvs history terms), so it's hard to discern the intention. I wonder
where else it's used.
Anyhow, the 'best' fix (to save the optimisation) is to export
one symbol, s_HTML_Strings that is an enumerated array of strings
#define sHTML_S_aacute s_HTML_Strings[eHTML_S_aAcute]
type
thing. But for now, some search/replace action is perhaps easier.
- Spammed the suse gcc team with my patches for comment, poked
at Sankar's first iogrind patch - nice. Dug at mail, investigated generating
live memory snapshots for running processes, ie. a simple core-dump to run
'strings' on, to try to work out how XYZ process managed to blow up the
heap so comprehensively. Unfortunately, it's not easy
/proc/<pid>/mem
looks like just what you want, until you try to use it - you have to
pread the addresses out. Dug at
cryopid for some stealable code, before giving up. Presumably manual
fun and dump binary memory filename start_addr end_addr
in gdb is the best that can be done easily.
- Huge 24inch screen arrived, got that setup - albeit the Mobility
FireGL 5200 insists on flickering when you type (or move the mouse -
interesting), then got SL10.3 installed on the AMD UVC prototype, now it
has it's own screen: great to get a USB image booted and a (remote) network
install done in a couple of hours. Amazon ordering frenzy, bed.
- Full build died at start, with a missing patch - bother.
Did more measurements on libsvx - annoyingly -Bsymbolic-functions is
a higher impact change than vtrelocs:
| variant | size | unique named relocs | fn / thunk unique named relocs |
|---|
| current | 10154324 | 10375 | 6650, 1718 |
|---|
| -Bsymbolic-functions | 10020556 | 3891 | 1404, 481 |
|---|
| -Bsym + vtrelocs | 9840180 | 2135 | 37, 0 |
|---|
Of course, a large proportion of OO.o startup is dominated
by the cost of linking, in particular unique named relocation
processing, around ~20% of the time; so extrapolating wildly,
10k -> 2k is a 5x speedup, yielding a ~15% CPU-time startup
win. Or, with just -Bsymbolic-functions an overall 12% win.
- Amused by the onslaught of 'virtual' as a new marketing
term: it's no longer a thin-client, its a "virtual display client",
its no longer a 'pencil' but a "virtual marking solution" etc. Of
course when there are already several other marketing terms in a
product name it could become increasingly difficult to understand:
eg. is a plain-old Xserver a "virtual java desktop display
client solution" ?
- Comparing reloc runs is much helped by
echo "0" > /proc/sys/kernel/randomize_va_space.
Found I wasn't linking the new libsvx correctly, fixed that. Of
course the real benefit of vtrelocs is (having switched virtual
function calls to going via the relevant vtable) not exporting any
virtual symbols / thunks; but that's for the future.
- Kicked off a full build of OO.o with the various new
options, persuaded svn to update apply (it has the unfortunate weakness
of failing to fully update, but retaining the same final status
message). OO.o build fell-foul of cppu's gcc3.map hiding some
vtables it shouldn't. Poked at pulling my iogrind / valgrind snapshot
up to the lastest version, breaking it, bother. Some debugging of
my evo. alarm issue.
- Up lateish, H. off school & ill. Mail, back to gcc,
somehow an off-by-one error in clobbering slots breaks everything,
chased that down. Call with Rich: good chap.
- Hit more annoyance with copy relocations, interesting,
albeit tiresome, consulted Richard & Michael, drew pictures of
DSO initialization. Finally did the vtreloc processing just before
'init' pass - thus getting the ordering right, and avoiding
copy-reloc problems; albeit un-protecting all shlibs for longer.
- Transfered vaction times for last year from one web tool,
to another: fun. Restored / transfered my go-oo home directory, with
various files / presentations etc. to the new server with Hub's help.
- Got svx working with new relocs: nice, 300k smaller too,
sfx2 as well: looks like all is well, kicked off a full re-build
overnight.
- Up, poked at gcc some more - nice, new vtreloc generation
entry point looks far, far more promising & reliable. Hacked away
at some length, eventually back to square one(ish) but finally
comprehensible and (hopefully) robust.
- Up early, took babes to church while J. slept, did creche
with Ruth. Back, lunch, J, woke up late, Mary Poppins lots of
lying around looking pitiful. Bed.
- Up in the night putting M. back in her bed, hacked on gcc.
Lie in. J. in bed most of the afternoon, the whole family sickening,
bother. Watched Angelina Ballerina again on the wall. Why not Angelina
Boxerina ? after all, Gladiation is a noble sport that gives young,
disenfranchised, ethnic persons their best chance to escape the ghetto
(or something?).
- Up early, took H. to school. Hacking - misc. simple
fixes, acutely tantelising - copy relocations are just unpleasant.
More hackerage, now thunks arrive to plague me, clearly it's necessary
to expand copy relocation processing to thunks.
- Up early, H. to school, Ultrasound - baby looking good.
Home, checked mail, J. packed & set off to Suzannah & Clive's
to help with their first baby: Adam Mark Hawkins, 9lb1oz.
- Packed babes off to bed, poked at gcc - clobbered my
alignment issue, .suse.vtrelocs generating really nicely, read the
relinker script docs, and terminated the section nicely with QUAD(0).
- Quick call with Rob & Mark. Lunch, Laura dropped H.
home kindly; cleaned up variously. Showed Angelina Ballerina
while hacking on glibc: a couple of breakthroughs in simplicity of
approach, and big strides: a cunning approach to sort vtrelocs by
depth of derivation: got soffice.bin relocating nicely, but not vcl.
- Out for H's swimming lesson, back, jacket potatoes, bath
and bed: quite an effort without J. Cleaned up, back to vtrelocs.
Wrote a reference for Muthu of calc hacking fame, hopefully it'll help
his career.
- Up early, poked mail, people. Dug at gcc / binutils,
it seems the arch-specific, total-evilness of collating &
generating a custom section by munging relocs in other sections
might not be necessary after all: which would be lovely. Perhaps
just by merging & sorting sections in the right order, one
per vtable, life will be good.
- Amazed by Sun's purchase of MySQL, of course one wishes
them all the best. Lots of people mentioned possible synergies with
OO.o base - which sounds like a great idea; unfortunately linking GPL
stuff into the LGPL OO.o is in general a bad plan. Then of course, there is
the binary, and (I very much suspect) not very backwards-compatible
binary database which it would pay to keep outside of your .odb files.
Having said that, clearly a load of work is needed on Base,
in particular I'd love to see (and most of the code is there) a nice
migration wizard (using a remote/ODBC connection of course) from
Access to MySQL (or anything else) using the OO.o front-end - with
the VBA work from Novell, and extracting the forms etc. that would
be highly sweet.
- Ran evo in valgrind for much of the day; chewed through
the output looking for my problem, filed a couple of bugs.
- Cell group in the evening, call with Suzannah. Kate &
Milo Robson staying the night, great to see her & discuss our
plans.
- Slightly less groggy, mail pokeage; listened to OPS
all-hands call. Submitted FOSDEM lightning talk on iogrind. Played
with KWin, it too made an appalling mess of my screen: perhaps I
really have a broken composite XServer impl, then again compiz
works which is strange: perhaps starting from scratch with a
composite manager makes things work better.
- More joy with gcc - by splitting vtreloc tables into
their own sections, and adding (bogus) references to them from the
tail of the vtable - to get the cgraph bits right.
- Up, feeling awfully coldy, took H. to school, read mail,
caught up. Still having nightmarish problems trying to debug the new
glibc hackery for vtable relocs - tried to build as a system package
perhaps that'll help gdb.
- Finally located my stupidity - magically linking the
'copyrel' test as a shlib, and expecting it to run as an app: silly;
now to start compiling & testing OO.o with it.
- Fought various garbage collection problems whereby we
output vtable construction data for vtables that didn't get emitted
anyway (hmm), evil things lurk there, feeling pretty awful head-wise
too.
- Finished DVD in the evening, J. out for PCC meeting, and
surprise conf-call in the evening. Chat with Thomas until late, bed.
- M. crying fitfully most of the night, nightmarish: exahusted
parent syndrome. J. up early to clean the church, breakfast - off to NCC,
Janice speaking on Barnabus.
- Nyki and Nydia back for lunch, which was nice, joined by
their Mum later, really good to meet them. Slugged by the fire,
Prince of Egypt on the wall in the evening, bed extremely early.
- Up early, off to the market, shopped frenetically, back,
cleaned, prepared house for N.'s party. Fun-sized people started arriving,
played the guitar for musical-bumps, pass-the-parcel variously, another
fabric painting bonanza; a fine tea, and finally most children left.
- Kate & James arrived, packed the babes off to bed, and
enjoyed a fine evening with them - up rather late; cleaned up the house,
while J. dropped them back. Bed.
- Up early, pleased to read Joerg's nice
blog on speeding up the OO.o qatesttool - this is some fantastic
work. The more frequently we can run the full test suite, the
more frequently we will catch bugs in both the code, and the tests,
and the quicker we will iterate towards higher quality for both.
My hope is - that with more automation, and (of course preferably)
head-less, in-tree testing, we can increase the pace of integrations
and reduce the fear & uncertainty of change: good stuff.
- Chased an evo crash - turned out to be a missing
versioned dependency. Located the evo. crash on new appointment bug -
a division by zero, filed a few more bugs with Srini. Committed an
interesting ORBit2 fix from a J.Specht.
- Bruce & Anne over - Naomi's birthday, also Uncle
Anthony & Auntie Louis, sadly missed them due to over-conference-callage.
- Off to the dentist - amazingly nothing requiring
immediate, expensive surgery J. needs her first filling though: if
she goes on like this, 3 fillings might see her out.
- Back, fighting the gcc build now - I could swear this
used to be simpler, plugged on with it, the kindly Herr Matz reports
that
--disable-bootstrap is the solution, nice.
- Foolishly installed kdebase4 from stable-x86 to poke at
KWin - is my Composite oddness an X-server bug ? apparently, getting
kdebase4 required a newer OO.o and evolution. New evolution broken
on start, dug at the code - discovered a locking rats nest; poked
Srini / Psankar - perhaps fixing it will improve gwise stability
nicely.
- Poked Psankar, who it seems is interested in hacking on
iogrind - in his ITO, good
man. Found, fixed the evo. bug - and got my calendar back.
- Finally nailed down all the stupid glibc problems - really
working without any form of debugger at all is just exciting - doing
the vtable relocs nicely now: great.
- Up early, cycled H. to school, onto the mail. The book
Getting things Done arrived - though finding time to read it is
clearly an issue; as Fridrich kindly pointed out - I'm not doing
anything (apparently).
- Got glibc built, and a development iteration gone around,
added a DT_SUSE_VTRELOC tag into the mix, built the latest binutils &
gcc, and started patching them; it helps to move 'elf' to the top of
sysd-sorted.
- Out to a prayer meeting, took a chap called Chris that
turned up to a friend's afterwards. Back, stayed up late for a call
that didn't materialize, hmm.
- Onwards with the mail, N. very sad and ill this morning,
also hungry at lunch time poor thing. Got the OO.o GL transitions to
work for me - amazingly cool, particularly Radek's new ones.
- LXF column bits, team meeting, filed a bug. Set of conf
calls in the evening, tried to get glibc to configure and build in
parallel - grief it's a beast.
- Up, started out up the mail mountain, reviewed some
patches, read interesting status reports, poked at bugs; good to
be back. Call with Jan wrt. UI translation, dug at transex3 briefly.
- Lunch, more mail, call with JP, ESC call (late) - urgh.
Got back to glibc hackery, trying to make things build. Babes
sick & crying in the night sadly.
- Up early, dealt with babes, off to NCC - did creche with
Dave Staff; back for lunch - all babes to bed. Book reading &
slugging much of the afternoon, Transformers in the evening -
how can it be that J. was ok with it & I disliked it ?
- Up at midday, lunch, towel rail erected in the bathroom
only 3+ years behind schedule, J. suitably pleased.
- Played with children, read stories, slugged, snakes &
ladders, dinner.
- Up earlyish, took H. to school on the bike.
Set too removing the side of the bath - an
attempt to stop the sink filling the bath as it drains: emptied the
U-bend, pretty clear anyway. Extended fun trying to get the overflow
to stop leaking; bother.
- Lunch, out to collect Hannah from school, and on to the
(expensive) Funky Fun House for some romping, clambering,
sliding sort of action. Home, dinner, bed.
- Listened to a rather good sermon with the wifelet
on Genesis 24, which it transpired I heard ~5 years ago in Boston,
in person: re-assuringly the content was somewhat familiar.
- Up early, dealt with babes, set too
fixing the never enough plugs situation in my office:
mounting new power-strip on desk etc. should have enough for
the immediate future at least.
- Laura over for lunch & Lydia too after a bit,
good to catch up with them.
- Lie-in, lunch, J. feeling dreadful went to sleep.
Out into town with a gaggle of babies: traipsed around the place
debugged DVD player S-video output & bought a longer cable.
- On to a car-wash: children enjoyed that of course,
back - H. and N. helped unscrew screws on old radio kicking
around to extract stereo jack socket: turned out to be mono
instead, bother. Cut & spliced cables to create a suitable
cable for the player. Watched DVD - a yet more convenient
projector mounting operation.
- Up early, looked after babes while J. slept.
Packed car and set-off: an incredibly empty motorway sensation:
presumably everyone will attempt to drive at the same time
tonight in some hung-over fashion.
- Home, lunch, rotated bedrooms: Hannah to her
own room, Miriam to the bottom bunk, Naomi to the top:
hopefully no falling out of bed for anyone. Arranged
'pretties' in the new room (small ornaments on a set of
shelves).
- Onwards to dunging out the U-bend and piping
under the kitchen sink: apparently while I'm not looking a
potent combination of fat & glue gets poured down there
to make the task more interesting (and pungent).
- Dinner in the evening, Miriam being acutely
objectionable, not to mention throwing herself on the ground
to hurt herself (presumably to attract sympathy), eventually
all to bed: a blog with artistic rememberance since the 19th.
In case it's not painfully obvious: the reflections reflected here are my
own; mine, all mine ! and don't reflect the views of Novell, The
Lithuanian Gov't or Arnold Schwarzenegger. It's also important to
realise that I'm not in on the Swedish Conspiracy.
Occasionally people ask for formal photos for conferences,
bio.
or fun.
Michael Meeks (michael.meeks@novell.com)
