UK Linux User Groups

Arbitrary tweets made by TheGingerDog (i.e. David Goodwin) up to 01 March 2013

• Why do all network file systems suck so much? (2013/02/26)
• To do list : 1) benchmark NFS clients to stop the minions moaning at work. 2) phpunit + spell check generated html. “Hygine” FTW! (2013/02/26)
• I suppose I’d better stare at the women doing water-robics or something. (2013/02/26)
• Check out the #zombiesrun profile for GingerDog https://t.co/JiRnHMqxv0 (2013/02/25)
• RT @phpclasses: PHP Job: Software Engineer (Mobile) (telecommute) t.co/B8b39UceNE (2013/02/22)
• Apparently generating SSH key pairs is too difficult for someone who claims to be a programmer. *sigh* (2013/02/20, Bromsgrove, Worcestershire)
• At the dentists again. (2013/02/20, Bromsgrove, Worcestershire)
• The boss of Iceland comes across as a tit in one clip on the BBC news website. #horse #fail (2013/02/19, Bromsgrove, Worcestershire)
• Scientific tests have shown it is possible to eat too much chocolate :-/ (2013/02/18, Bromsgrove, Worcestershire)
• Today I’m 22. Sort of. 0×22. (2013/02/17, Bromsgrove, Worcestershire)
• Coo. All clear. For now. #AtLeastMyBloodIsRed (2013/02/14, Bromsgrove, Worcestershire)
• Stupid nose. Stop bleeding! I know I ran for 30 mins after it started. But even so …. Soldering iron welcome #DIYMedicine (2013/02/14, Bromsgrove, Worcestershire)
• For the purposes of science (content testing and to ensure they’re horse free) … (I blame @moreteadoctor ) t.co/0u6VFFtM (2013/02/14, Bromsgrove, Worcestershire)
• Going running has become fun after using @ZombiesRunGame … Not sure what my character is going to do with all the underwear he’s found (2013/02/13, Bromsgrove, Worcestershire)
• RT @mikko: TV Emergency Alert System in Montana hacked to interrupt the broadcast, warning that the dead are rising from graves: http:// … (2013/02/12)
• RT @DrunkRogue68: Resignation refused I think as lightning strikes St Peter’s Basilica as Pope resigns t.co/hRnoDGFF (2013/02/12)
• 3/10 b60 #uksnow thin layer. Sticking. Falling reasonably well. (2013/02/11, Bromsgrove, Worcestershire)
• The trees and things outside are covered in a thin layer of dandruff again. I must remember to buy some better shampoo…. (2013/02/11, Bromsgrove, Worcestershire)
• Perhaps I’ll stop listening to t.co/rSOONpOc soon … one day…. #Groovelectric (2013/02/11)
• It doesn’t seem ideal weather for kite flying. t.co/SsGCQurx (2013/02/09)
• So far this morning – “… You’re not a real [monster,] dad” and “you’re not a proper grown up!”. #children #noParentingBadgesForMe (2013/02/09)
• Weee… faster broadband (~18mbit) is appreciated @BeBroadband … but where’s your FTTC ? (2013/02/08)
• BeThere (ISP) are offering to renew my ADSL for £18/month (saving £4). Yet they have it half price (£11/month) on their front page. Hmm. (2013/02/07)
• It’s tempting to answer the PPI phone calls / texts – just to waste their time / money. (2013/02/06)
• RT @StackParenting: Should you let a toddler win? t.co/96WV3d9T #toddler (2013/02/05)
• I wish I hadn’t started making a work related to do list. It keeps growing. (2013/02/02)

Just a quick note to point something out.

Today the pope will have his ring broken. In fact by the time of writing it is probably already broken.

The ring in question is known as the Ring of the Fisherman. It is personal sign of office of the pope and is there to remind him of the fact that he occupies a job who's first holder was a fisherman.

It is solid gold.

A ring to remind the wearer of the humble origin of his church in the most valuable mental on earth.

Of course not all Popes have gone have been quite that humble. Pope Pius IX had a ring with a picture of himself on it. In diamonds. Proper bling that!



In case anyone thinks I'm getting at the Roman Catholic Church here, I should point out that my own church with it prelates who sit in the House of Lords and live in palaces suffers from exactly this problem.

We use the language of humility more often that we are humble after all one of the things that people are drawn to in  religion is its ceremonies and ancient powers. Jesus however was not a fan of religion and tried his best to beat it out of his followers. It did not take long to come back.

We need to notice this dichotomy at the heart of our contemporary faith  how the church founded on the powerlessness of the cross is now so tainted with the scandals of inappropriate exercise of power. We need to own this our greatest failure and where we cannot be all we should be, be honest about what we are not.




Also I really wanna see the College of Cardinals do the Harlem Shake.

Big shout out to the awesome community over at XDA Developers who have been getting involved in the Ubuntu Touch Port-o-thon to bring the Ubuntu Touch images to more and more devices. Daniel Holbach kicked off the port-o-thon the day after we released the code and images last week, and we are already seeing fantastic work going on.

When the initial announcement hit their forum it generated over a 100 posts within a day and there is currently 101 pages of posts on that thread. There is also an Ubuntu Touch Subforum which has seen over 4000 posts already. We are just blown away by the level of interest.

As you can see on the devices wiki page we are already seeing some fantastic work going on to port Ubuntu Touch to additional devices. Here are some great examples of this work (click each link to see the XDA Developers thread):

• Galaxy Nexus (toro)
• Galaxy Nexus (toroplus)
• Sony Xperia S
• Sony Xperia T
• Samsung Galaxy Note II
• Samsung Galaxy Note
• Asus Transformer Infinity
• Nexus One
• Galaxy Tab 2 10.1 Wifi
• Asus Transformer Pad TF300T

Awesome work!

I asked David Planella and Daniel Holbach on my team to kick off a regular engagement with XDA Developers to help us grow an great relationship together. The first call was today and we are kicking some ideas around of how to work more closely together. Stay tuned for more!

Looking up French figures you can convert European Energy ratings to an annual kWh·m-2:

Rating French kWh·m-2 UK SAP A < 50 92 - 100 B 50 - 90 81 - 91 C 91 - 150 69 - 80 D 151 - 230 55 - 68 E 231 - 330 39 - 54 F 331 - 450 21 - 38 G > 451 1 - 20

Our house basted on last year at 85 kWh·m-2 for a 1936 house is doing very well in band B and not the band E that the idiot surveyers put it in.

Last week someone in the village was asking about installing PV panels on their south facing roof. From 01 April 2013 the government will only allow the full feed in tariff if you have already fully insulated your house - it is obviously pointless generating electricity if your house leaks heat like a sieve.

I checked the EPC chart for our house that came with the HIP when we bought the house. It's quite pathetic, it reports loft insulation of 250 mm when in fact there was only 100 mm when we moved in. It does however note the old boiler with primitive controller and lack of efficient light bulbs. It then gives some bland advice and ranks the house at grade "E" with a SAP score of 48. The best the house could be is apparently grade "D" with a SAP score of 55. That corresponds to an annual energy use of 314 kWh·m-2 falling to a minimum possible of 275 kWh·m-2. Annoyingly there is no explanation of how a SAP rating relates to a energy use per unit area.

This is all utter rubbish! We changed all the old bulbs to mostly CFL, plus some LED and some halogen the day we moved in. Last autumn we put in a modern boiler and controller and started to insulated under the floors. We added 200 mm of loft insulation taking it up to a minimum of 300 mm over the whole loft with the spare in the centre to >400 mm. We have yet to replace the windows and doors (A rated) and complete the under-floor insulation.

For the first full year we were in the house we used 115 kWh·m-2 (a cold year too and with the old central heating boiler & controller), which is better than half the potential minimum energy use and last year we used only 85 kWh·m-2 (mild year plus new boiler for the last 2 cold months).

I am in favour of improving things, but if the official schemes and assessments are so rubbish how are people who don't know ever going to figure out what to do...

Some of you may have seen the news about us transitioning to an online Ubuntu Developer Summit and running the event every three months. If you didn’t see the news, you can read it here. I just wanted to share my personal perspective on this change.

For a long time now I have been attending Ubuntu Developer Summits as part of my work, but for the last event in Copenhagen my wife was about to give birth and so I attended the event remotely. As someone who has been heavily involved in the planning and execution of UDS for the last 10 or so events, I was intimately aware of the remote participation features of the event, but I had never actually utilized them myself. I was excited to dive into the sessions remotely and participate.

For the sessions I dialed into I found the remote participation worked well, but not as well as it could. Sometimes it was a little difficult to hear people (despite us alway encouraging speakers to sit near the middle of the fishbowl), and for the sessions I wasn’t able to actively participate in (due to the timezone differences), only some of those sessions had videos available that I could review after the session had ended. As such, this made it something of a challenge at times to get an overall view of the event; it depended on attendees taking good notes (which generally happens), but I missed the specifics of the discussions.

Remote participation has always been a critical part of UDS and I think it worked efficiently as it could, but these issues were primarily due to the challenge of delivering an in-person event to an online audience and the practicalities therein.

Of course, the real challenge is getting you people to eat these things.

The move to an online event effectively solves the majority of these issues: every single session will be recorded and available for viewing after the fact (which is awesome for not only attendees, but also for the press, partners and others), and with everyone in the hangout facing a webcam and a microphone, the quality of the content should be better too.

For those people who can’t join the session hangout video stream, IRC participation is available, and those IRC discussions will be logged too and provided in addition to the video of the session and the Etherpad notes. This provides a great overview of all the content and discussion in the session.

An online event is also going to open up the event to more potential participants. There are many folks who either can’t physically travel or justify the travel expenses or time away from their work and family commitments who can now participate in the event by simply opening their web browser. With the wide focus in Ubuntu across the desktop, devices and the cloud, we need more specialists rather than fewer to guide us on our mission, and the online event will make it easier for those folks to attend. I think that this will result in wider and more diverse discussion, ultimately helping us to do a better job planning UDS.

Some folks have expressed a concern about not having as much face-to-face time as in a physical event. Of course, video-conferencing will never ultimately replace being in the same room as someone, but I think much of that personal connection is still shared via hangouts. As an example, my team at Canonical used to have team meetings on Skype or a Conference Call and ever since we switched to Google+ Hangouts the sense of personal connection and team spirit has skyrocketed. Sure, it doesn’t replace being in the same room, but when we balance out the benefits of an online event for the reasons I mentioned earlier, it seems like a reasonable trade-off to me.

Iterative Improvements

One thing that many folks don’t see from behind the scenes of planning the physical UDSs is that we have always taken an really rigorous approach to improving and refining the event. This not only includes the structure of the event, but we have iterated after every detail to improve room layouts, A/V needs, timing, remote participation requirements, scheduling patterns, and more. Every detail of UDS has been scrutinized after every event, and the survey we send out is reviewed with a fine tooth comb, all with the goal of squeezing out as much efficiency as possible so the time everyone commits to UDS is as worthwhile as possible.

We are still exploring the alleged productivity-enhancing benefits of light ping-pong.

With UDS previously happening every six months this has helped us to build a pretty bullet proof formula for the physical event, and many attendees comment at each UDS about just how efficient it is and how much gets done. This is largely due to this iterative refinement process.

The first online UDS takes place next week and I think we have a pretty good plan for it, but we are going to go through exactly the same process for reviewing how each event goes and buffing off the rough edges so that works better and more efficiently each time. With us now doing a UDS every three months it should not take too long to get us into a winning formula, and our community are an essential part of helping us to refine these different pieces. As I mentioned in the announcement blog, after the second event we are also going to take a general look to see if an online UDS is serving the needs of the project well in terms of how we plan Ubuntu development.

Got Questions?

I am sure many of you will still have questions about the new format of UDS. Tomorrow (Wednesday) at 7pm UTC. I will be doing my usual weekly Q+A videocast on Ubuntu On Air and will dedicate part of the session to covering how the online event will work and answering your questions. Feel free to bring your UDS and any other questions to the session!

New in libguestfs ≥ 1.21.15, the virt-df and virt-alignment-scan tools now use parallel appliances when scanning your libvirt guests.

The amount of parallelism is selected heuristically when the tool starts up — by dividing the amount of free memory in MB by 500. You can also override this choice by using the new -P option to both tools, but the default should be fine for everyone. -P 1 disables multiple threads.

Users won’t see much difference, although I found that both tools are noticeably faster.

The implementation of threads in these tools is a little bit interesting. Of course there is a pool of worker threads. These take the libvirt guests from a list sorted in alphabetical order and process them.

However each guest takes a variable amount of time to process, and the trick is that the output from each thread mustn’t overlap or be in non-alphabetical order.

The worker threads do two things to ensure this: Firstly output from each guest scan is saved up in an open_memstream buffer. Secondly, domains are retired in order using a pthread condition variable — each worker waits until the previous domain has been retired, before retiring (ie. printing) its own result.

The outcome is that there should be no difference between what the old tools and the rewritten tools print out.

Dear Mail.app: I’m a big fan of your maths. “0 messages, 1 unread”. How does that work, exactly? img.ly/sYlN

Chilling in Jervis Bay. Weather has turned out much better than the forecasts. Lovely.

See the full gallery on Posterous

Permalink | Leave a comment  »

A couple of weeks ago I wrote that we were planning the sixth season of the Ubuntu Podcast. Well, after much curry and tea, we decided to sally forth and put on a show!

The first live recording session of the season will be on Wednesday 27th February – that’s this Wednesday! The first episode of the season will be available to download from the website the following day.

The eagle-eyed will have noticed that we’ve moved the live recording session from a Tuesday evening to a Wednesday evening. Wednesdays fit better with our personal schedules. We know that some people who used to be able to listen live won’t be able to, and we’re sad about that. But there are also people who couldn’t listen on Tuesdays who will now be able to. And we’re pleased about that.

The other big change is that the Ubuntu Podcast will be coming to you weekly. Every week there will be a new episode available from the website, or iTunes or wherever else you find the show. We like to shake things up, it helps keep things interesting for us as well as you, the lovely listeners. Some shows will be more topical, with more analysis of the news and events in the Ubuntu community. Others will include interviews and some returning favourite features from previous seasons.

So join us at 2030 UTC on Wednesday 27th February for the first live recording! http://podcast.ubuntu-uk.org/live

Pin It

[Part 1, part 2, part 3.]

Finally I modified the test to do some representative work: We now load a real Windows XP guest, inspect it (a heavyweight operation), and mount and stat each filesystem. I won’t reproduce the entire test program again because only the test subroutine has changed:

sub test { my $g = Sys::Guestfs->new; $g->add_drive_ro ("/tmp/winxp.img"); $g->launch (); # Inspect the guest (ignore the result). $g->inspect_os (); # Approximate what virt-df does. my %fses = $g->list_filesystems (); foreach (keys %fses) { my $mounted = 0; eval { $g->mount_ro ($_, "/"); $mounted = 1; }; if ($mounted) { $g->statvfs ("/"); $g->umount_all (); } } return $g; }

Even with all that work going on, I was able to inspect more than 1 disk per second on my laptop, and run 60 threads in parallel with good performance and scalability:

A problem encountered in part 2 was that I couldn’t measure the maximum number of parallel libguestfs appliances that can be run at the same time. There are two reasons for that. The simpler one is that libvirt has a limit of 20 connections, which is easily overcome by setting LIBGUESTFS_ATTACH_METHOD=appliance to eliminate libvirt and run qemu directly. The harder one is that by the time the last appliances in the test are starting to launch, earlier ones have already shut down and their threads have exited.

What is needed is for the test to work in two phases: In the first phase we start up all the threads and launch all the appliances. Only when this is complete do we enter the second phase where we shut down all the appliances.

The easiest way to do this is by modifying the test to use a barrier (or in fact to implement a barrier using the condition primitives). See the modified test script below.

With the modified test script I was able to run ≥ 110 and < 120 parallel appliances in ~ 13 GB of free RAM, or around 120 MB / appliance, still with excellent performance and nearly linear scalability:

#!/usr/bin/perl -w use strict; use threads qw(yield); use threads::shared qw(cond_broadcast cond_wait lock); use Sys::Guestfs; use Time::HiRes qw(time); my $nr_threads_launching :shared; sub test { my $g = Sys::Guestfs->new; $g->add_drive_ro ("/dev/null"); $g->launch (); return $g; } # Get everything into cache. test (); test (); test (); sub thread { my $g = test (); { lock ($nr_threads_launching); $nr_threads_launching--; cond_broadcast ($nr_threads_launching); cond_wait ($nr_threads_launching) until $nr_threads_launching == 0; } $g->close (); } # Test increasing numbers of threads until it fails. for (my $nr_threads = 10; $nr_threads < 1000; $nr_threads += 10) { my $start_t = time (); $nr_threads_launching = $nr_threads; my @threads; foreach (1..$nr_threads) { push @threads, threads->create (\&thread) } foreach (@threads) { $_->join (); if (my $err = $_->error ()) { die "launch failed with $nr_threads threads: $err" } } my $end_t = time (); printf ("%d %.2f\n", $nr_threads, $end_t - $start_t); }

One problem with the previous test is that I hit a limit of 20 parallel appliances and mistakenly thought that I’d hit a memory limit. In fact libvirt out of the box limits the number of client connections to 20. You can adjust libvirt’s limit by editing /etc/libvirt/libvirtd.conf, but easier for us is to simply eliminate libvirt from the equation by doing:

export LIBGUESTFS_ATTACH_METHOD=appliance

which causes libguestfs to run qemu directly. In my first test I reached 48 parallel launches before I killed the program (because that’s a lot of parallelism and there seemed no end in sight). Scalability of the libguestfs / qemu combination was excellent again:

But there’s more! (In the next part …)

I wrote the Perl script below to find out how many libguestfs appliances we can start in parallel. The results are surprising (-ly good):

What’s happening here is that we’re booting up a KVM guest with 500 MB of memory, booting the Linux kernel, booting a minimal userspace, then shutting the whole lot down. And then doing that in parallel with 1, 2, .. 20 threads.

[Note: Hardware is my Lenovo x230 laptop with an Intel Core(TM) i7-3520M CPU @ 2.90GHz, 2 cores with 4 threads, 16 GB of RAM with approx. 13 GB free. Software is: Fedora 18 with libguestfs 1.20.2, libvirt 1.0.2 (from Rawhide), qemu 1.4.0 (from Rawhide)]

The test fails at 21 threads because there isn’t enough free memory, so each qemu instance is allocating around 660 MB of RAM. This is wrong: It failed because libvirt out of the box limits the maximum number of clients to 20. See next part in this series.

Up to 4 parallel launches, you can clearly see the effect of better utilization of the parallelism of the CPU — the total elapsed time hardly moves, even though we’re doing up to 4 times more work.

#!/usr/bin/perl -w use strict; use threads; use Sys::Guestfs; use Time::HiRes qw(time); sub test { my $g = Sys::Guestfs->new; $g->add_drive_ro ("/dev/null"); $g->launch (); } # Get everything into cache. test (); test (); test (); # Test increasing numbers of threads until it fails. for my $nr_threads (1..100) { my $start_t = time (); my @threads; foreach (1..$nr_threads) { push @threads, threads->create (\&test) } foreach (@threads) { $_->join (); if (my $err = $_->error ()) { die "launch failed with nr_threads = $nr_threads: $err" } } my $end_t = time (); printf ("%d %.2f\n", $nr_threads, $end_t - $start_t); }