UK Linux User Groups

https://www.sciencedirect.com/science/article/pii/S0168900297002118

Oops.
A few months ago, we (that is: Jono and I, the greatest sysadmin team the world have ever known) moved various things around on various servers. And in the course of this action, we completely forgot to put the Shot of Jaq website somewhere. So shotofjaq.org currently is down.
As I say, oops.
Anyway, we haven’t lost the audio (we’re not that bad), so I trawled archive.org for all the episode descriptions and threw them together into a brief listing of all the SoJ episodes with download links. You can therefore see Shot of Jaq again at http://www.kryogenix.org/shotofjaq.html.
Sorry about that, all. We’re rubbish. Let this be a lesson to you.

This tweet has to have been the most popular thing I have ever said. At time of posting it has gained 80 retweets, 25 favourites and many replies/questions.

Work recently bought me a new workstation, so the 1st thing I always do is to dual boot with Ubuntu.

Some might consider me an edge case user. Though as a developer, I like a rather particular set-up. That is, 3 wide screen monitors with the central one rotated 90 degrees for my IDE.

This is something that Windows gets right without having to dig about installing things. While Linux distros have always struggled (in my experience).

Because my tweet gained quite a few questions, I thought it best to reply to them here for everybody to see.

@ankitvad asks what specs. I use for Ubuntu.
Titanium Rimless Glasses from Spex4Less.com
Couldn’t resist, sorry

Dell Alienware X51
CPU: i7
Memory: 8GB
Graphics card: nVidia GeForce GTX 660
Storage: 1TB HDD (Windows) 120GB SSD (Linux)
Mouse: Logictech M570 trackball
OSs: Window 7 SP1, Ubuntu 12.10 64bit
Monitors: 2 x 22″ Dell, 1 x 22″ LG

All 3 monitors are connected to the one graphics card. Two by DVI and one by HDMI.

As I said, this is a working system from a fresh install without updates being applied or any 3rd party packages installed. So the default graphics driver is doing quite well these days

The only downside to this is that the default graphics driver is dog slow and won’t let me play games on Steam The next step will be to get the nVidia binary driver working.

Well, that's another FOSDEM over with. In general this year seemed the same as the last couple of years but slightly bigger than usual (although it seems that way every year). The (newish) K building was in full swing with dozens of project stalls and dev rooms. The usual suspects - virtualisation / cloud, configuration management and MySQL rooms had nearly as many people trying to get in to the rooms as they did sitting down.

I think some of the main dev rooms have reached the level of popularity that forces you to either arrive early, get a seat and not move for the rest of the day or accept a very high level of probability that you won't get to see the talks you want. I know a few of us had trouble cherry picking sessions across tracks - which obviously means we have excellent taste in topics. I wonder if having the same talks on both days would make it easier to move around as a visitor - you'd attempt to catch it the first time and if that fails, come back tomorrow. I realise however that this puts even more of a burden on speakers that graciously give their own time in both the preparation and performing of their talks. It does seem that scaling the rooms is the problem of the day once again.

I'd like to say a big thank you to all the organisers, speakers and other attendees for making it another enjoyable couple of days. See you next year.

Lewisham High Street nagging Subway. #horsemeat #subway http://t.co/8mSnbdrq

Ouch. The Heroku Swindle Factor: rapgenius.com/James-somers-h…

Customer: "I've been sent an email from Russia but ESET Anti Virus has quarantined it."
Engineer: "And?"
Customer: "Well I want to read it."
Engineer: "But it clearly has a virus?"
Customer: "Can you read it?"
Engineer: "No, it has a virus."
Customer: "Should I get them to send it again then?"
Engineer: "You can but I suspect that too will be infected."
Customer: "Is there a way to stop ESET quarantining it?"
Engineer: "Yes, get the sender to clean the viruses off their machine."

Mankind has reached it's evolutionary peak.

HP doomed to repeat mistakes of their past. Previously tied their fate to Microsoft; now tying their fate to Google. readwrite.com/2013/02/13/hp-…

Start: 2013-12-14 11:00 End: 2013-12-14 11:00

We have regular sessions on the second Saturday of each month. Location this month yet to be arranged.

Who

New members are very welcome. We're not a cliquey bunch, so you won't feel out of place! Usually between 10 and 30 people come along. 

Start: 2013-11-09 11:00 End: 2013-11-09 11:00

We have regular sessions on the second Saturday of each month. Location this month yet to be arranged.

Who

New members are very welcome. We're not a cliquey bunch, so you won't feel out of place! Usually between 10 and 30 people come along. 

Start: 2013-10-12 11:00 End: 2013-10-12 11:00

We have regular sessions on the second Saturday of each month. Location this month yet to be arranged.

Who

New members are very welcome. We're not a cliquey bunch, so you won't feel out of place! Usually between 10 and 30 people come along. 

Start: 2013-09-14 11:00 End: 2013-09-14 11:00

We have regular sessions on the second Saturday of each month. Location this month yet to be arranged.

Who

New members are very welcome. We're not a cliquey bunch, so you won't feel out of place! Usually between 10 and 30 people come along. 

Start: 2013-08-10 11:00 End: 2013-08-10 11:00

We have regular sessions on the second Saturday of each month. Location this month yet to be arranged.

Who

New members are very welcome. We're not a cliquey bunch, so you won't feel out of place! Usually between 10 and 30 people come along. 

Start: 2013-07-13 11:00 End: 2013-07-13 11:00

We have regular sessions on the second Saturday of each month. Location this month yet to be arranged.

Who

New members are very welcome. We're not a cliquey bunch, so you won't feel out of place! Usually between 10 and 30 people come along. 

To clarify, what is the memory overhead, or how many guests can you cram onto a single host, memory being the typical limiting factor when you virtualize.

This was the question someone asked at work today. I don’t know the answer either, but the small program I wrote (below) aims to find out. If you believe the numbers below from qemu 1.2.2 running on Fedora 18, then the overhead is around 150 MB per qemu process that cannot be shared, plus around 200 MB per host (that is, shared between all qemu processes).

guest size 256 MB: Shared memory backed by a file: 201.41 MB Anonymous memory (eg. malloc, COW, stack), not shared: 404.20 MB Shared writable memory: 0.03 MB guest size 512 MB: Shared memory backed by a file: 201.41 MB Anonymous memory (eg. malloc, COW, stack), not shared: 643.76 MB Shared writable memory: 0.03 MB guest size 1024 MB: Shared memory backed by a file: 201.41 MB Anonymous memory (eg. malloc, COW, stack), not shared: 1172.38 MB Shared writable memory: 0.03 MB guest size 2048 MB: Shared memory backed by a file: 201.41 MB Anonymous memory (eg. malloc, COW, stack), not shared: 2237.16 MB Shared writable memory: 0.03 MB guest size 4096 MB: Shared memory backed by a file: 201.41 MB Anonymous memory (eg. malloc, COW, stack), not shared: 4245.13 MB Shared writable memory: 0.03 MB

The number to pay attention to is “Anonymous memory” since that is what cannot be shared between guests (except if you have KSM and your guests are such that KSM can be effective).

There are some known shortcomings with my testing methodology that I summarise below. You may be able to see others.

1. We’re testing a libguestfs appliance. A libguestfs appliance does not have the full range of normal qemu devices that a real guest would have, and so the overhead of a real guest is likely to be higher. The main difference is probably lack of a video device (so no video RAM is evident).
2. This uses virtio-scsi. Real guests use IDE, virtio-blk, etc which may have quite different characteristics.
3. This guest has one user network device (ie. SLIRP) which could be quite different from a real network device.
4. During the test, the guest only runs for a few seconds. A normal, long-running guest would experience qemu memory growth or even memory leaks. You could fix this relatively easily by adding some libguestfs busy-work after the launch.
5. The guest does not do any significant writes, so during the test qemu won’t be storing any cached or in-flight data blocks.
6. It only accounts for memory used by qemu in userspace, not memory used by the host kernel on behalf of qemu.
7. The effectiveness or otherwise of KSM is not tested. It’s likely that KSM depends heavily on your workload, so it wouldn’t be fair to publish any KSM figures.
8. The script uses /proc/PID/maps but it would be better to use smaps so that we can see how much of the file-backed copy-on-write segments have actually been copied. Currently the script overestimates these by assuming that (eg) all the data pages from a library would be dirtied by qemu.

Another interesting question would be whether qemu is getting better or worse over time.

#!/usr/bin/perl -w # Estimate memory usage of qemu-kvm at different guest RAM sizes. # By Richard W.M. Jones <rjones@redhat.com> use strict; use Sys::Guestfs; no warnings "portable"; # 64 bit platform required. # Loop over different guest RAM sizes. my $mbytes; for $mbytes (256, 512, 1024, 2048, 4096) { print "guest size ", $mbytes, " MB:\n"; my $g = Sys::Guestfs->new; # Ensure we're using the direct qemu launch backend, otherwise # libvirt stops us from finding the qemu PID. $g->set_attach_method ("appliance"); # Set guest memory size. $g->set_memsize ($mbytes); # Enable user networking just to be more like a "real" guest. $g->set_network (1); # Launch guest with one dummy disk. $g->add_drive ("/dev/null"); $g->launch (); # Get process ID of qemu. my $pid = $g->get_pid (); die unless $pid > 0; # Read the memory maps of the guest. open MAPS, "/proc/$pid/maps" or die "cannot open memory map of pid $pid"; my @maps = <MAPS>; close MAPS; # Kill qemu. $g->close (); # Parse the memory maps. my $shared_file_backed = 0; my $anonymous = 0; my $shared_writable = 0; my $map; foreach $map (@maps) { chomp $map; if ($map =~ m/ ^([0-9a-f]+)-([0-9a-f]+) \s (....) \s [0-9a-f]+ \s ..:.. \s (\d+) \s+ (\S+)? /x) { my ($start, $end) = (hex $1, hex $2); my $size = $end - $start; my $mode = $3; my $inode = $4; my $filename = $5; # could also be "[heap]", "[vdso]", etc. # Shared file-backed text: r-xp, r--p, etc. with a file backing. if ($inode != 0 && ($mode eq "r-xp" || $mode eq "r--p" || $mode eq "---p")) { $shared_file_backed += $size; } # Anonymous memory: rw-p. elsif ($mode eq "rw-p") { $anonymous += $size; } # Writable and shared. Not sure what this is ... elsif ($mode eq "rw-s") { $shared_writable += $size; } # Ignore [vdso], [vsyscall]. elsif (defined $filename && ($filename eq "[vdso]" || $filename eq "[vsyscall]")) { } # Ignore ---p with no file. What's this? elsif ($inode == 0 && $mode eq "---p") { } # Ignore kvm-vcpu. elsif ($filename eq "anon_inode:kvm-vcpu") { } else { warn "warning: could not parse '$map'\n"; } } else { die "incorrect maps format: '$map'"; } } printf("Shared memory backed by a file: %.2f MB\n", $shared_file_backed / 1024.0 / 1024.0); printf("Anonymous memory (eg. malloc, COW, stack), not shared: %.2f MB\n", $anonymous / 1024.0 / 1024.0); printf("Shared writable memory: %.2f MB\n", $shared_writable / 1024.0 / 1024.0); print "\n"; }

OK - so I've just been reading the Gaurdian Article on Terry Deary saying that Libraries are outdated and should be got rid of. I entirely disagree with him, I spent a fair amount of my youth borrowing books from my local library, when I had no income, and so the only alternative to borrowing from the library would have been saving up for second hand books... which he wouldn't have seen a penny from. These days I buy Quite A Few books (OK - so, mostly on the kindle these days, but still), if it hadn't have been for libraries when I was younger and couldn't afford to purchase books, I may never have properly picked up the reading habit.

His claim that they're killing bookshops is also, in my opinion, entirely wrong. Bookshops are closing more because of the ease of ordering books online and getting them delivered to your door, with a huge collection of books available from large warehouses rather than the stock that a book store can sell easily. We've got a vast collection of literature available to us now, and it's only ever going upwards, no book shop or single library is going to be able to cater for the entirely different needs of their customers. Libraries do inter library lending, which means that the collection of rarer, less popular books are still available (potentially with a bit of a wait), and every time the book is lent the author gets some funds. If he seriously thinks that if libraries closed the number of people reading his material would stay the same I think he's mistaken. He also doesn't seem to take in to account at all the second hand book market.

All I have to say is NYARGH.