LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Tue, 07/05/2013 - 13:30

libguestfs has high quality Python bindings. Using rpyc you can make a remote libguestfs server with almost no effort at all.

Firstly start an rpyc server:

$ /usr/lib/python2.7/site-packages/rpyc/servers/ [SLAVE INFO 13:21:17 tid=140019939981120] server started on [SLAVE INFO 13:21:17 tid=140019784894208] started background auto-register thread (interval = 60) [REGCLNT INFO 13:21:17] registering on [REGCLNT WARNING 13:21:19] no registry acknowledged

Now, possibly from the same machine or some other machine, you can connect to this server and use Python objects remotely as if they were local:

$ python Python 2.7.3 (default, Aug 9 2012, 17:23:57) [GCC 4.7.1 20120720 (Red Hat 4.7.1-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import rpyc >>> c = rpyc.classic.connect('localhost')

You can now create a libguestfs handle, following the example here.

>>> g = c.modules.guestfs.GuestFS() >>> g.version() {'release': 36L, 'major': 1L, 'minor': 21L, 'extra': 'fedora=20,release=1.fc20,libvirt'} >>> g.add_drive('/dev/fedora/f18x64', readonly=True) >>> g.launch() >>> roots = g.inspect_os() >>> g.inspect_get_product_name(roots[0]) 'Fedora release 18 (Spherical Cow)' >>> g.inspect_get_mountpoints(roots[0]) [('/', '/dev/mapper/fedora-root'), ('/boot', '/dev/sda1')]

As you can see, the g object is transparently remoted without you needing to do anything.

Categories: LUG Community Blogs

Dean Wilson: Deprecation Warnings From Puppet Resources

Planet GLLUG - Sat, 27/04/2013 - 12:53
Over time parts of your puppet manifests will become unneeded. You might move a cronjob or a users in to a package or no longer need a service to be enabled after a given release. I've recently had this use case and had two options - either rely on comments in the Puppet code and write an out of band tool to scan the code base and present a report or add them to the puppet resources themselves. I chose the latter.

Below you'll find a simple metaparameter (a parameter that works with any resource type) that adds this feature to puppet. As this is an early prototype I've hacked it directly in to my local puppet fork. Below you'll see a sample resource that declares a deprecation date and message, the code that implements it and a simple command line test you can run to confirm it works.

# sample puppet resource using :deprecation file { '/ec/cron.d/remove_foos': ensure => 'file', source => 'puppet:///modules/foo/foo.cron', deprecation => '20130425:Release 6 removes the need for the foo cronjob', } $ sudo vi puppet-3.1.1/lib/puppet/type.rb newmetaparam(:deprecation) do desc " Add a deprecation warning to resources. file { '/etc/foo': content => 'Bar', deprecation => '20130425:We no longer need the foo' } The deprecation comes in two parts, separated by a : The date is in format YYYYMMDD and the message is a free form string. " munge do |deprecation| date, message = deprecation.split(':') # YYY MM DD - one true timestamp now ='%Y%m%d') if (now >= date) rsrc = "#{@resource.type.capitalize}[#{}]" Puppet.warning "#{rsrc} expired on #{date}: #{message}" end end end # command line test $ puppet apply -e 'file { "/tmp/dep": content => "foo\n", deprecation => "20120425:We can remove this file after release 4" }' Warning: File[/tmp/dep] expired on 20120425: We can remove this file after release 4 Notice: Finished catalog run in 0.06 seconds

Using the metaparameter is easy enough, just specify 'deprecation' as a property on a resource and provide a string that contains the date to start flagging the deprecation on (in YYYYMMDD format) and the message puppet should show. I don't currently fail the run on an expired resource but this is an option.

The are some other aspects of this to consider - Richard Clamp raised the idea of having a native type that could indicate this for an entire class (I'd rather use a function, but only because they are much easier to write) and Trevor Vaughan suggested a Puppet face that could present a report of the expired, and soon to be expired, code.

I don't know how widely useful this is but it made a nice change to write some puppet code. The small size of the example will hopefully show how easy it is to extend nearly every part of puppet - including more 'complicated' aspects like metaparameters. Although not the relationship ones, those are horrible ;) I've submitted the idea to the upstream development list so we'll see what happens.

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Fri, 26/04/2013 - 19:17

Brian is Red Hat’s CTO, and hence my boss’s boss’s boss (or something like that). This is a pretty good (and honest) talk about Red Hat’s plans for OpenStack.

Edit: By the way, the thumbnail (the one I see at any rate) is not Brian.

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Tue, 23/04/2013 - 20:46

There are some items which are just not possible to order online any more, assuming you want something of minimal quality. These include:

  1. chargers for mobile phones — all now fake as far as I can tell
  2. batteries for older mobile phones — second hand, dead, fake, or simply the wrong battery
  3. consumer cables, like USB to micro USB
  4. convection heaters — didn’t expect this, but I can’t find a reliable one (for the new office) online

That’s not counting the stuff where it’s still just about possible to get non-fake stuff, but it’s a crapshoot, eg. computer memory, hard disks, batteries, flash memory.

Categories: LUG Community Blogs

Richard WM Jones: 20130423_155905.jpg

Planet GLLUG - Tue, 23/04/2013 - 16:00

Mike the electrician is here wiring up the garden office to the mains (I’m doing the networking). The armoured cable (black) and the two cat7 network cables (yellow) will be buried in this trench:

They go in through the wall. This will later be hidden with metal conduit. That wall is 12 inches thick, and close behind it is the gas main so he had to be very careful where he was drilling.

Finally the armoured cable finishes next to the consumer unit (to the right — not shown). Look at the lovely 1950s-era mains supply:

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Mon, 22/04/2013 - 10:09

Since libguestfs 1.20 it has been possible to use rsync to upload or download files to a disk image incrementally. This is one way to do backups, but note that it won’t work on live guests unless you take a snapshot.

rsync involves using a network connection into or out of the appliance, and is therefore a lot more involved to set up. The script below shows one way to do this, by running an rsync daemon on the host, and letting the libguestfs appliance connect to it.

The script runs rsync inside the appliance, copying /home from the attached disk image out to /tmp/backup on the host. If the operation is repeated, then only incrementally changed files will be copied out. (To incrementally delete files on the target, add the deletedest:true flag).

Note you will need to open port 2999 on your host’s firewall for this to work.

#!/bin/bash - set -x # The target directory. mkdir -p /tmp/backup # Create the daemon. rm -f /tmp/ cat <<EOF > /tmp/rsyncd.conf port = 2999 pid file = /tmp/ [backup] path = /tmp/backup use chroot = false read only = false EOF rsync --daemon --config=/tmp/rsyncd.conf # Run guestfish and attach to the guest. guestfish --ro --network -a /dev/fedora/f19rawhidex32 -i <<EOF trace on rsync-out /home rsync://rjones@ archive:true EOF # Kill the rsync daemon. kill `cat /tmp/`
Categories: LUG Community Blogs

Richard WM Jones: 20130420_175710.jpg

Planet GLLUG - Sat, 20/04/2013 - 17:59

Inside there’s a ring for sockets and a lighting circuit. The windows and doors are pre-made double-glazed units.

Categories: LUG Community Blogs

Richard WM Jones: image

Planet GLLUG - Fri, 19/04/2013 - 15:15

Categories: LUG Community Blogs

Richard WM Jones: 20130419_094942.jpg

Planet GLLUG - Thu, 18/04/2013 - 18:39

Update: Further progress this morning …

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Thu, 18/04/2013 - 17:26

Dominic Cleal’s short introduction to the Augeas configuration API.

We use Augeas a lot in libguestfs and virt-v2v, and it’s been very effective for us.

I asked Dominic how he made this video.

He uses gtk-recordmydesktop, max 100/100 audio/video quality, 30fps, 2 channel audio at 48kHz.

Sound and video are recorded at the same time, with a Sennheiser headset.

Editing is done in kdenlive.

Categories: LUG Community Blogs

Richard WM Jones: 20130418_145150.jpg

Planet GLLUG - Thu, 18/04/2013 - 14:55

The cabin has a double wall which will contain insulation. Shown here is the ring main for the four sets of sockets around the room.

Categories: LUG Community Blogs

Richard WM Jones: 20130417_170829.jpg

Planet GLLUG - Thu, 18/04/2013 - 13:17

This trench will take 1 armoured mains cable and 2 cat 7 network cables.

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Wed, 17/04/2013 - 14:31

Also (not shown) they dug a trench down the side of the garden for electrics and network cables. We’re going to run some hefty armoured cable, plus two or three cat 7 network cables, covered with bricks and warning tape, at a depth of 18″ (mandated by building regulations).

Categories: LUG Community Blogs

Richard WM Jones: office-3

Planet GLLUG - Tue, 16/04/2013 - 16:11

As some readers might know, I work from home, and have worked from home for about 12 years. However I’ve never had a proper dedicated office, just a corner of a second bedroom or a bit of space on a sofa. That’s about to change.

Tomorrow, work starts on constructing a real office in my garden.

Work already started (for me) about a month ago, when I took down a shed and jack-hammered a patio to bits:

Shifting the whole lot into two skips:

Resulting in this empty, mostly flat space (about 20′ wide and 16′ deep, damn you council for chopping that tree down at the back …):

To be continued …

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Tue, 16/04/2013 - 15:23

New in libguestfs ≥ 1.21.30 is the ability to use guestfish and some of the virt tools with remote disks.

Currently you can use remote disks over NBD, GlusterFS, Ceph, Sheepdog and (recently upstream) SSH.

For this example I’ll use SSH because it needs no setup, although this requires absolutely the latest qemu and libguestfs (both from git).

Since we don’t have libvirt support for ssh yet, so this only works with the direct backend:

$ export LIBGUESTFS_BACKEND=direct

I can use a ssh:// URI to add disks with guestfish, guestmount and most of the virt tools. For example:

$ virt-rescue -a ssh://localhost/tmp/f17x64.img [... lots of boot messages ...] Welcome to virt-rescue, the libguestfs rescue shell. Note: The contents of / are the rescue appliance. You have to mount the guest's partitions under /sysroot before you can examine them. ><rescue> mount /dev/vg_f17x64/lv_root /sysroot ><rescue> cat /sysroot/etc/redhat-release Fedora release 17 (Beefy Miracle)

Apart from being a tiny bit slower, it just works as if the disk was local:

$ virt-df -a ssh://localhost/tmp/f17x64.img Filesystem 1K-blocks Used Available Use% f17x64.img:/dev/sda1 487652 63738 398314 14% f17x64.img:/dev/vg_f17x64/lv_root 28316680 4285576 22586036 16% $ guestmount -a ssh://localhost/tmp/f17x64.img -i /tmp/mnt $ ls /tmp/mnt bin dev home lib64 media opt root sbin sys usr boot etc lib lost+found mnt proc run srv tmp var $ cat /tmp/mnt/etc/redhat-release Fedora release 17 (Beefy Miracle) $ guestunmount /tmp/mnt
Categories: LUG Community Blogs

Rev. Simon Rumble: Snorkelling photos

Planet GLLUG - Mon, 15/04/2013 - 23:41

I've been getting into snorkelling a bit recently. I've always enjoyed it but recently I bought some good quality gear, replacing the toy shop crap I've been using. It's another world with good equipment! It's not easy to get time, but so far I've snorkelled Jervis Bay, Bushrangers Bay, Clovelly and The Haven in Terrigal.

My son has been asking what it's like, so I bought the Kogan waterproof camera case for $19. Took it out last weekend for a spin at The Haven but the visibility was terrible. The camera case works a treat though, and I'm looking forward to using it some more. Need to work out a strap to attach it to my arm or something though.

See the full gallery on Posterous

Permalink | Leave a comment  »

Categories: LUG Community Blogs

Karanbir Singh: changes

Planet GLLUG - Fri, 12/04/2013 - 00:08


In the last 45 days,'s 2 public facing machines delivered just under 66 TiB of data. So while we try and spread this load a bit ( its growing at 25 - 35% month on month ), we've had to make a few changes.

Firstly, isos are no longer directly downloadable from, you will need to go the torrent route if you want older, deprecated release isos

Secondly, we've turned off multi range requests ( httpd will still accept upto 5 range's, and then block after that )

Over the next few days, we are going to recycle some of the larger disk nodes into; If someone wants to contribute to this effort, please come find us on in channel #centos-devel or #centos-mirror or tweet us @centos or email us the address mentioned at - but keep in mind that need machines with more than 1 TiB of usable space, and more than 300mbps of network capacity, and since we will consume that bandwidth high density hosting facilities with high contention on the links wont work.

- KB

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Fri, 05/04/2013 - 21:56

The qemu ssh block device is now up to version 7 … although sadly not upstream yet.

Nevertheless by applying this patch to libguestfs you can use libguestfs to access remote disks over ssh:

$ export LIBGUESTFS_QEMU=~/d/qemu/qemu.wrapper $ export LIBGUESTFS_BACKEND=direct $ ./run ./fish/guestfish Welcome to guestfish, the libguestfs filesystem interactive shell for editing virtual machine filesystems. Type: 'help' for help on commands 'man' to read the manual 'quit' to quit the shell ><fs> add /tmp/f17x64.img readonly:true format:raw \ protocol:ssh server:onuma ><fs> run 100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ 00:00 ><fs> inspect-os /dev/vg_f17x64/lv_root ><fs> inspect-get-product-name /dev/vg_f17x64/lv_root Fedora release 17 (Beefy Miracle) ><fs> list-filesystems /dev/sda1: ext4 /dev/vg_f17x64/lv_root: ext4 /dev/vg_f17x64/lv_swap: swap ><fs> mount /dev/vg_f17x64/lv_root / ><fs> cat /etc/redhat-release Fedora release 17 (Beefy Miracle)

Everything just works as if this were a local disk.

There are a couple of minor caveats (the major caveat being none of this is upstream): Firstly you have to have ssh-agent set up. Secondly the remote host must be in your known_hosts file (if not, do ssh remotehost first to add it).

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Thu, 04/04/2013 - 21:47

Although grub support in libguestfs is currently on hold because of an unfortunate situation, the latest libguestfs now supports SYSLINUX and EXTLINUX, which is (let’s be frank about this) a much simpler and more sane bootloader than grub/grub2.

In fact, you can make a bootable Linux guest real easily now. Here’s a script:

#!/usr/bin/perl # Copyright (C) 2013 Red Hat Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. # This ambitious script creates a complete, bootable guest. use strict; use warnings; use Sys::Guestfs; my $disk = "syslinux-guest.img"; # Find prerequisites. my $mbr = "/usr/share/syslinux/mbr.bin"; unless (-f $mbr) { $mbr = "/usr/lib/syslinux/mbr.bin"; unless (-f $mbr) { die "$0: mbr.bin (from SYSLINUX) not found\n"; } } print "mbr: $mbr\n"; my $mbr_data; { local $/ = undef; open MBR, "$mbr" or die "$mbr: $!"; $mbr_data = <MBR>; } die "invalid mbr.bin" unless length ($mbr_data) == 440; my $kernel = `ls -1rv /boot/vmlinuz* | head -1`; chomp $kernel; unless ($kernel) { die "$0: kernel could not be found\n"; } print "kernel: $kernel\n"; print "writing to: $disk ...\n"; # Create the disk. unlink "$disk"; open DISK, ">$disk" or die "$disk: $!"; truncate DISK, 100*1024*1024; close DISK; my $g = Sys::Guestfs->new (); $g->add_drive ($disk, format => "raw"); $g->launch (); unless ($g->feature_available (["syslinux"])) { die "$0: 'syslinux' feature not available in this version of libguestfs\n"; } # Format the disk. $g->part_disk ("/dev/sda", "mbr"); $g->mkfs ("msdos", "/dev/sda1"); $g->mount ("/dev/sda1", "/"); # Install the kernel. $g->upload ($kernel, "/vmlinuz"); # Install the SYSLINUX configuration file. $g->write ("/syslinux.cfg", <<_END); DEFAULT linux LABEL linux SAY Booting the kernel from /vmlinuz KERNEL vmlinuz APPEND ro root=/dev/sda1 _END $g->umount_all (); # Install the bootloader. $g->pwrite_device ("/dev/sda", $mbr_data, 0); $g->syslinux ("/dev/sda1"); $g->part_set_bootable ("/dev/sda", 1, 1); # Finish off. $g->shutdown ();

After running the script, you can try booting the minimal “guest” (note it only contains a kernel, not any userspace):

$ qemu-kvm -hda syslinux-guest.img
Categories: LUG Community Blogs
Syndicate content