LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Sat, 11/05/2013 - 17:19

In libguestfs ≥ 1.21.38 you can access at least some iSCSI disks.

On my server (RHEL 6 in this case) I create an iSCSI target backed by a Windows XP disk image:

# service tgtd start Starting SCSI target daemon: [ OK ] # tgtadm --lld iscsi --op new --mode target --tid 1 \ -T # chcon system_u:object_r:tgtd_var_lib_t:s0 /tmp/winxp.img # tgtadm --lld iscsi --op new --mode logicalunit --tid 1 \ --lun 1 -b /tmp/winxp.img # tgt-admin -s ...

Previously I opened port 3250 on the server. Because libguestfs doesn’t yet support authentication against the iSCSI server, I had to bypass that:

# tgtadm --lld iscsi --mode target --op bind --tid 1 -I ALL

Now on the client, I can connect to the iSCSI target using libguestfs like this:

$ export LIBGUESTFS_BACKEND=direct $ guestfish --format=raw -a iscsi://x.x.x.x/ -i Welcome to guestfish, the guest filesystem shell for editing virtual machine filesystems and disk images. Type: 'help' for help on commands 'man' to read the manual 'quit' to quit the shell Operating system: Microsoft Windows XP /dev/sda1 mounted on / ><fs> ll / total 1573209 drwxrwxrwx 1 root root 4096 Apr 16 2012 . drwxr-xr-x 23 1000 1000 4096 May 11 17:16 .. -rwxrwxrwx 1 root root 0 Oct 11 2011 AUTOEXEC.BAT -rwxrwxrwx 1 root root 0 Oct 11 2011 CONFIG.SYS drwxrwxrwx 1 root root 4096 Oct 11 2011 Documents and Settings -rwxrwxrwx 1 root root 0 Oct 11 2011 IO.SYS -rwxrwxrwx 1 root root 0 Oct 11 2011 MSDOS.SYS -rwxrwxrwx 1 root root 47564 Apr 14 2008 NTDETECT.COM drwxrwxrwx 1 root root 4096 Oct 11 2011 Program Files drwxrwxrwx 1 root root 4096 Oct 11 2011 System Volume Information drwxrwxrwx 1 root root 28672 Oct 11 2011 WINDOWS -rwxrwxrwx 1 root root 211 Oct 11 2011 boot.ini -rwxrwxrwx 1 root root 250048 Apr 14 2008 ntldr -rwxrwxrwx 1 root root 1610612736 Oct 11 2011 pagefile.sys
Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Fri, 10/05/2013 - 16:18

(Thanks to Jeff Cody for both writing the code and helping me to get it to work).

New in qemu 1.5 is read-only support for Hyper-V‘s native disk format, vhdx. You can now open vhdx files in libguestfs just like any other:

$ guestfish --ro --format=vhdx -a /tmp/f18x64.vhdx -i Welcome to guestfish, the guest filesystem shell for editing virtual machine filesystems and disk images. Type: 'help' for help on commands 'man' to read the manual 'quit' to quit the shell Operating system: Fedora release 18 (Spherical Cow) /dev/fedora/root mounted on / /dev/sda1 mounted on /boot


  1. As stated above, you will need the very latest qemu for this to work.
  2. You must specify --format=vhdx at the moment because of a missing feature in libvirt.
  3. You must specify --ro because writing is not supported (by the qemu driver).

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Thu, 09/05/2013 - 15:11

Situation: You have a Windows DVD (or ISO), but like any sane person in 2013 you don’t have a DVD drive on the computer. You want to convert the Windows DVD into a bootable USB key. There are many recipes for this online, but they all require another Windows machine and of course cannot be automated.

However with guestfish (and the always brilliant SYSLINUX doing most of the heavy lifting), this script will unpack the ISO and turn it into a bootable USB key.


  1. I am not going to support this script. You will need to read the script, look up the commands in the guestfish man page, and understand what it does. Any requests for help will be deleted unread.
  2. You need to edit the USB key device before using the script.
  3. You need libguestfs ≥ 1.21 with SYSLINUX support compiled in.
#!/bin/bash - guestfish <<'EOF' trace on add-ro en_microsoft_hyper-v_server_2012_x64_dvd_915600.iso # NB: The next line MUST be changed to your USB drive. # ANYTHING ON THIS DRIVE WILL BE OVERWRITTEN WITHOUT WARNING. add /dev/sdX run # Inside the appliance, /dev/sda = DVD, /dev/sdb = USB. # THESE ARE NOT RELATED TO HOST DISK NAMES. echo "Partitioning the USB disk ..." part-init /dev/sdb mbr part-add /dev/sdb p 63 -1 part-set-mbr-id /dev/sdb 1 0xb part-set-bootable /dev/sdb 1 true mkfs vfat /dev/sdb1 echo "Copying the contents of the DVD to the USB key ..." mkmountpoint /cd mkmountpoint /usb mount /dev/sda /cd mount /dev/sdb1 /usb # XXX We should add cp-r command XXX debug sh "cp -rP /sysroot/cd/* /sysroot/usb" #glob cp-a /cd/* /usb umount /cd umount /usb rmmountpoint /cd rmmountpoint /usb echo "Making the USB key bootable using SYSLINUX ..." syslinux /dev/sdb1 mount /dev/sdb1 / upload /usr/share/syslinux/chain.c32 /chain.c32 write /syslinux.cfg "DEFAULT windows\n\nLABEL windows\nCOM32 chain.c32\nAPPEND fs ntldr=/bootmgr\n" umount /dev/sdb1 upload /usr/share/syslinux/mbr.bin /dev/sdb echo "Finished." EOF
Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Tue, 07/05/2013 - 13:30

libguestfs has high quality Python bindings. Using rpyc you can make a remote libguestfs server with almost no effort at all.

Firstly start an rpyc server:

$ /usr/lib/python2.7/site-packages/rpyc/servers/ [SLAVE INFO 13:21:17 tid=140019939981120] server started on [SLAVE INFO 13:21:17 tid=140019784894208] started background auto-register thread (interval = 60) [REGCLNT INFO 13:21:17] registering on [REGCLNT WARNING 13:21:19] no registry acknowledged

Now, possibly from the same machine or some other machine, you can connect to this server and use Python objects remotely as if they were local:

$ python Python 2.7.3 (default, Aug 9 2012, 17:23:57) [GCC 4.7.1 20120720 (Red Hat 4.7.1-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import rpyc >>> c = rpyc.classic.connect('localhost')

You can now create a libguestfs handle, following the example here.

>>> g = c.modules.guestfs.GuestFS() >>> g.version() {'release': 36L, 'major': 1L, 'minor': 21L, 'extra': 'fedora=20,release=1.fc20,libvirt'} >>> g.add_drive('/dev/fedora/f18x64', readonly=True) >>> g.launch() >>> roots = g.inspect_os() >>> g.inspect_get_product_name(roots[0]) 'Fedora release 18 (Spherical Cow)' >>> g.inspect_get_mountpoints(roots[0]) [('/', '/dev/mapper/fedora-root'), ('/boot', '/dev/sda1')]

As you can see, the g object is transparently remoted without you needing to do anything.

Categories: LUG Community Blogs

Dean Wilson: Deprecation Warnings From Puppet Resources

Planet GLLUG - Sat, 27/04/2013 - 12:53
Over time parts of your puppet manifests will become unneeded. You might move a cronjob or a users in to a package or no longer need a service to be enabled after a given release. I've recently had this use case and had two options - either rely on comments in the Puppet code and write an out of band tool to scan the code base and present a report or add them to the puppet resources themselves. I chose the latter.

Below you'll find a simple metaparameter (a parameter that works with any resource type) that adds this feature to puppet. As this is an early prototype I've hacked it directly in to my local puppet fork. Below you'll see a sample resource that declares a deprecation date and message, the code that implements it and a simple command line test you can run to confirm it works.

# sample puppet resource using :deprecation file { '/ec/cron.d/remove_foos': ensure => 'file', source => 'puppet:///modules/foo/foo.cron', deprecation => '20130425:Release 6 removes the need for the foo cronjob', } $ sudo vi puppet-3.1.1/lib/puppet/type.rb newmetaparam(:deprecation) do desc " Add a deprecation warning to resources. file { '/etc/foo': content => 'Bar', deprecation => '20130425:We no longer need the foo' } The deprecation comes in two parts, separated by a : The date is in format YYYYMMDD and the message is a free form string. " munge do |deprecation| date, message = deprecation.split(':') # YYY MM DD - one true timestamp now ='%Y%m%d') if (now >= date) rsrc = "#{@resource.type.capitalize}[#{}]" Puppet.warning "#{rsrc} expired on #{date}: #{message}" end end end # command line test $ puppet apply -e 'file { "/tmp/dep": content => "foo\n", deprecation => "20120425:We can remove this file after release 4" }' Warning: File[/tmp/dep] expired on 20120425: We can remove this file after release 4 Notice: Finished catalog run in 0.06 seconds

Using the metaparameter is easy enough, just specify 'deprecation' as a property on a resource and provide a string that contains the date to start flagging the deprecation on (in YYYYMMDD format) and the message puppet should show. I don't currently fail the run on an expired resource but this is an option.

The are some other aspects of this to consider - Richard Clamp raised the idea of having a native type that could indicate this for an entire class (I'd rather use a function, but only because they are much easier to write) and Trevor Vaughan suggested a Puppet face that could present a report of the expired, and soon to be expired, code.

I don't know how widely useful this is but it made a nice change to write some puppet code. The small size of the example will hopefully show how easy it is to extend nearly every part of puppet - including more 'complicated' aspects like metaparameters. Although not the relationship ones, those are horrible ;) I've submitted the idea to the upstream development list so we'll see what happens.

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Fri, 26/04/2013 - 19:17

Brian is Red Hat’s CTO, and hence my boss’s boss’s boss (or something like that). This is a pretty good (and honest) talk about Red Hat’s plans for OpenStack.

Edit: By the way, the thumbnail (the one I see at any rate) is not Brian.

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Tue, 23/04/2013 - 20:46

There are some items which are just not possible to order online any more, assuming you want something of minimal quality. These include:

  1. chargers for mobile phones — all now fake as far as I can tell
  2. batteries for older mobile phones — second hand, dead, fake, or simply the wrong battery
  3. consumer cables, like USB to micro USB
  4. convection heaters — didn’t expect this, but I can’t find a reliable one (for the new office) online

That’s not counting the stuff where it’s still just about possible to get non-fake stuff, but it’s a crapshoot, eg. computer memory, hard disks, batteries, flash memory.

Categories: LUG Community Blogs

Richard WM Jones: 20130423_155905.jpg

Planet GLLUG - Tue, 23/04/2013 - 16:00

Mike the electrician is here wiring up the garden office to the mains (I’m doing the networking). The armoured cable (black) and the two cat7 network cables (yellow) will be buried in this trench:

They go in through the wall. This will later be hidden with metal conduit. That wall is 12 inches thick, and close behind it is the gas main so he had to be very careful where he was drilling.

Finally the armoured cable finishes next to the consumer unit (to the right — not shown). Look at the lovely 1950s-era mains supply:

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Mon, 22/04/2013 - 10:09

Since libguestfs 1.20 it has been possible to use rsync to upload or download files to a disk image incrementally. This is one way to do backups, but note that it won’t work on live guests unless you take a snapshot.

rsync involves using a network connection into or out of the appliance, and is therefore a lot more involved to set up. The script below shows one way to do this, by running an rsync daemon on the host, and letting the libguestfs appliance connect to it.

The script runs rsync inside the appliance, copying /home from the attached disk image out to /tmp/backup on the host. If the operation is repeated, then only incrementally changed files will be copied out. (To incrementally delete files on the target, add the deletedest:true flag).

Note you will need to open port 2999 on your host’s firewall for this to work.

#!/bin/bash - set -x # The target directory. mkdir -p /tmp/backup # Create the daemon. rm -f /tmp/ cat <<EOF > /tmp/rsyncd.conf port = 2999 pid file = /tmp/ [backup] path = /tmp/backup use chroot = false read only = false EOF rsync --daemon --config=/tmp/rsyncd.conf # Run guestfish and attach to the guest. guestfish --ro --network -a /dev/fedora/f19rawhidex32 -i <<EOF trace on rsync-out /home rsync://rjones@ archive:true EOF # Kill the rsync daemon. kill `cat /tmp/`
Categories: LUG Community Blogs

Richard WM Jones: 20130420_175710.jpg

Planet GLLUG - Sat, 20/04/2013 - 17:59

Inside there’s a ring for sockets and a lighting circuit. The windows and doors are pre-made double-glazed units.

Categories: LUG Community Blogs

Richard WM Jones: image

Planet GLLUG - Fri, 19/04/2013 - 15:15

Categories: LUG Community Blogs

Richard WM Jones: 20130419_094942.jpg

Planet GLLUG - Thu, 18/04/2013 - 18:39

Update: Further progress this morning …

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Thu, 18/04/2013 - 17:26

Dominic Cleal’s short introduction to the Augeas configuration API.

We use Augeas a lot in libguestfs and virt-v2v, and it’s been very effective for us.

I asked Dominic how he made this video.

He uses gtk-recordmydesktop, max 100/100 audio/video quality, 30fps, 2 channel audio at 48kHz.

Sound and video are recorded at the same time, with a Sennheiser headset.

Editing is done in kdenlive.

Categories: LUG Community Blogs

Richard WM Jones: 20130418_145150.jpg

Planet GLLUG - Thu, 18/04/2013 - 14:55

The cabin has a double wall which will contain insulation. Shown here is the ring main for the four sets of sockets around the room.

Categories: LUG Community Blogs

Richard WM Jones: 20130417_170829.jpg

Planet GLLUG - Thu, 18/04/2013 - 13:17

This trench will take 1 armoured mains cable and 2 cat 7 network cables.

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Wed, 17/04/2013 - 14:31

Also (not shown) they dug a trench down the side of the garden for electrics and network cables. We’re going to run some hefty armoured cable, plus two or three cat 7 network cables, covered with bricks and warning tape, at a depth of 18″ (mandated by building regulations).

Categories: LUG Community Blogs

Richard WM Jones: office-3

Planet GLLUG - Tue, 16/04/2013 - 16:11

As some readers might know, I work from home, and have worked from home for about 12 years. However I’ve never had a proper dedicated office, just a corner of a second bedroom or a bit of space on a sofa. That’s about to change.

Tomorrow, work starts on constructing a real office in my garden.

Work already started (for me) about a month ago, when I took down a shed and jack-hammered a patio to bits:

Shifting the whole lot into two skips:

Resulting in this empty, mostly flat space (about 20′ wide and 16′ deep, damn you council for chopping that tree down at the back …):

To be continued …

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Tue, 16/04/2013 - 15:23

New in libguestfs ≥ 1.21.30 is the ability to use guestfish and some of the virt tools with remote disks.

Currently you can use remote disks over NBD, GlusterFS, Ceph, Sheepdog and (recently upstream) SSH.

For this example I’ll use SSH because it needs no setup, although this requires absolutely the latest qemu and libguestfs (both from git).

Since we don’t have libvirt support for ssh yet, so this only works with the direct backend:

$ export LIBGUESTFS_BACKEND=direct

I can use a ssh:// URI to add disks with guestfish, guestmount and most of the virt tools. For example:

$ virt-rescue -a ssh://localhost/tmp/f17x64.img [... lots of boot messages ...] Welcome to virt-rescue, the libguestfs rescue shell. Note: The contents of / are the rescue appliance. You have to mount the guest's partitions under /sysroot before you can examine them. ><rescue> mount /dev/vg_f17x64/lv_root /sysroot ><rescue> cat /sysroot/etc/redhat-release Fedora release 17 (Beefy Miracle)

Apart from being a tiny bit slower, it just works as if the disk was local:

$ virt-df -a ssh://localhost/tmp/f17x64.img Filesystem 1K-blocks Used Available Use% f17x64.img:/dev/sda1 487652 63738 398314 14% f17x64.img:/dev/vg_f17x64/lv_root 28316680 4285576 22586036 16% $ guestmount -a ssh://localhost/tmp/f17x64.img -i /tmp/mnt $ ls /tmp/mnt bin dev home lib64 media opt root sbin sys usr boot etc lib lost+found mnt proc run srv tmp var $ cat /tmp/mnt/etc/redhat-release Fedora release 17 (Beefy Miracle) $ guestunmount /tmp/mnt
Categories: LUG Community Blogs

Rev. Simon Rumble: Snorkelling photos

Planet GLLUG - Mon, 15/04/2013 - 23:41

I've been getting into snorkelling a bit recently. I've always enjoyed it but recently I bought some good quality gear, replacing the toy shop crap I've been using. It's another world with good equipment! It's not easy to get time, but so far I've snorkelled Jervis Bay, Bushrangers Bay, Clovelly and The Haven in Terrigal.

My son has been asking what it's like, so I bought the Kogan waterproof camera case for $19. Took it out last weekend for a spin at The Haven but the visibility was terrible. The camera case works a treat though, and I'm looking forward to using it some more. Need to work out a strap to attach it to my arm or something though.

See the full gallery on Posterous

Permalink | Leave a comment  »

Categories: LUG Community Blogs
Syndicate content