News aggregator

Karanbir Singh: Microphones and Online Chats

Planet GLLUG - Thu, 14/03/2013 - 00:04

Hi,

Over the last couple of weeks, I've spent quite a lot of time online talking with people about various things. Either using software voice over IP or google-hangout or Skype or whatever. I've noticed recently there seems to be a growing trend for people to have more and more expensive and professional Microphones for these conversations.

Awesome.

Except, get a grip. There is a limit to what and how these Mic's are meant to be used. The point at which I can hear a cat crossing the road two blocks away from your house - too much. Either get a proper sound / recording room setup, or just use that mic on the webcam or laptop. Really, its good enough for these chats.

A £250 Sennheiser Mic for this sort of a thing, not needed. Talking about the mic setup and getting the wind-blow from when you mention P, not needed. You neither sound better not look prettier. Stick to that 1bit laptop mic. We'll all be happier for it.

- KB

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Wed, 13/03/2013 - 09:00

Just over 3 years ago I wrote that guestfish supported 267 commands and I listed them out.

Today, in libguestfs 1.21.19, guestfish has 502 commands, listed below.

Command Description help display a list of commands or help on a command quit quit guestfish acl-delete-def-file delete the default POSIX ACL of a directory acl-get-file get the POSIX ACL attached to a file acl-set-file set the POSIX ACL attached to a file add-cdrom add a CD-ROM disc image to examine add-domain add the disc(s) from a named libvirt domain add-drive add an image to examine or modify add-drive-ro add a drive in snapshot mode (read-only) add-drive-ro-with-if add a drive read-only specifying the QEMU block emulation to use add-drive-with-if add a drive specifying the QEMU block emulation to use alloc allocate and add a disc file aug-clear clear Augeas path aug-close close the current Augeas handle aug-defnode define an Augeas node aug-defvar define an Augeas variable aug-get look up the value of an Augeas path aug-init create a new Augeas handle aug-insert insert a sibling Augeas node aug-load load files into the tree aug-ls list Augeas nodes under augpath aug-match return Augeas nodes which match augpath aug-mv move Augeas node aug-rm remove an Augeas path aug-save write all pending Augeas changes to disk aug-set set Augeas path to value available test availability of some parts of the API available-all-groups return a list of all optional groups base64-in upload base64-encoded data to file base64-out download file and encode as base64 blkid print block device attributes blockdev-flushbufs flush device buffers blockdev-getbsz get blocksize of block device blockdev-getro is block device set to read-only blockdev-getsize64 get total size of device in bytes blockdev-getss get sectorsize of block device blockdev-getsz get total size of device in 512-byte sectors blockdev-rereadpt reread partition table blockdev-setbsz set blocksize of block device blockdev-setro set block device to read-only blockdev-setrw set block device to read-write btrfs-device-add add devices to a btrfs filesystem btrfs-device-delete remove devices from a btrfs filesystem btrfs-filesystem-balance balance a btrfs filesystem btrfs-filesystem-resize resize a btrfs filesystem btrfs-filesystem-sync sync a btrfs filesystem btrfs-fsck check a btrfs filesystem btrfs-set-seeding enable or disable the seeding feature of device btrfs-subvolume-create create a btrfs snapshot btrfs-subvolume-delete delete a btrfs snapshot btrfs-subvolume-list list btrfs snapshots and subvolumes btrfs-subvolume-set-default set default btrfs subvolume btrfs-subvolume-snapshot create a writable btrfs snapshot canonical-device-name return canonical device name cap-get-file get the Linux capabilities attached to a file cap-set-file set the Linux capabilities attached to a file case-sensitive-path return true path on case-insensitive filesystem cat list the contents of a file checksum compute MD5, SHAx or CRC checksum of file checksum-device compute MD5, SHAx or CRC checksum of the contents of a device checksums-out compute MD5, SHAx or CRC checksum of files in a directory chmod change file mode chown change file owner and group command run a command from the guest filesystem command-lines run a command, returning lines compress-device-out output compressed device compress-out output compressed file config add qemu parameters copy-device-to-device copy from source device to destination device copy-device-to-file copy from source device to destination file copy-file-to-device copy from source file to destination device copy-file-to-file copy from source file to destination file copy-in copy local files or directories into an image copy-out copy remote files or directories out of an image copy-size copy size bytes from source to destination using dd cp copy a file cp-a copy a file or directory recursively dd copy from source to destination using dd debug debugging and internals debug-drives debug the drives (internal use only) debug-upload upload a file to the appliance (internal use only) delete-event delete a previously registered event handler device-index convert device to index df report file system disk space usage df-h report file system disk space usage (human readable) disk-format detect the disk format of a disk image disk-has-backing-file return whether disk has a backing file disk-virtual-size return virtual size of a disk display display an image dmesg return kernel messages download download a file to the local machine download-offset download a file to the local machine with offset and size drop-caches drop kernel page cache, dentries and inodes du estimate file space usage e2fsck check an ext2/ext3 filesystem e2fsck-f check an ext2/ext3 filesystem echo display a line of text echo-daemon echo arguments back to the client edit edit a file egrep return lines matching a pattern egrepi return lines matching a pattern equal test if two files have equal contents event register a handler for an event or events exists test if file or directory exists fallocate preallocate a file in the guest filesystem fallocate64 preallocate a file in the guest filesystem fgrep return lines matching a pattern fgrepi return lines matching a pattern file determine file type file-architecture detect the architecture of a binary file filesize return the size of the file in bytes filesystem-available check if filesystem is available fill fill a file with octets fill-dir fill a directory with empty files fill-pattern fill a file with a repeating pattern of bytes find find all files and directories find0 find all files and directories, returning NUL-separated list findfs-label find a filesystem by label findfs-uuid find a filesystem by UUID fsck run the filesystem checker fstrim trim free space in a filesystem get-append get the additional kernel options get-attach-method get the attach method get-autosync get autosync mode get-cachedir get the appliance cache directory get-direct get direct appliance mode flag get-e2attrs get ext2 file attributes of a file get-e2generation get ext2 file generation of a file get-e2label get the ext2/3/4 filesystem label get-e2uuid get the ext2/3/4 filesystem UUID get-libvirt-requested-credential-challenge challenge of i'th requested credential get-libvirt-requested-credential-defresult default result of i'th requested credential get-libvirt-requested-credential-prompt prompt of i'th requested credential get-libvirt-requested-credentials get list of credentials requested by libvirt get-memsize get memory allocated to the qemu subprocess get-network get enable network flag get-path get the search path get-pgroup get process group flag get-pid get PID of qemu subprocess get-qemu get the qemu binary get-recovery-proc get recovery process enabled flag get-selinux get SELinux enabled flag get-smp get number of virtual CPUs in appliance get-tmpdir get the temporary directory get-trace get command trace enabled flag get-umask get the current umask get-verbose get verbose mode getcon get SELinux security context getxattr get a single extended attribute getxattrs list extended attributes of a file or directory glob expand wildcards in command glob-expand expand a wildcard path grep return lines matching a pattern grepi return lines matching a pattern grub-install install GRUB 1 head return first 10 lines of a file head-n return first N lines of a file hexdump dump a file in hexadecimal hexedit edit with a hex editor hivex-close close the current hivex handle hivex-commit commit (write) changes back to the hive hivex-node-add-child add a child node hivex-node-children return list of nodes which are subkeys of node hivex-node-delete-child delete a node (recursively) hivex-node-get-child return the named child of node hivex-node-get-value return the named value hivex-node-name return the name of the node hivex-node-parent return the parent of node hivex-node-set-value set or replace a single value in a node hivex-node-values return list of values attached to node hivex-open open a Windows Registry hive file hivex-root return the root node of the hive hivex-value-key return the key field from the (key, datatype, data) tuple hivex-value-type return the data type from the (key, datatype, data) tuple hivex-value-utf8 return the data field from the (key, datatype, data) tuple hivex-value-value return the data field from the (key, datatype, data) tuple initrd-cat list the contents of a single file in an initrd initrd-list list files in an initrd inotify-add-watch add an inotify watch inotify-close close the inotify handle inotify-files return list of watched files that had events inotify-init create an inotify handle inotify-read return list of inotify events inotify-rm-watch remove an inotify watch inspect-get-arch get architecture of inspected operating system inspect-get-distro get distro of inspected operating system inspect-get-drive-mappings get drive letter mappings inspect-get-filesystems get filesystems associated with inspected operating system inspect-get-format get format of inspected operating system inspect-get-hostname get hostname of the operating system inspect-get-icon get the icon corresponding to this operating system inspect-get-major-version get major version of inspected operating system inspect-get-minor-version get minor version of inspected operating system inspect-get-mountpoints get mountpoints of inspected operating system inspect-get-package-format get package format used by the operating system inspect-get-package-management get package management tool used by the operating system inspect-get-product-name get product name of inspected operating system inspect-get-product-variant get product variant of inspected operating system inspect-get-roots return list of operating systems found by last inspection inspect-get-type get type of inspected operating system inspect-get-windows-current-control-set get Windows CurrentControlSet of inspected operating system inspect-get-windows-systemroot get Windows systemroot of inspected operating system inspect-is-live get live flag for install disk inspect-is-multipart get multipart flag for install disk inspect-is-netinst get netinst (network installer) flag for install disk inspect-list-applications get list of applications installed in the operating system inspect-list-applications2 get list of applications installed in the operating system inspect-os inspect disk and return list of operating systems found is-blockdev test if block device is-chardev test if character device is-config is in configuration state is-dir test if a directory is-fifo test if FIFO (named pipe) is-file test if a regular file is-lv test if device is a logical volume is-socket test if socket is-symlink test if symbolic link is-whole-device test if a device is a whole device is-zero test if a file contains all zero bytes is-zero-device test if a device contains all zero bytes isoinfo get ISO information from primary volume descriptor of ISO file isoinfo-device get ISO information from primary volume descriptor of device kill-subprocess kill the qemu subprocess launch launch the qemu subprocess lcd change working directory lchown change file owner and group ldmtool-create-all scan and create Windows dynamic disk volumes ldmtool-diskgroup-disks return the disks in a Windows dynamic disk group ldmtool-diskgroup-name return the name of a Windows dynamic disk group ldmtool-diskgroup-volumes return the volumes in a Windows dynamic disk group ldmtool-remove-all remove all Windows dynamic disk volumes ldmtool-scan scan for Windows dynamic disks ldmtool-scan-devices scan for Windows dynamic disks ldmtool-volume-hint return the hint field of a Windows dynamic disk volume ldmtool-volume-partitions return the partitions in a Windows dynamic disk volume ldmtool-volume-type return the type of a Windows dynamic disk volume lgetxattr get a single extended attribute lgetxattrs list extended attributes of a file or directory list-9p list 9p filesystems list-devices list the block devices list-disk-labels mapping of disk labels to devices list-dm-devices list device mapper devices list-events list event handlers list-filesystems list filesystems list-ldm-partitions list all Windows dynamic disk partitions list-ldm-volumes list all Windows dynamic disk volumes list-md-devices list Linux md (RAID) devices list-partitions list the partitions ll list the files in a directory (long format) llz list the files in a directory (long format with SELinux contexts) ln create a hard link ln-f create a hard link ln-s create a symbolic link ln-sf create a symbolic link lremovexattr remove extended attribute of a file or directory ls list the files in a directory ls0 get list of files in a directory lsetxattr set extended attribute of a file or directory lstat get file information for a symbolic link lstatlist lstat on multiple files luks-add-key add a key on a LUKS encrypted device luks-close close a LUKS device luks-format format a block device as a LUKS encrypted device luks-format-cipher format a block device as a LUKS encrypted device luks-kill-slot remove a key from a LUKS encrypted device luks-open open a LUKS-encrypted block device luks-open-ro open a LUKS-encrypted block device read-only lvcreate create an LVM logical volume lvcreate-free create an LVM logical volume in % remaining free space lvm-canonical-lv-name get canonical name of an LV lvm-clear-filter clear LVM device filter lvm-remove-all remove all LVM LVs, VGs and PVs lvm-set-filter set LVM device filter lvremove remove an LVM logical volume lvrename rename an LVM logical volume lvresize resize an LVM logical volume lvresize-free expand an LV to fill free space lvs list the LVM logical volumes (LVs) lvs-full list the LVM logical volumes (LVs) lvuuid get the UUID of a logical volume lxattrlist lgetxattr on multiple files man open the manual max-disks maximum number of disks that may be added md-create create a Linux md (RAID) device md-detail obtain metadata for an MD device md-stat get underlying devices from an MD device md-stop stop a Linux md (RAID) device mkdir create a directory mkdir-mode create a directory with a particular mode mkdir-p create a directory and parents mkdtemp create a temporary directory mke2fs create an ext2/ext3/ext4 filesystem on device mke2fs-J make ext2/3/4 filesystem with external journal mke2fs-JL make ext2/3/4 filesystem with external journal mke2fs-JU make ext2/3/4 filesystem with external journal mke2journal make ext2/3/4 external journal mke2journal-L make ext2/3/4 external journal with label mke2journal-U make ext2/3/4 external journal with UUID mkfifo make FIFO (named pipe) mkfs make a filesystem mkfs-b make a filesystem with block size mkfs-btrfs create a btrfs filesystem mklost-and-found make lost+found directory on an ext2/3/4 filesystem mkmountpoint create a mountpoint mknod make block, character or FIFO devices mknod-b make block device node mknod-c make char device node mkswap create a swap partition mkswap-L create a swap partition with a label mkswap-U create a swap partition with an explicit UUID mkswap-file create a swap file mktemp create a temporary file modprobe load a kernel module more view a file mount mount a guest disk at a position in the filesystem mount-9p mount 9p filesystem mount-local mount on the local filesystem mount-local-run run main loop of mount on the local filesystem mount-loop mount a file using the loop device mount-options mount a guest disk with mount options mount-ro mount a guest disk, read-only mount-vfs mount a guest disk with mount options and vfstype mountpoints show mountpoints mounts show mounted filesystems mv move a file nr-devices return number of whole block devices (disks) added ntfs-3g-probe probe NTFS volume ntfsclone-in restore NTFS from backup file ntfsclone-out save NTFS to backup file ntfsfix fix common errors and force Windows to check NTFS ntfsresize resize an NTFS filesystem ntfsresize-size resize an NTFS filesystem (with size) parse-environment parse the environment and set handle flags accordingly parse-environment-list parse the environment and set handle flags accordingly part-add add a partition to the device part-del delete a partition part-disk partition whole disk with a single primary partition part-get-bootable return true if a partition is bootable part-get-gpt-type get the type GUID of a GPT partition part-get-mbr-id get the MBR type byte (ID byte) from a partition part-get-parttype get the partition table type part-init create an empty partition table part-list list partitions on a device part-set-bootable make a partition bootable part-set-gpt-type set the type GUID of a GPT partition part-set-mbr-id set the MBR type byte (ID byte) of a partition part-set-name set partition name part-to-dev convert partition name to device name part-to-partnum convert partition name to partition number ping-daemon ping the guest daemon pread read part of a file pread-device read part of a device pvchange-uuid generate a new random UUID for a physical volume pvchange-uuid-all generate new random UUIDs for all physical volumes pvcreate create an LVM physical volume pvremove remove an LVM physical volume pvresize resize an LVM physical volume pvresize-size resize an LVM physical volume (with size) pvs list the LVM physical volumes (PVs) pvs-full list the LVM physical volumes (PVs) pvuuid get the UUID of a physical volume pwrite write to part of a file pwrite-device write to part of a device read-file read a file read-lines read file as lines readdir read directories entries readlink read the target of a symbolic link readlinklist readlink on multiple files realpath canonicalized absolute pathname remove-drive remove a disk image removexattr remove extended attribute of a file or directory rename rename a file on the same filesystem reopen close and reopen libguestfs handle resize2fs resize an ext2, ext3 or ext4 filesystem resize2fs-M resize an ext2, ext3 or ext4 filesystem to the minimum size resize2fs-size resize an ext2, ext3 or ext4 filesystem (with size) rm remove a file rm-f remove a file ignoring errors rm-rf remove a file or directory recursively rmdir remove a directory rmmountpoint remove a mountpoint rsync synchronize the contents of two directories rsync-in synchronize host or remote filesystem with filesystem rsync-out synchronize filesystem with host or remote filesystem scrub-device scrub (securely wipe) a device scrub-file scrub (securely wipe) a file scrub-freespace scrub (securely wipe) free space set-append add options to kernel command line set-attach-method set the attach method set-autosync set autosync mode set-cachedir set the appliance cache directory set-direct enable or disable direct appliance mode set-e2attrs set ext2 file attributes of a file set-e2generation set ext2 file generation of a file set-e2label set the ext2/3/4 filesystem label set-e2uuid set the ext2/3/4 filesystem UUID set-label set filesystem label set-libvirt-requested-credential pass requested credential back to libvirt set-libvirt-supported-credentials set libvirt credentials supported by calling program set-memsize set memory allocated to the qemu subprocess set-network set enable network flag set-path set the search path set-pgroup set process group flag set-qemu set the qemu binary set-recovery-proc enable or disable the recovery process set-selinux set SELinux enabled or disabled at appliance boot set-smp set number of virtual CPUs in appliance set-tmpdir set the temporary directory set-trace enable or disable command traces set-verbose set verbose mode setcon set SELinux security context setenv set an environment variable setxattr set extended attribute of a file or directory sfdisk create partitions on a block device sfdiskM create partitions on a block device sfdisk-N modify a single partition on a block device sfdisk-disk-geometry display the disk geometry from the partition table sfdisk-kernel-geometry display the kernel geometry sfdisk-l display the partition table sh run a command via the shell sh-lines run a command via the shell returning lines shutdown shutdown the qemu subprocess sleep sleep for some seconds sparse create a sparse disk image and add stat get file information statvfs get file system statistics strings print the printable strings in a file strings-e print the printable strings in a file supported list supported groups of commands swapoff-device disable swap on device swapoff-file disable swap on file swapoff-label disable swap on labeled swap partition swapoff-uuid disable swap on swap partition by UUID swapon-device enable swap on device swapon-file enable swap on file swapon-label enable swap on labeled swap partition swapon-uuid enable swap on swap partition by UUID sync sync disks, writes are flushed through to the disk image tail return last 10 lines of a file tail-n return last N lines of a file tar-in unpack tarfile to directory tar-out pack directory into tarfile tgz-in unpack compressed tarball to directory tgz-out pack directory into compressed tarball time print elapsed time taken to run a command touch update file timestamps or create a new file truncate truncate a file to zero size truncate-size truncate a file to a particular size tune2fs adjust ext2/ext3/ext4 filesystem parameters tune2fs-l get ext2/ext3/ext4 superblock details txz-in unpack compressed tarball to directory txz-out pack directory into compressed tarball umask set file mode creation mask (umask) umount unmount a filesystem umount-all unmount all filesystems umount-local unmount a locally mounted filesystem unsetenv unset an environment variable upload upload a file from the local machine upload-offset upload a file from the local machine with offset user-cancel cancel the current upload or download operation utimens set timestamp of a file with nanosecond precision utsname appliance kernel version version get the library version number vfs-label get the filesystem label vfs-type get the Linux VFS type corresponding to a mounted device vfs-uuid get the filesystem UUID vg-activate activate or deactivate some volume groups vg-activate-all activate or deactivate all volume groups vgchange-uuid generate a new random UUID for a volume group vgchange-uuid-all generate new random UUIDs for all volume groups vgcreate create an LVM volume group vglvuuids get the LV UUIDs of all LVs in the volume group vgmeta get volume group metadata vgpvuuids get the PV UUIDs containing the volume group vgremove remove an LVM volume group vgrename rename an LVM volume group vgs list the LVM volume groups (VGs) vgs-full list the LVM volume groups (VGs) vgscan rescan for LVM physical volumes, volume groups and logical volumes vguuid get the UUID of a volume group wc-c count characters in a file wc-l count lines in a file wc-w count words in a file wipefs wipe a filesystem signature from a device write create a new file write-append append content to end of file write-file create a file xfs-admin change parameters of an XFS filesystem xfs-growfs expand an existing XFS filesystem xfs-info get geometry of XFS filesystem xfs-repair repair an XFS filesystem zegrep return lines matching a pattern zegrepi return lines matching a pattern zero write zeroes to the device zero-device write zeroes to an entire device zero-free-space zero free space in a filesystem zerofree zero unused inodes and disk blocks on ext2/3 filesystem zfgrep return lines matching a pattern zfgrepi return lines matching a pattern zfile determine file type inside a compressed file zgrep return lines matching a pattern zgrepi return lines matching a pattern Use -h / help to show detailed help for a command.
Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Tue, 12/03/2013 - 23:01

Thanks to infernix who contributed this tip on how to use libguestfs to access Ceph (and in theory, sheepdog, gluster, iscsi and more) devices.

If you apply this small patch to libguestfs you can use these distributed filesystems straight away by doing:

$ guestfish ><fs> set-attach-method appliance ><fs> add-drive /dev/null ><fs> config -set drive.hd0.file=rbd:pool/volume ><fs> run

… followed by usual guestfish commands.

This is a temporary hack, until we properly model Ceph (etc) through the libguestfs stable API. Nevertheless it works as follows:

  1. The add-drive /dev/null adds a drive, known to libguestfs.
  2. Implicitly this means that libguestfs adds a -drive option when it runs qemu.
  3. The custom qemu -set drive.hd0.file=... parameter modifies the preceding -drive option added by libguestfs so that the file is changed from /dev/null to whatever you want. In this case, to a Ceph rbd:... path.

Categories: LUG Community Blogs

Karanbir Singh: nazar.karan.org in maint mode

Planet GLLUG - Tue, 12/03/2013 - 01:15

Hi,

https://nazar.karan.org/ services are going to be partially down as I migrate services over to a faster, more memory, lesser power consuming, many more cores machine. Everything should be back to production by midday Mar 12th, 2013. Services impacted include:

  • git repos
  • Reimzul's irc interface
  • Alt.Bsys triggers

There is a backup instance running, so if anyone needs to get to some specific data in a rush, ping me on irc and we can get access setup.

- KB

Categories: LUG Community Blogs

davblog - Dave Cross: Doctor Who News

Planet GLLUG - Sun, 10/03/2013 - 16:22

I’m getting bored of the number of media outlets who are taking the slightest of comments that someone makes about the upcoming Doctor Who anniversary special and spinning it into a story packed full of completely unsubstantiated nonsense. Headlines like “No Doctors To Return For 50th Special” which, when you read them turn out to be based on the fact that Colin Baker hasn’t had a phone call from Steven Moffat.

Obviously it’s good for the show that it gets all of this publicity and I don’t, for one second, expect the production team to do anything to put a stop to it. They’ll tell us what they want us to know when they want us to know it. Not a moment sooner.

But in the meantime, anyone who has ever appeared in Doctor Who has to watch what they say for fear of it being overhead by a tabloid journalist and being used to reinforce what ever story the journalist wants to write.

In an attempt to counter this, I’ve set up whonews.tv. The plan is that I’ll read these stories, extract the actual facts that they are based on and explain what we can actually believe based on those facts. Forensic analysis of entertainment news, I suppose.

I’ve also got a page where I list the best current information we have about what is actually happening for the show’s 50th anniversary. I’ll try to keep that up to date as more details emerge over the coming months.

Oh, and there’s at Twitter account too – WhoNews50. You might want to follow that.

Let me know if you find it useful.

Related Posts:
Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Tue, 05/03/2013 - 12:55

Matthew Astley just emailed me to say he has forked TechTalk and added support for markdown (the wiki-ish markup language).


Categories: LUG Community Blogs

Martin A. Brooks: An open letter to National Car Parks

Planet GLLUG - Mon, 04/03/2013 - 17:31

Dear National Car Parks,

As it seems the existence of your call centre is to goad people into a state of rage and you insist that everything has to be put in writing, I shall put my complaint to you in writing via my blog, before sending it to you in writing.

I am a season ticket holder for the station car park at Harold Wood.  My car is parked there on a daily basis.  On return to my vehicle on February 28th, I was surprised to find I had been ticketed.  Here’s the ticket:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Note the contravention: “Parked on double yellow lines”.  This was a surprise to me because there were and are no double yellow lines present.  For the non-drivers reading this, and confused NCP parking attendants, double yellow lines tend to look like this:

 

 

 

 

 

 

 

 

 

This is quite clear, right?  Two parallel yellow lines at specific distances from the side of the road with a crossing bar at each end.  Now, here’s a picture of where I was parked:

 

See the bay to the left of the red car? That’s where my car was, wholly within the confines of the white border.  You might want to check out the full size version of the photo but, looking at the example double yellow lines above, can you see where the double yellow lines I was ticketed for parking on are? I’ll sit here quietly while you check.

Done? Good.  You didn’t spot the double yellow lines, did you? Correct, that’s because there aren’t any to spot. I can read your mind.  “Ahhh, Martin, you silly boy.  You’ve parked on a restricted area, there are yellow hatchings. Tsk.”

That’s almost true, there are indeed yellow hatchings but that’s irrelevant.  Why so?  Just this:  When NCP took over the car park many years ago, they repainted all of the bays with fresh white markings.  Here’s a closeup of part of that bay:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Note that the white line is painted over the yellow one.  Had NCP not intended for people to use this as a parking bay, why on earth would they specifically paint bay markings there?  The answer is, they wouldn’t.  Those markings used to denoted an area where you weren’t supposed to park because it’s where a fast food place used to have its bins.  Rather than scraping the old lines up, they just painted new ones over the top.

Add to this the fact that I have been parked in that exact same spot dozens of times over the past couple of years, and this is the first time my car has been ticketed, I think what we’re dealing with here is an overzealous or brainless parking attendant.I currently can’t even talk to you about this over the phone because even after 4 days, the ticket hasn’t appeared on your system. I now have to waste some of my life sorting this out.

No love whatsoever, merely tired rage,

Martin A. Brooks

 

Categories: LUG Community Blogs

davblog - Dave Cross: Money From HMRC

Planet GLLUG - Sat, 02/03/2013 - 10:57

I got a letter from HMRC this morning – to my company, not to me personally. It basically said “we’ve been looking at the PAYE you paid in 2010/11 and it looks like you’ve overpaid by [a surprisingly large number of pounds]“.

Now 2010/11 was the year that I was having some difficulties with my accountants. The difficulties eventually got so bad that I switched to my new accountants (who I’m still very happy with). So it doesn’t really surprise me that something went wrong that year, although the amount (it’s about 25% of the PAYE/NI I paid that year) is impressive.

What really surprises me is the tone of this letter. Having told me that I’ve overpaid (and, helpfully, pointed out the exact extra payment that I made) the letter goes on to say:

Before I can agree to either a refund or a credit, please let me the reason the overpayment has arisen

And:

Please complete the enclosed P35D giving a full explanation as to how the overpayment occurred.

Below are some example of reasons I cannot accept to justify an overpayment

  • Duplicate payments with no evidence to explain why
  • Statements such as ‘Payment(s) made in error’ with no further explanation
  • The overpayment is due to monthly payments which do not match our records
  • Any other explanation without evidence to support it

I’m finding it hard to read that in any way other than “we know we’ve got some of your money but you can’t have it back until you’ve explained in detail just how crap your record-keeping is”.

You’ve got my money. You know it’s my money. Either my accountants or I screwed up in some way. There’s no more detailed explanation than that. Just give it back to me, you bastards.

Related Posts:
Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Tue, 26/02/2013 - 19:31

New in libguestfs ≥ 1.21.15, the virt-df and virt-alignment-scan tools now use parallel appliances when scanning your libvirt guests.

The amount of parallelism is selected heuristically when the tool starts up — by dividing the amount of free memory in MB by 500. You can also override this choice by using the new -P option to both tools, but the default should be fine for everyone. -P 1 disables multiple threads.

Users won’t see much difference, although I found that both tools are noticeably faster.

The implementation of threads in these tools is a little bit interesting. Of course there is a pool of worker threads. These take the libvirt guests from a list sorted in alphabetical order and process them.

However each guest takes a variable amount of time to process, and the trick is that the output from each thread mustn’t overlap or be in non-alphabetical order.

The worker threads do two things to ensure this: Firstly output from each guest scan is saved up in an open_memstream buffer. Secondly, domains are retired in order using a pthread condition variable — each worker waits until the previous domain has been retired, before retiring (ie. printing) its own result.

The outcome is that there should be no difference between what the old tools and the rewritten tools print out.


Categories: LUG Community Blogs

Rev. Simon Rumble: Current status

Planet GLLUG - Tue, 26/02/2013 - 03:56

Chilling in Jervis Bay. Weather has turned out much better than the forecasts. Lovely.

See the full gallery on Posterous

Permalink | Leave a comment  »

Categories: LUG Community Blogs

Richard WM Jones: data

Planet GLLUG - Mon, 25/02/2013 - 15:46

[Part 1, part 2, part 3.]

Finally I modified the test to do some representative work: We now load a real Windows XP guest, inspect it (a heavyweight operation), and mount and stat each filesystem. I won’t reproduce the entire test program again because only the test subroutine has changed:

sub test { my $g = Sys::Guestfs->new; $g->add_drive_ro ("/tmp/winxp.img"); $g->launch (); # Inspect the guest (ignore the result). $g->inspect_os (); # Approximate what virt-df does. my %fses = $g->list_filesystems (); foreach (keys %fses) { my $mounted = 0; eval { $g->mount_ro ($_, "/"); $mounted = 1; }; if ($mounted) { $g->statvfs ("/"); $g->umount_all (); } } return $g; }

Even with all that work going on, I was able to inspect more than 1 disk per second on my laptop, and run 60 threads in parallel with good performance and scalability:


Categories: LUG Community Blogs

Richard WM Jones: data

Planet GLLUG - Mon, 25/02/2013 - 15:22

A problem encountered in part 2 was that I couldn’t measure the maximum number of parallel libguestfs appliances that can be run at the same time. There are two reasons for that. The simpler one is that libvirt has a limit of 20 connections, which is easily overcome by setting LIBGUESTFS_ATTACH_METHOD=appliance to eliminate libvirt and run qemu directly. The harder one is that by the time the last appliances in the test are starting to launch, earlier ones have already shut down and their threads have exited.

What is needed is for the test to work in two phases: In the first phase we start up all the threads and launch all the appliances. Only when this is complete do we enter the second phase where we shut down all the appliances.

The easiest way to do this is by modifying the test to use a barrier (or in fact to implement a barrier using the condition primitives). See the modified test script below.

With the modified test script I was able to run ≥ 110 and < 120 parallel appliances in ~ 13 GB of free RAM, or around 120 MB / appliance, still with excellent performance and nearly linear scalability:

#!/usr/bin/perl -w use strict; use threads qw(yield); use threads::shared qw(cond_broadcast cond_wait lock); use Sys::Guestfs; use Time::HiRes qw(time); my $nr_threads_launching :shared; sub test { my $g = Sys::Guestfs->new; $g->add_drive_ro ("/dev/null"); $g->launch (); return $g; } # Get everything into cache. test (); test (); test (); sub thread { my $g = test (); { lock ($nr_threads_launching); $nr_threads_launching--; cond_broadcast ($nr_threads_launching); cond_wait ($nr_threads_launching) until $nr_threads_launching == 0; } $g->close (); } # Test increasing numbers of threads until it fails. for (my $nr_threads = 10; $nr_threads < 1000; $nr_threads += 10) { my $start_t = time (); $nr_threads_launching = $nr_threads; my @threads; foreach (1..$nr_threads) { push @threads, threads->create (\&thread) } foreach (@threads) { $_->join (); if (my $err = $_->error ()) { die "launch failed with $nr_threads threads: $err" } } my $end_t = time (); printf ("%d %.2f\n", $nr_threads, $end_t - $start_t); }
Categories: LUG Community Blogs

Richard WM Jones: data

Planet GLLUG - Mon, 25/02/2013 - 14:28

One problem with the previous test is that I hit a limit of 20 parallel appliances and mistakenly thought that I’d hit a memory limit. In fact libvirt out of the box limits the number of client connections to 20. You can adjust libvirt’s limit by editing /etc/libvirt/libvirtd.conf, but easier for us is to simply eliminate libvirt from the equation by doing:

export LIBGUESTFS_ATTACH_METHOD=appliance

which causes libguestfs to run qemu directly. In my first test I reached 48 parallel launches before I killed the program (because that’s a lot of parallelism and there seemed no end in sight). Scalability of the libguestfs / qemu combination was excellent again:

But there’s more! (In the next part …)


Categories: LUG Community Blogs

Richard WM Jones: data

Planet GLLUG - Mon, 25/02/2013 - 12:54

I wrote the Perl script below to find out how many libguestfs appliances we can start in parallel. The results are surprising (-ly good):

What’s happening here is that we’re booting up a KVM guest with 500 MB of memory, booting the Linux kernel, booting a minimal userspace, then shutting the whole lot down. And then doing that in parallel with 1, 2, .. 20 threads.

[Note: Hardware is my Lenovo x230 laptop with an Intel Core(TM) i7-3520M CPU @ 2.90GHz, 2 cores with 4 threads, 16 GB of RAM with approx. 13 GB free. Software is: Fedora 18 with libguestfs 1.20.2, libvirt 1.0.2 (from Rawhide), qemu 1.4.0 (from Rawhide)]

The test fails at 21 threads because there isn’t enough free memory, so each qemu instance is allocating around 660 MB of RAM. This is wrong: It failed because libvirt out of the box limits the maximum number of clients to 20. See next part in this series.

Up to 4 parallel launches, you can clearly see the effect of better utilization of the parallelism of the CPU — the total elapsed time hardly moves, even though we’re doing up to 4 times more work.

#!/usr/bin/perl -w use strict; use threads; use Sys::Guestfs; use Time::HiRes qw(time); sub test { my $g = Sys::Guestfs->new; $g->add_drive_ro ("/dev/null"); $g->launch (); } # Get everything into cache. test (); test (); test (); # Test increasing numbers of threads until it fails. for my $nr_threads (1..100) { my $start_t = time (); my @threads; foreach (1..$nr_threads) { push @threads, threads->create (\&test) } foreach (@threads) { $_->join (); if (my $err = $_->error ()) { die "launch failed with nr_threads = $nr_threads: $err" } } my $end_t = time (); printf ("%d %.2f\n", $nr_threads, $end_t - $start_t); }
Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Fri, 22/02/2013 - 12:04

This just popped up in my Google alerts: https://github.com/M2IHP13-admin/JonesForth-arm


Categories: LUG Community Blogs

Rev. Simon Rumble: Alternatives to Posterous?

Planet GLLUG - Tue, 19/02/2013 - 23:50

My blog has been on Posterous for some years now and it's been awesome. The best thing about it is that you just email a bunch of stuff, with whatever attachments in whatever format are relevant, and they just work.

Sadly they're shutting down following their talent acquisition by Twitter. That's a real shame. Now I have to find an alternative.

Requirements:
  • Hosted. I'm not going to maintain a server just for blogging thanks.
  • Allows custom JS. I'm always testing out new analytics tools on my own sites.
  • Post through email
Squarespace is lovely but pretty expensive for what I need, unless I can consolidate all three sites into one platform while keeping the domains (waiting on their ticket response).

I had high hopes for Markdown-based blog tools like Jekyll, but I find them a bit clunky. Posterous has got me used to a really easy blogging workflow that works well for me.

Any suggestions? I'm happy to pay.

Permalink | Leave a comment  »

Categories: LUG Community Blogs

Richard WM Jones: 20130218_112227

Planet GLLUG - Mon, 18/02/2013 - 11:30

It works too …

sd 4:0:0:0: [sdc] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) sd 4:0:0:0: [sdc] 4096-byte physical blocks sd 4:0:0:0: [sdc] Write Protect is off sd 4:0:0:0: [sdc] Mode Sense: 00 3a 00 00 sd 4:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA sd 5:0:0:0: [sdd] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) sd 5:0:0:0: [sdd] 4096-byte physical blocks sd 5:0:0:0: [sdd] Write Protect is off sd 5:0:0:0: [sdd] Mode Sense: 00 3a 00 00 sd 5:0:0:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Categories: LUG Community Blogs

Dean Wilson: FOSDEM 2013

Planet GLLUG - Sat, 16/02/2013 - 13:39
Well, that's another FOSDEM over with. In general this year seemed the same as the last couple of years but slightly bigger than usual (although it seems that way every year). The (newish) K building was in full swing with dozens of project stalls and dev rooms. The usual suspects - virtualisation / cloud, configuration management and MySQL rooms had nearly as many people trying to get in to the rooms as they did sitting down.

I think some of the main dev rooms have reached the level of popularity that forces you to either arrive early, get a seat and not move for the rest of the day or accept a very high level of probability that you won't get to see the talks you want. I know a few of us had trouble cherry picking sessions across tracks - which obviously means we have excellent taste in topics. I wonder if having the same talks on both days would make it easier to move around as a visitor - you'd attempt to catch it the first time and if that fails, come back tomorrow. I realise however that this puts even more of a burden on speakers that graciously give their own time in both the preparation and performing of their talks. It does seem that scaling the rooms is the problem of the day once again.

I'd like to say a big thank you to all the organisers, speakers and other attendees for making it another enjoyable couple of days. See you next year.

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Wed, 13/02/2013 - 18:39

To clarify, what is the memory overhead, or how many guests can you cram onto a single host, memory being the typical limiting factor when you virtualize.

This was the question someone asked at work today. I don’t know the answer either, but the small program I wrote (below) aims to find out. If you believe the numbers below from qemu 1.2.2 running on Fedora 18, then the overhead is around 150 MB per qemu process that cannot be shared, plus around 200 MB per host (that is, shared between all qemu processes).

guest size 256 MB: Shared memory backed by a file: 201.41 MB Anonymous memory (eg. malloc, COW, stack), not shared: 404.20 MB Shared writable memory: 0.03 MB guest size 512 MB: Shared memory backed by a file: 201.41 MB Anonymous memory (eg. malloc, COW, stack), not shared: 643.76 MB Shared writable memory: 0.03 MB guest size 1024 MB: Shared memory backed by a file: 201.41 MB Anonymous memory (eg. malloc, COW, stack), not shared: 1172.38 MB Shared writable memory: 0.03 MB guest size 2048 MB: Shared memory backed by a file: 201.41 MB Anonymous memory (eg. malloc, COW, stack), not shared: 2237.16 MB Shared writable memory: 0.03 MB guest size 4096 MB: Shared memory backed by a file: 201.41 MB Anonymous memory (eg. malloc, COW, stack), not shared: 4245.13 MB Shared writable memory: 0.03 MB

The number to pay attention to is “Anonymous memory” since that is what cannot be shared between guests (except if you have KSM and your guests are such that KSM can be effective).

There are some known shortcomings with my testing methodology that I summarise below. You may be able to see others.

  1. We’re testing a libguestfs appliance. A libguestfs appliance does not have the full range of normal qemu devices that a real guest would have, and so the overhead of a real guest is likely to be higher. The main difference is probably lack of a video device (so no video RAM is evident).
  2. This uses virtio-scsi. Real guests use IDE, virtio-blk, etc which may have quite different characteristics.
  3. This guest has one user network device (ie. SLIRP) which could be quite different from a real network device.
  4. During the test, the guest only runs for a few seconds. A normal, long-running guest would experience qemu memory growth or even memory leaks. You could fix this relatively easily by adding some libguestfs busy-work after the launch.
  5. The guest does not do any significant writes, so during the test qemu won’t be storing any cached or in-flight data blocks.
  6. It only accounts for memory used by qemu in userspace, not memory used by the host kernel on behalf of qemu.
  7. The effectiveness or otherwise of KSM is not tested. It’s likely that KSM depends heavily on your workload, so it wouldn’t be fair to publish any KSM figures.
  8. The script uses /proc/PID/maps but it would be better to use smaps so that we can see how much of the file-backed copy-on-write segments have actually been copied. Currently the script overestimates these by assuming that (eg) all the data pages from a library would be dirtied by qemu.

Another interesting question would be whether qemu is getting better or worse over time.

#!/usr/bin/perl -w # Estimate memory usage of qemu-kvm at different guest RAM sizes. # By Richard W.M. Jones <rjones@redhat.com> use strict; use Sys::Guestfs; no warnings "portable"; # 64 bit platform required. # Loop over different guest RAM sizes. my $mbytes; for $mbytes (256, 512, 1024, 2048, 4096) { print "guest size ", $mbytes, " MB:\n"; my $g = Sys::Guestfs->new; # Ensure we're using the direct qemu launch backend, otherwise # libvirt stops us from finding the qemu PID. $g->set_attach_method ("appliance"); # Set guest memory size. $g->set_memsize ($mbytes); # Enable user networking just to be more like a "real" guest. $g->set_network (1); # Launch guest with one dummy disk. $g->add_drive ("/dev/null"); $g->launch (); # Get process ID of qemu. my $pid = $g->get_pid (); die unless $pid > 0; # Read the memory maps of the guest. open MAPS, "/proc/$pid/maps" or die "cannot open memory map of pid $pid"; my @maps = <MAPS>; close MAPS; # Kill qemu. $g->close (); # Parse the memory maps. my $shared_file_backed = 0; my $anonymous = 0; my $shared_writable = 0; my $map; foreach $map (@maps) { chomp $map; if ($map =~ m/ ^([0-9a-f]+)-([0-9a-f]+) \s (....) \s [0-9a-f]+ \s ..:.. \s (\d+) \s+ (\S+)? /x) { my ($start, $end) = (hex $1, hex $2); my $size = $end - $start; my $mode = $3; my $inode = $4; my $filename = $5; # could also be "[heap]", "[vdso]", etc. # Shared file-backed text: r-xp, r--p, etc. with a file backing. if ($inode != 0 && ($mode eq "r-xp" || $mode eq "r--p" || $mode eq "---p")) { $shared_file_backed += $size; } # Anonymous memory: rw-p. elsif ($mode eq "rw-p") { $anonymous += $size; } # Writable and shared. Not sure what this is ... elsif ($mode eq "rw-s") { $shared_writable += $size; } # Ignore [vdso], [vsyscall]. elsif (defined $filename && ($filename eq "[vdso]" || $filename eq "[vsyscall]")) { } # Ignore ---p with no file. What's this? elsif ($inode == 0 && $mode eq "---p") { } # Ignore kvm-vcpu. elsif ($filename eq "anon_inode:kvm-vcpu") { } else { warn "warning: could not parse '$map'\n"; } } else { die "incorrect maps format: '$map'"; } } printf("Shared memory backed by a file: %.2f MB\n", $shared_file_backed / 1024.0 / 1024.0); printf("Anonymous memory (eg. malloc, COW, stack), not shared: %.2f MB\n", $anonymous / 1024.0 / 1024.0); printf("Shared writable memory: %.2f MB\n", $shared_writable / 1024.0 / 1024.0); print "\n"; }
Categories: LUG Community Blogs
Syndicate content