Planet HantsLUG

Syndicate content
Planet HantsLUG - http://hantslug.org.uk/planet/
Updated: 50 min 19 sec ago

Steve Kemp: Slowly releasing bits of code

Sun, 29/06/2014 - 14:37

As previously mentioned I've been working on git-based DNS hosting for a while now.

The site was launched for real last Sunday, and since that time I've received enough paying customers to cover all the costs, which is nice.

Today the site was "relaunched", by which I mean almost nothing has changed, except the site looks completely different - since the templates/pages/content is all wrapped up in Bootstrap now, rather than my ropy home-made table-based layout.

This week coming I'll be slowly making some of the implementation available as free-software, having made a start by publishing CGI::Application::Plugin::AB - a module designed for very simple A/B testing.

I don't think there will be too much interest in most of the code, but one piece I'm reasonably happy with is my webhook-receiver.

Webhooks are at the core of how my service is implemented:

  • You create a repository to host your DNS records.
  • You configure a webhook to be invoked when pushing to that repository.
  • The webhook will then receive updates, and magically update your DNS.

Because the webhook must respond quickly, otherwise github/bitbucket/whatever will believe you've timed out and report an error, you can't do much work on the back-end.

Instead I've written a component that listens for incoming HTTP POSTS, parses the body to determine which repository that came from, and then enqueues the data for later processing.

A different process will be constantly polling the job-queue (which in my case is Redis, but could be beanstalkd, or similar. Hell even use MySQL if you're a masochist) and actually do the necessary magic.

Most of the webhook processor is trivial, but handling different services (github, bitbucket, etc) while pretending they're all the same is hard. So my little toy-service will be released next week and might be useful to others.

ObRandom: New pricing unveiled for users of a single zone - which is a case I never imagined I'd need to cover. Yay for A/B testing :)

Categories: LUG Community Blogs

Martin Wimpress: Humax Foxsat HDR Custom Firmware

Sun, 29/06/2014 - 12:00

I've had the notes kicking around for absolutely ages. I haven't checked to see if this stuff is still accurate because during the last 12 months or so our viewing habbits have changed and we almost exclusively watch streamed content now.

That said, my father-in-law gave us a Humax Foxsat HDR Freesat digital recorder as a thank you for some work I did for him. It turns out the Humax Foxsat HDR is quite hackable.

Hard Disk Upgrade

I contributed to the topic below. The Humax firmware only supports disks up to 1TB but the hackers method works for 2TB drives, possibly bigger but I haven't tried. I've upgraded mine and my father-in-laws to 2TB without any problems.

Custom Firmware

These are the topics that discuss the custom firmware itself and includes the downloads. Once the custom firmware is installed and it's advanced interface has been enabled you can enable several add-ons such as Samba, Mediatomb, ssh, ftp, etc.

Web Interface

This is the topic about the web interface. No download required it is bundled in the custom firmware above.

Channel Editor

The channel editor is installed and configured via the web interface and is one of my favourite add-ons. It also allows you to enable non-freesat channels in the normal guide.

Non Freesat channels

We get our broadcasts from Astra 2A / Astra 2B / Astra 2D / Eurobird 1 (28.2°E). Other free to air channels are available and KingOfSat lists them all. These channels can only be added via the Humax setup menus, but can then be presented in the normal EPG using the Channel Editor above.

Decrypt HD recordings

I never actually tried this, but what follows might be useful.

OTA updates

This is not so relevant now, since Humax haven't released an OTA update for some time.

I have not yet found a way to prevent automatic over the air updates from being automatically applied. When an over the air update is applied the Humax still works, but the web interface and all the add-ons stop working. The can be solved by waiting for the custom firmware to be updated (which happen remarkably quickly) and then re-flashing the custom firmware. All the add-ons should start working again.

Categories: LUG Community Blogs

Adam Trickett: Bog Roll: New Time Piece

Sat, 28/06/2014 - 17:11

Personal portable tine pieces are strange things. My first watch was a small red wind-up Timex that I was given when I could read the time. The strap eventually frayed out but it was still in good order when I was given a Timex LCD digital watch when they became popular and cheap. That watch lasted all of 12 months before it died to be replaced by a Casio digital watch which lasted many years and many straps.

At University I had my grandfather's watch, a mechanical Swiss watch that was very accurate as long as I remembered to keep it wound up. I continued to wear it as my regular watch in my first job. The only problem with this watch was it wasn't shock or waterproof and it was no good when I forgot to wind it up! My better half got me a Casio hybrid analogue/digital watch which I used for many years as my out doors/DIY watch.

I was somewhat disappointed when the resin case failed and though the watch was okay it wasn't wearable anymore and I replaced it with two watches, another similar Casio and a titanium cased Lorus quartz analogue display watch. The Casio failed in exactly the same was as the previous ones, dead straps and the a case failure.

My most recent watch is a Seiko titanium cased quartz analogue display similar to the Lorus (they are the same company) but this one is slightly nicer and battery is solar re-charged so it should never need replacing. The strap is also made of titanium so unlike the continually failing resin straps it should also last quite a bit longer...

The irony of all these watches is that I sit in-front of a computer for a living, and often at home - all of which have network synchronised clocks on them that are far more reliable than any of my wrist-watches. When I'm not at a computer I usually have my mobile phone with me or another high precision time pieces is available...

Categories: LUG Community Blogs

Martin Wimpress: Integrating Dropbox photo syncing with Open Media Vault and Plex

Sat, 28/06/2014 - 12:00

I've installed Open Media Vault on a HP ProLiant MicroServer G7 N54L and use it as media server for the house. OpenMediaVault (OMV) is a network attached storage (NAS) solution based on Debian Linux. At the time of writing OMV 0.5.x is based on Debian 6.0 (Squeeze).

I use a free Dropbox account to sync photos from mine and my wife's Android phones and wanted to automate to import of these photo upload into Plex, which is also running on Open Media Vault.

Installing Dropbox on Open Media Vault 0.5.x

I looked for a Dropbox Plugin for Open Media Vault and found this:

Sadly, at the time of writing, it is unfinished and I didn't have the time to go and learn the Open Media Vault plugin API.

The Open Media Vault forum does include a Dropbox HOW-TO which is very similar to how I've run Dropbox on headless Linux servers in the past. So, I decided to adapt my existing notes to Open Media Vault.

Create a Dropbox Share

Create a Dropbox share via the OMV WebUI.

  • Access Right Management -> Shared Folders

I gave my the name "Dropbox". I know, very original.

Installing Dropbox on a headless server

Download the latest Dropbox stable release for 32-bit or 64-bit.

wget -O dropbox.tar.gz "http://www.dropbox.com/download/?plat=lnx.x86" wget -O dropbox.tar.gz "http://www.dropbox.com/download/?plat=lnx.x86_64"

Extract the archive and install Dropbox in /opt.

cd tar -xvzf dropbox.tar.gz sudo mv ~/.dropbox-dist /opt/dropbox sudo find /opt/dropbox/ -type f -exec chmod 644 {} \; sudo chmod 755 /opt/dropbox/dropboxd sudo chmod 755 /opt/dropbox/dropbox sudo ln -s /opt/dropbox/dropboxd /usr/local/bin/dropboxd

Run dropboxd.

/usr/local/bin/dropboxd

You should see output like this:

This client is not linked to any account... Please visit https://www.dropbox.com/cli_link?host_id=d3adb33fcaf3d3adb33fcaf3d3adb33f to link this machine.

Visit the URL, login with your Dropbox account and link the account. You should see the following.

Client successfully linked, Welcome Web!

dropboxd will now create a ~/Dropbox folder and start synchronizing. Stop dropboxd with CTRL+C.

Symlink the Dropbox share

Login to the OMV server as root and sym-link the Dropbox share you created earlier to the Dropbox directory in the root home directory.

mv ~/Dropbox ~/Dropbox-old ln -s /media/<UUID>/Dropbox Dropbox rsync -av -W --progress ~/Dropbox-old/ ~/Dropbox/ init.d

To run Dropbox as daemon with init.d. Create /etc/init.d/dropbox with the following content.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65#!/bin/sh ### BEGIN INIT INFO # Provides: dropbox # Required-Start: $local_fs $remote_fs $network $syslog $named # Required-Stop: $local_fs $remote_fs $network $syslog $named # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # X-Interactive: false # Short-Description: dropbox service ### END INIT INFO DROPBOX_USERS="root" DAEMON=/opt/dropbox/dropbox start() { echo "Starting dropbox..." for dbuser in $DROPBOX_USERS; do HOMEDIR=`getent passwd $dbuser | cut -d: -f6` if [ -x $DAEMON ]; then HOME="$HOMEDIR" start-stop-daemon -b -o -c $dbuser -S -u $dbuser -x $DAEMON fi done } stop() { echo "Stopping dropbox..." for dbuser in $DROPBOX_USERS; do HOMEDIR=`getent passwd $dbuser | cut -d: -f6` if [ -x $HOMEDIR/$DAEMON ]; then start-stop-daemon -o -c $dbuser -K -u $dbuser -x $DAEMON fi done } status() { for dbuser in $DROPBOX_USERS; do dbpid=`pgrep -u $dbuser dropbox` if [ -z $dbpid ] ; then echo "dropboxd for USER $dbuser: not running." else echo "dropboxd for USER $dbuser: running (pid $dbpid)" fi done } case "$1" in start) start ;; stop) stop ;; restart|reload|force-reload) stop start ;; status) status ;; *) echo "Usage: /etc/init.d/dropbox {start|stop|reload|force-reload|restart|status}" exit 1 esac exit 0

Enable the init.d script.

sudo chmod +x /etc/init.d/dropbox sudo update-rc.d dropbox defaults Starting and Stopping the Dropbox daemon

Use /etc/init.d/dropbox start to start and /etc/init.d/dropbox stop to stop.

Dropbox client

It is recommended to download the official Dropbox client to configure Dropbox and get its status.

wget "http://www.dropbox.com/download?dl=packages/dropbox.py" -O dropbox chmod 755 dropbox sudo mv dropbox /usr/local/bin/

You can check on Dropbox status by running the following.

dropbox status

For usage instructions run dropbox help.

Photo importing

So, the reason for doing all this is that I now have a Dropbox instance running on my home file server and everyday it runs a script, that I wrote, to automatically import new photos into a directory that Plex monitors. I'll post details about my photo sorting script, Phort, at a later date.

References
Categories: LUG Community Blogs

Steve Kemp: So I accidentally ... a service.

Mon, 23/06/2014 - 20:44

This post is partly introspection, and partly advertising. Skip if it either annoys you.

Back in February I was thinking about what to do with myself. I had two main options "Get a job", and "Start a service". Because I didn't have any ideas that seemed terribly interesting I asked people what they would pay for.

There were several replies, largely based "infrastructure hosting" (which was pretty much 50/50 split between "DNS hosting", and project hosting with something like trac, redmine, or similar).

At the time DNS seemed hard, and later I discovered there were already at least two well-regarded people doing DNS things, with revision control.

So I shelved the idea, after reaching out to both companies to no avail. (This later lead to drama, but we'll pretend it didn't.) Ultimately I sought and acquired gainful employment.

Then, during the course of my gainful employment I was exposed to Amazons Route53 service. It looked like I was going to be doing many things with this, so I wanted to understand it more thoroughly than I did. That lead to the creation of a Dynamic-DNS service - which seemed to be about the simplest thing you could do with the ability to programatically add/edit/delete DNS records via an API.

As this was a random hack put together over the course of a couple of nights I didn't really expect it to be any more popular than anything else I'd deployed, and with the sudden influx of users I wanted to see if I could charge people. Ultimately many people pretended they'd pay, but nobody actually committed. So on that basis I released the source code and decided to ignore the two main missing features - lack of MX records, and lack of sub-sub-domains. (Isn't it amazing how people who claim they want "open source" so frequently mean they want something with zero cost, they can run, and never modify and contribute toward?)

The experience of doing that though, and the reminder of the popularity of the original idea made me think that I could do a useful job with Git + DNS combined. That lead to DNS-API - GitHub based DNS hosting.

It is early days, but it looks like I have a few users, and if I can get more then I'll be happy.

So if you want to to store your DNS records in a (public) GitHub repository, and get them hosted on geographically diverse anycasted servers .. well you know where to go: Github-based DNS hosting.

Categories: LUG Community Blogs

Tony Whitmore: Tom Baker at 80

Mon, 23/06/2014 - 18:31

Back in March I photographed the legendary Tom Baker at the Big Finish studios in Kent. The occasion was the recording of a special extended interview with Tom, to mark his 80th birthday. The interview was conducted by Nicholas Briggs, and the recording is being released on CD and download by Big Finish.

I got to listen in to the end of the recording session and it was full of Tom’s own unique form of inventive story-telling, as well as moments of reflection. I got to photograph Tom on his own using a portable studio set up, as well as with Nick and some other special guests. All in about 7 minutes! The cover has been released now and it looks pretty good I think.

The CD is available for pre-order from the Big Finish website now. Pre-orders will be signed by Tom, so buy now!

Pin It
Categories: LUG Community Blogs

Andy Smith: How to work around lack of array support in puppetlabs-firewall?

Mon, 23/06/2014 - 04:28

After a couple of irritating firewalling oversights I decided to have a go at replacing my hacked-together firewall management scripts with the puppetlabs-firewall module.

It’s going okay, but one thing I’ve found quite irritating is the lack of support for arrays of things such as source IPs or ICMP types.

For example, let’s say I have a sequence of shell commands like this:

#!/bin/bash   readonly IPT=/sbin/iptables   for icmptype in redirect router-advertisement router-solicitation \ address-mask-request address-mask-reply; do $IPT -A INPUT -p icmp --icmp-type ${icmptype} -j DROP done

You’d think that with puppetlabs-firewall you could do this:

class bffirewall::prev4 { Firewall { require => undef, }   firewall { '00002 Disallow possibly harmful ICMP': proto => 'icmp', icmp => [ 'redirect', 'router-advertisement', 'router-solicitation', 'address-mask-request', 'address-mask-reply' ], action => 'drop', provider => 'iptables', } }

Well it is correct syntax which installs fine on the client, but taking a closer look it hasn’t worked. It’s just applied the first firewall rule out of the array, i.e.:

iptables -A INPUT -p icmp --icmp-type redirect -j DROP

There’s already a bug in Puppet’s JIRA about this.

Similarly, what if you need to add a similar rule for each of a set of source hosts? For example:

readonly MONITORS="192.168.0.244 192.168.0.238 192.168.4.71" readonly CACTI="192.168.0.246" readonly ENTROPY="192.168.0.215"   # Allow access from: # - monitoring hosts # - cacti # - the entropy VIP for host in ${MONITORS} ${CACTI} ${ENTROPY}; do $IPT -A INPUT -p tcp --dport 8888 -s ${host} -j ACCEPT done

Again, your assumption about what would work…

firewall { '08888 Allow egd connections': proto => 'tcp', dport => '8888', source => [ '192.168.0.244', '192.168.0.238', '192.168.4.71', '192.168.0.246', '192.168.0.215' ], action => 'accept', provider => 'iptables', }

…just results in the inclusion of a rule for only the first source host, with the rest being silently discarded.

This one seems to have an existing bug too; though it has a status of closed/fixed it certainly isn’t working in the most recent release. Maybe I need to be using the head of the repository for that one.

So, what to do?

Duplicating the firewall {} blocks is one option that’s always going to work as a last resort.

Puppet’s DSL doesn’t support any kind of iteration as far as I’m aware, though it will in future — no surprise, as iteration and looping is kind of a glaring omission.

Until then, does anyone know any tricks to cut down on the repetition here?

Categories: LUG Community Blogs

Martin Wimpress: OpenMediaVault on Debian

Sun, 22/06/2014 - 12:00

At the time of writing OpenMediaVault 0.6 is pre-release. But it is possible to install OpenMediaVault on Debian Wheezy in order to get some testing done.

Install Debian Wheezy on your target VM or test server. Go with the defaults until the 'Software selection' dialogue. Make sure everything is unselected, like this:

[ ] Debian desktop environment [ ] Web server [ ] Print server [ ] SQL database [ ] DNS Server [ ] File server [ ] Mail server [ ] SSH server [ ] Laptop [ ] Standard system utilities

After the install is complete, reboot and login to the new Debian system as root.

Update the repository sources and add the contrib and non-free repositories.

nano /etc/apt/sources.list

It should look something like this:

deb http://ftp.uk.debian.org/debian/ wheezy main contrib non-free deb-src http://ftp.uk.debian.org/debian/ wheezy main contrib non-free deb http://security.debian.org/ wheezy/updates main contrib non-free deb-src http://security.debian.org/ wheezy/updates main contrib non-free # wheezy-updates, previously known as 'volatile' deb http://ftp.uk.debian.org/debian/ wheezy-updates main contrib non-free deb-src http://ftp.uk.debian.org/debian/ wheezy-updates main contrib non-free

Now add the OpenMediaVault repository.

echo "deb http://packages.openmediavault.org/public kralizec main" > /etc/apt/sources.list.d/openmediavault.list

Update.

apt-get update

Install the OpenMediaVault repository key and Postfix.

apt-get install openmediavault-keyring postfix
  • When the 'Postfix Configuration' dialogue is displayed choose No configuration.

Update again and install OpenMediaVault.

apt-get update apt-get install openmediavault
  • When the 'Configuring mdadm' dialogue is displayed enter none.
  • Do you want to start MD arrays automatically? YES
  • When the 'ProFTPD configuration' dialogue is displayed choose standalone.

Initialise OpenMediaVault and reboot.

omv-initsystem reboot

After the reboot you should be able to connect to the OpenMediaVault WebUI and login as admin with the password of openmediavault.

That's it. Get testing.

Categories: LUG Community Blogs

Martin Wimpress: Setting up BitSync on Debian

Sat, 21/06/2014 - 12:00

I've replaced Dropbox with BitTorrent Sync. In order to do this I've have btsync running on a VPS (2CPU, 2GB, 400GB), my home server and assorted Arch Linux workstations.

I had a couple of reasons for migrating away from Dropbox.

  • Dropbox was costing $100 per year.
  • Dropbox encryption model is weak and I have data security/privacy.

The VPS I am running BitTorrent Sync on costs $50 per year and provides four times the storage. I run btsync on a VPS so that there is always a server "in the cloud" that is available to sync with so that my setup emulates what Dropbox used to do.

All my servers are running Debian and this is how I install btsync on Debian.

sh -c "$(curl -fsSL http://debian.yeasoft.net/add-btsync-repository.sh)" sudo apt-get install btsync

This is how I respond to the prompts:

  • Do you want to define a default BitTorrent Sync instance? : YES
  • BitTorrent Sync Daemon Credentials:
  • BitTorrent Sync Daemon Group:
  • Niceness of the BitTorrent Sync Daemon: 0
  • On which portnumber should BitTorrent Sync listen? 0
  • BitTorrent Sync Listen Port: 12345
  • Do you want BitTorrent Sync to request port mapping via UPNP? NO
  • Download Bandwith Limit: 0
  • Upload Bandwith Limit: 0
  • Web Interface Bind IP Address: 0.0.0.0
  • Web Interface Listen Port: 8888
  • The username for accessing the web interface: yourusername
  • The password for accessing the web interface: yourpassword

As you'll see, I don't use UPNP on my VPS. I elect a specific port (not actually 12345 by the way) and open that port up with ufw. I also only allow access to the WebUI port from another server I own which reverse proxies via nginx.

btsync works really well, I have it syncing hundreds of thousands of files that amount to several hundred gigabytes of data. On my Arch Linux workstations I use the brilliant btsync-gui and BitTorrent Sync is also available for Android.

That said, I still use a free Dropbox account to sync photos from mine and my wife's Android phones. I have a Dropbox instance running on my home file server and everyday it runs a script to automatically import these photos into Plex.

Such a shame, that at the time of writing, btsync is closed source :-( Maybe that will change but if it doesn't SyncThing may well be the answer when it has matured a little.

References
Categories: LUG Community Blogs

Steve Kemp: The perils of the cloud..

Fri, 20/06/2014 - 13:18

Recently two companies have suffed problems due to compromised AWS credentials:

  • Code Spaces
    • The company has effectively folded. Thier AWS account was compromised, and all their data and backups were deleted.
  • Bonsai
    • Within two minutes all their instances were terminated.
    • This is still live - watch updates of the recovery process.

I'm just about to commit to using Amazon for hosting DNS for paying customers, so this is the kind of thing that makes me paranoid.

I'll be storing DNS-data in Git, and if the zones were nuked on the Amazon-side I could re-upload them, but users would be dead regardless - because they'd need to update the nameservers in whois before the re-uploaded data would be useful.

I suspect I need to upload to two DNS providers, to get more redundency.

Currently I have a working system which allows me to push DNS records to a Git repository, and that seamlessly triggers a DNS update (i.e. A webhook trigged by github/bitbucket/whatever).

Before I publish anything I need to write more code, more documentation, and agree on pricing details. Then I'll setup a landing-page at http://dns-api.com/.

I've been challenged to find paying customers before launching, and thus far have two, which is positive.

The DHCP.io site has now been freed. I'm no longer going to try to commercialize it, instead I will only offer the Git-based product as a commercial service. On that basis I upped the service so users could manage up to five names per account, more if you mail me privately and beg ;)

(ObRandom: Google does hosted DNS with an API. They're expensive. I'm surprised I'd not heard of them doing this.)

Categories: LUG Community Blogs

Martin Wimpress: MATE Desktop on Debian Wheezy

Thu, 19/06/2014 - 22:00

I'm a member of the MATE Desktop team and until recently the majority of my involvement has been focused around Arch Linux.

However, I'm working on a MATE project that is based on a Debian derivative. MATE has recently been accepted into the Debian Backports repository for Wheezy, so I decided to do a "MATE from scratch" on Debian using an old netbook to get familiar with the MATE package naming differences between Arch Linux and Debian.

Install Debian

I installed Debian Wheezy from the netinst ISO to ensure the target install was as minimal as possible. I went with the defaults until the 'Software selection' dialogue, at this point unselect everything except "SSH server". Like this:

[ ] Debian desktop environment [ ] Web server [ ] Print server [ ] SQL database [ ] DNS Server [ ] File server [ ] Mail server [X] SSH server [ ] Laptop [ ] Standard system utilities Debian ISO with Firmware

If you're installing on hardware that requires additional firmware in order for it to work with Linux then use the netinst ISO that includes firmware.

Configure Debian

When the install is finished, reboot and configure Debian a little.

Repositories

You'll need to install lsb-release for the following to work.

apt-get install lsb-release

This is what I put in /etc/apt/sources.list.

cat >/etc/apt/sources.list<<EOF deb http://ftp.uk.debian.org/debian/ $(lsb_release -cs) main contrib non-free deb-src http://ftp.uk.debian.org/debian/ $(lsb_release -cs) main contrib non-free deb http://security.debian.org/ $(lsb_release -cs)/updates main contrib non-free deb-src http://security.debian.org/ $(lsb_release -cs)/updates main contrib non-free # $(lsb_release -cs)-updates, previously known as 'volatile' deb http://ftp.uk.debian.org/debian/ $(lsb_release -cs)-updates main contrib non-free deb-src http://ftp.uk.debian.org/debian/ $(lsb_release -cs)-updates main contrib non-free EOF Backports

MATE is only available in the wheezy-backports repository.

cat >/etc/apt/sources.list.d/backports.list <<EOF deb http://ftp.uk.debian.org/debian $(lsb_release -cs)-backports main contrib non-free deb-src http://ftp.uk.debian.org/debian $(lsb_release -cs)-backports main contrib non-free EOF

Update.

sudo apt-get update

All backports are deactivated by default (i.e. the packages are pinned to 100 by using ButAutomaticUpgrades: yes in the Release files. If you want to install something from backports run:

apt-get -t wheezy-backports install "package" Install MATE Desktop

First install the LightDM display manager.

apt-get install accountsservice lightdm lightdm-gtk-greeter

Now for the MATE Desktop itself.

apt-get -t wheezy-backports install mate-desktop-environment-extras NetworkManager

I typically use NetworkManager, so lets install that too.

apt-get install network-manager-gnome Supplementary

Depending on your hardware you may require CPU frequency utilities or additional firmware.

apt-get install cpufreqd cpufrequtil firmware-linux firmware-linux-nonfree

And, that's it! Reboot and you'll see the LightDM greeter waiting for your login credentials.

References
Categories: LUG Community Blogs