Planet HantsLUG

Syndicate content
Planet HantsLUG -
Updated: 2 min 53 sec ago

Philip Stubbs: Gear profile generator

Mon, 05/10/2015 - 11:32
Having been inspired by the gear generator found at I decided to have a go at doing this myself.

Some time ago, I had tried to do this in Java as a learning exercise. I only got so far and gave up before I managed to generate any involute curves required for the tooth profile. Trying to learn Java and the math required at the same time was probably too much and it got put aside.

Recently I had a look at the Go programming language. Then Matthias Wandel produced the page mentioned above, and I decided to have another crack at drawing gears.

The results so far can be seen on Github, and an example is shown here.

What I have learnt
  • Math makes my head hurt.
  • The Go programming language fits the way my brain works better than most other languages. I much prefer it to Java, and will try and see if I can tackle other problems with it, just for fun.
Categories: LUG Community Blogs

Adam Trickett: Bog Roll: Green gage chutney

Sat, 03/10/2015 - 10:20

Yesterday we made two batches of chutney: one was green gage and apple plus lots of spice and the second was just green gage based and less spicy. We now have used up all our green gages from this year - at long last...

Categories: LUG Community Blogs

Alan Pope: DevRelCon 2015 Trip Report

Fri, 02/10/2015 - 15:00

Huh, this turned out to be longer than I expected. Don’t feel obliged to read it, it’s more notes for myself, and to remind me of why I liked the event.


On Wednesday I went to DevRelCon in London. DevRelCon is “a one day single track conference for technical evangelists, developer advocates and anyone interested in developer relations” setup by Matthew Revell. I don’t think there’s a lot of difference between my role (defined as Community Manager) at Canonical and Developer Relations so figured it would probably have appropriate content for my role. Boy was I right!

DevRelCon was easily the single most valuable short conference I’ve ever attended. The speakers were knowledgeable, friendly and accessible, and easy to understand. I took a ton of notes, and will distil some of them down here, but will almost certainly keep referring back to them over the coming months as I look to implement some of the suggestions I heard.


The event took place at The Trampery Old Street, in Shoreditch, the trendy/hipster part of London. We had access to a bright and airy ‘ballroom’ and were served with regular drinks, snacks and a light lunch. Free WiFi was also available, which worked well, but I didn’t use it much as we had little time away from the talks.


The day consisted of a mix of long (40 minute) talks, some shorter (20 min) ones, and a few ‘lightning’ talks. Having a mix of durations worked well I think. We started a little late, but Matthew massaged the timetable to claw back some time, and as it was a single track day there was no real issue if things didn’t run to time, as you weren’t likely to run off to another talk, and miss something.

All the talks were great, but I took considerably more notes in some than others, so this is represented below in that I haven’t listed every talk.

Morning Talks

Rob SpectreTwilio – Scaling Developer Evangelism.

This started off well as Rob plugged in his laptop and we were greeted with an Ubuntu desktop! He started off detailing some interesting stats to focus our minds on who we’re evangelising to. Starting with the 18.2m developers worldwide, given ~3Bn smartphone users, and ~4Bn Internet users that means ~0.08% have the capability to write code. There’s a 6% year on year increase in developers, mostly in developing nations, the ratio is less in the western world. So for example India could overtake every other countries’ developer count by ~2017.

Rob talked at length about the structure of Developer Evangelists, Developer Educators and the Community Team at Twilio. The talk continued to outline how valuable developers are, how at Twilio their Developer Evangelists are the ‘Red Carpet’ to their community. I was struck by how very differently we (Open Source projects) and Ubuntu specifically treat contributors to the project.

There was also a section on running developer events, and Rob spent some time talking about strategies for successful events, and how those can feed back to improve your product. He also talked a little about measurement, which was also going to be covered in later talks that day.

Another useful anecdote Rob detailed was regarding conversion of talks into blog posts. While a talk at an event can catalyse the 20-100 people in the room, converting that into a detailed tutorial blog post can bring in hundreds or thousands more.

The final slide in Rob’s talk was “Would you recommend this talk?” with a phone number attendees could send a score to. I thought this was a particularly cunning strategy. There was also talk of using the external IP address of the venue WiFi as one factor to determine the effectiveness / conversion rate of attendees.

Cristiano BettaBraintree – Tooling your way to a great devrel team

Cristiano started off talking about BattleHack which I’d not heard of. These are in person events where teams of developers get 24 hours to work on a project fuelled by coffee, cake and Red Bull to be in with a chance of winning a cash prize and an amusing axe.

He then went on to talk about a personal project to manage event sign-ups. This replaces tools like Eventbrite and MailChimp and enables Cristiano to get a better handle on the success of his events.

Laura CowenIBM – Building a developer community in an enterprise world

Laura started off giving some history of the products and groups inside IBM who are responsible for WAS, the public facing developer sites and the struggles she’s had updating them

The interesting parts for me came when Laura was detailing the pain she had getting developer time to update documentation and engage with users and communities outside their own four walls. Laura also talked about the difficulty when interfacing developers and marketing, their differing goals and some strategies for coping.

I recognised for example the frustration in people wanting to publish everything on a developer site, whether it’s appropriate to the target audience or not. Sometimes we (in Ubuntu) fail to deeply consider the target audience before we publish articles, guides or documentation. I think we can do better here. Pushing back on content creators, and finding the right place for a published article is worth it, if the target audience is to be defended.

Lightning Talks

Shaunak Kashyapelastic – Getting the measure of DevRel

In this short talk Shaunak gave some interesting snippets on how elastic measure community engagement. I found a couple interesting which I felt we might use in Ubuntu. Measuring “time to first response” for questions and issues by looking for responses from someone other than the first poster. While I don’t think they were actively using this data yet, getting an initial base line would be useful.

Shaunak also detailed one factor in measuring meet-up effectiveness. Typically elastic have 3-4 meet-ups a week, globally. For each meet-up group they measured “time since last meetup”. For those where there was a long delta between one meetup and the next they would consider actions. This could be contacting the group to see if there’s issues, offering assistance, swag & ‘meet up in a box’ kits, and finally disbanding the group if there wasn’t sufficient critical mass.

I took away a few good ideas from this talk, especially given recent conversations in Ubuntu about sparking up more meet-ups.

Phil LeggetterPusher – ROI on DevRel

Phil kicked off his short talk by talking about the ROI on DevRel by explaining Acquisition vs Activation. Where Acquisition of new developers might be them signing up for an account or downloading a product/sdk/library. Activation would be the conversion which might be measured differently per product. So perhaps “purchased paid API key” or “submitted app with N downloads”.

Phil then moved on to talk a bit about how they can measure the effectiveness of online tutorials or blog articles by correlating sign ups with traffic coming from those online articles. There was some more discussion on this later on including the effectiveness of giving away vouchers/codes to incentivise downloads, with some disagreement on the results of doing so.

Afternoon Talks

Brandon WestSendGrid – Burnout

I’ve been to many talks and discussions about burnout in developer communities over the years. This talk from Brandon one was easily the most useful, factual and actionable one. I also enjoyed Brandon’s attempts to inject Britishness into his talk which lightened the mood on a potentially very dark topic.

Brendon kicked off with a bit of a ‘woe is me’ #firstworldproblems introduction to his own current life issues. The usual things that affect a lot of people, but all happening at once, becoming overwhelming. We then moved on to defining burnout clearly, and what types of people are likely to suffer (clue: anyone) and some strategies for recognizing and preventing burnout.

A few key assertions / take-aways:-

“Burnout & depression are pathalogically indistinguishable”

“Burnout and work engagement are not exclusive or correlatable”

“Those most likely to burnout believe they are least at risk”

“Learn a skill on holiday – the holiday will be more rewarding”

Tim FogartyMajor League Hacking – Hackathons as a part of your DevRel strategy

Another great talk which built upon what Cristiano talked about earlier in the day – hackathons. Tim introduced different types of hackathons and which in his experience were more popular with developers and why.

Tim started by breaking down the types of hackathon – ‘hacking’, ‘corporate’ and ‘civic’ with the second being least popular as it’s seen as free labour by developers, and so they’re distrustful. He went on to reasons why people might run hackathons including evangelism, gathering (+ve and -ve) feedback, recruiting and mindshare (marketing).

He then moved on to strategies for making an impact, measuring the effect, sponsoring and how to craft the perfect demo to kick off the event.

Having never been to an in-person hackathon I found this another fascinating talk and will be following up with Tim Later.

Jessica Rose – Stop talking about diversity and just do it

Well. This was enlightening. This talk was excellent, and covered two main topics. First the focus was on getting a more diverse set of people running / attending / talking at your event. Some strategies were discussed and Jessica highlighted where many people go terribly wrong, assumptions people make and excuses people give.

The second part was a conversation about the ways in which an event can cater for as many people as possible. Here’s a highlight of some of the ways we discussed, but this obviously doesn’t cover everything:-

  • Attendees and speakers should be able to get in under their own power
  • Meal choices should be available – possibly beyond vegetarian/vegan
  • Code of Conduct
  • Sign language for talks
  • Well lit and safe feeling route from venue to accomodation
  • Space for breastfeeding / pumping, with snacks / drinks nearby
  • Non boozy spaces
  • Prayer room
  • After party with low noise level – and covered by Code of Conduct
  • Childcare
  • Professional chapparones (for under 18’s)
  • Diversity tickets & travel grants
  • Scale inclusivity to budget (be realistic about what you can achieve)

Lots to think about!

Joe NashBraintree – Engaging Students

Joe kicked off his fast-paced talk with an introduction to things which influenced how he got where he is, including “Twilio Heroes”. The talk was focussed on the UK University system, how to engage with students and some tips for running events which engage effectively with both CS and non-CS students.

James Milnerersi UK – So you want to run a meet-up

James talked about his personal experience running GeoDev Meet-Ups. I found this information quite valuable as the subject is under discussion in Ubuntu. James gave some great tips for running good meet-ups, and had a number of things he’s clearly learned the hard way. I hope to put some of his tips into action in the UK.

Dawn FosterLessons about community from science fiction.

This was a great uplifting talk to end the day. Dawn drew inspiration from her prolific science fiction reading to come up with some tips for people running community projects. I’ll give you a flavour with a few of them. Each was accompanied by an appropriate picture.

Picture: Star Trek Red Shirt
Lesson: “Participate and contribute in a way that people will notice and value your work”

Picture: Doctor Who TARDIS
Lesson: “Communities look different from inside then when viewing as an outsider”

Picture: Enders Game
Lesson: “Age is often unknown, encourage young people to contribute”

Dawn is a thoughtful, entertaining and engaging speaker. I’d certainly like to see more of her talks.

After Party

We all left the venue after the last talk and headed to a nearby trendy bar for a pint then headed home, pretty exhausted. A great event, I look forward to the next one.

Categories: LUG Community Blogs

Adam Trickett: Bog Roll: French Clothing Sizes

Sun, 27/09/2015 - 12:24

On Friday we spent several hours and quite a bit of money buying stuff in Decathlon. We needed some specific things and some things were on sale after the summer. Owing to my body shape change as a result of losing nearly 20 kg this year, a lot of my clothes - regular and sport - don't fit properly. Some I've been able to alter and some I can get away with, but some are now uncomfortable to wear or look absurd...

Decathlon design a lot of their own kit and then have it made all over the world. In that respect they are no different from many other companies both British and foreign. What is striking though is that unlike British and American brands, the stated size is more often the actual stated size, rather than a vanity size. For example to buy M&S or Next trousers I need to buy one size smaller than the quoted size or they fall down, but at Decathlon, I just need to buy the correct size and they fit...

Categories: LUG Community Blogs

Adam Trickett: Bog Roll: Vanity Sizing

Wed, 16/09/2015 - 13:32

Owing to a reduction in my body mass I've been forced to buy new clothing. It's been an expensive process and annoying to replace otherwise perfectly usable clothes...

I happened to have some older clothes that were not used much as they had previously been a little on the small size. The good news is that I've been able to wear them more regularly so I've avoided buying a few things!

One thing does annoy me. Unlike women's clothing which uses strange numbers, most men's clothing using simple actual measurements, waist of x inches / y centimetres. It's a fairly straight forward system, you know how big your waist size etc and should should be able to buy off-the-peg clothing without too much of a worry. However it's not true anymore! My older clothes when measured with a tape are the size that the label says. The modern ones vary wildly - though they all seem to be much larger than the label says. For example:

  • 20 year old pairs of M&S St. Michael Jeans, says 34", are 34"
  • 20 year old pairs of BHS chinos, says 34", are 34"
  • Modern M&S relaxed fit chinos, says 36", at least 38" more as they age
  • Modern Craghoppers Kiwi walking trousers, says 30", more like 33"

Some clothes, like jeans and those with elasticated bits do get larger with wear and washing, but even so modern clothes seem to be at least once size (2"/5 cm) larger than the label says they should be. Older clothing from the same company is the size it says...

At the moemnt I'm about a real size of 33"/84 cm waist, which is a pain as British men's trousers don't come in odd sizes, so it's either a real size of 32" (tight) or 34" (loose), but in vanity sizing that could be anything from 30"...

Categories: LUG Community Blogs

Steve Kemp: All about sharing files easily

Sun, 13/09/2015 - 13:39

Although I've been writing a bit recently about file-storage, this post is about something much more simple: Just making a random file or two available on an ad-hoc basis.

In the past I used to have my email and website(s) hosted on the same machine, and that machine was well connected. Making a file visible just involved running ~/bin/publish, which used scp to write a file beneath an apache document-root.

These days I use "my computer", "my work computer", and "my work laptop", amongst other hosts. The SSH-keys required to access my personal boxes are not necessarily available on all of these hosts. Add in firewall constraints and suddenly there isn't an obvious way for me to say "Publish this file online, and show me the root".

I asked on twitter but nothing useful jumped out. So I ended up writing a simple server, via sinatra which would allow:

  • Login via the site, and a browser. The login-form looks sexy via bootstrap.
  • Upload via a web-form, once logged in. The upload-form looks sexy via bootstrap.
  • Or, entirely seperately, with HTTP-basic-auth and a HTTP POST (i.e. curl)

This worked, and was even secure-enough, given that I run SSL if you import my CA file.

But using basic auth felt like cheating, and I've been learning more Go recently, and I figured I should start taking it more seriously, so I created a small repository of learning-programs. The learning programs started out simply, but I did wire up a simple TOTP authenticator.

Having TOTP available made me rethink things - suddenly even if you're not using SSL having an eavesdropper doesn't compromise future uploads.

I'd also spent a few hours working out how to make extensible commands in go, the kind of thing that lets you run:

cmd sub-command1 arg1 arg2 cmd sub-command2 arg1 .. argN

The solution I came up with wasn't perfect, but did work, and allow the seperation of different sub-command logic.

So suddenly I have the ability to run "subcommands", and the ability to authenticate against a time-based secret. What is next? Well the hard part with golang is that there are so many things to choose from - I went with gorilla/mux as my HTTP-router, then I spend several hours filling in the blanks.

The upshot is now that I have a TOTP-protected file upload site:

publishr init - Generates the secret publishr secret - Shows you the secret for import to your authenticator publishr serve - Starts the HTTP daemon

Other than a lack of comments, and test-cases, it is complete. And stand-alone. Uploads get dropped into ./public, and short-links are generated for free.

If you want to take a peak the code is here:

The only annoyance is the handling of dependencies - which need to be "go got ..". I guess I need to look at godep or similar, for my next learning project.

I guess there's a minor gain in making this service available via golang. I've gained protection against replay attacks, assuming non-SSL environment, and I've simplified deployment. The downside is I can no longer login over the web, and I must use curl, or similar, to upload. Acceptible tradeoff.

Categories: LUG Community Blogs

Adam Trickett: Bog Roll: Weight Reduction Rate

Sun, 13/09/2015 - 12:21

Over the weekend I recalculated my weight reduction rate target. As you lose weight your BMR falls, mine has come down by about 700 kj. That means to lose weight at the same rate as before I have to eat even less than I was previously. That means eating such a low energy diet that I'll probably miss important stuff out and I could start to lose muscle mass rather than fat.

Since hitting my weight plateau a few weeks ago I've been careful to not over indulge and to push harder on the bike. Re-plotting my weight against target on the new weekly reduction rate of 550 g per week rather than 750 g per week has resulted in a more realistic trajectory that I'm sticking to. Even after my holiday I'm still on target and should hit a healthy weight at the end of November this year.

One problem I do face is clothing. Lots of my clothes now fit me like a tent. Trousers fall down and shirts flap about in the wind... I've bought some smaller clothing, men's size small or medium rather than large or extra-large as previous, but I'm waiting until I reach my healthy target weight so I don't end up with new clothes that are too large. One problem I will face is that, in Basingstoke at least, I can't buy men's casual trousers in a small enough size in any of the local department stores, they don't stock anything small enough! Jeans I can get as they sell them to teenagers who should be smaller than full grown men, but they aren't really allowed for work...

Categories: LUG Community Blogs

Adam Trickett: Bog Roll: Cambridge Gage Jam

Wed, 09/09/2015 - 23:11

Last year our gage tree (probably a Cambridge Gage) had plenty of fruit but they were all inedible. This year it had plenty of fruit, so much so that as fast as we collect it there is even more ready to collect....

It's been a while since we had gages to jam. I used the same method as previously, though I added a fraction more sugar as the fruit wasn't fully ripe. Today's batch was 1.7 kg of fruit (cleaned and destoned), 350 g water and 1.2 kg of sugar, plus the usual juice of a frozen and defrosted lemon. The yield was pretty good and as we have loads of fruit left, even after we give some of them away I'll do another batch later this week.

Categories: LUG Community Blogs

Steve Kemp: The Jessie 8.2 point-release broke for me

Mon, 07/09/2015 - 10:37

I have about 18 personal hosts, all running the Jessie release of Debian GNU/Linux. To keep up with security updates I use unattended-upgrades.

The intention is that every day, via cron, the system will look for updates and apply them. Although I mostly expect it to handle security updates I also have it configured such that point-releases will be applied by magic too.

Unfortunately this weekend, with the 8.2 release, things broke in a significant way - The cron deamon was left in a broken state, such that all cronjobs failed to execute.

I was amazed that nobody had reported a bug, as several people on twitter had the same experience as me, but today I read through a lot of bug-reports and discovered that #783683 is to blame:

  • Old-cron runs.
  • Scheduled unattended-upgrades runs.
  • This causes cron to restart.
  • When cron restarts the jobs it was running are killed.
  • The system is in a broken state.

The solution:

# dpkg --configure -a # apt-get upgrade

I guess the good news is I spotted it promptly, with the benefit of hindsight the bug report does warn of this as being a concern, but I guess there wasn't a great solution.

Anyway I hope others see this, or otherwise spot the problem themselves.


In unrelated news the seaweedfs file-store I previously introduced is looking more and more attractive to me.

I reported a documentation-related bug which was promptly handled, even though it turned out I was wrong, and I contributed CIDR support to whitelisting hosts which was merged in well.

I've got a two-node "cluster" setup at the moment, and will be expanding that shortly.

I've been doing a lot of little toy-projects in Go recently. This weekend I was mostly playing with the message-bus, and tying it together with sinatra.

Categories: LUG Community Blogs

Debian Bits: New Debian Developers and Maintainers (July and August 2015)

Tue, 01/09/2015 - 12:45

The following contributors got their Debian Developer accounts in the last two months:

  • Gianfranco Costamagna (locutusofborg)
  • Graham Inggs (ginggs)
  • Ximin Luo (infinity0)
  • Christian Kastner (ckk)
  • Tianon Gravi (tianon)
  • Iain R. Learmonth (irl)
  • Laura Arjona Reina (larjona)

The following contributors were added as Debian Maintainers in the last two months:

  • Senthil Kumaran
  • Riley Baird
  • Robie Basak
  • Alex Muntada
  • Johan Van de Wauw
  • Benjamin Barenblat
  • Paul Novotny
  • Jose Luis Rivero
  • Chris Knadle
  • Lennart Weller


Categories: LUG Community Blogs

Andy Smith: Scrobbling to from D-Bus

Sun, 23/08/2015 - 12:50

Yesterday afternoon I noticed that my music player, Banshee, had not been scrobbling to my for a few weeks. seem to be in the middle of reorganising their site but that shouldn’t affect their API (at least not for scrobbling). However, it seems that it has upset Banshee so no more scrobbling for me.

Banshee has a number of deficiencies but there’s a few things about it that I really do like, so I wasn’t relishing changing to a different player. It’s also written in Mono which doesn’t look like something I could learn very quickly.

I then noticed that Banshee has some sort of D-Bus interface where it writes things about what it it doing, such as the metadata for the currently-playing track… and so a hackish idea was formed.

Here’s a thing that listens to what Banshee is saying over D-Bus and submits the relevant “now playing” and scrobble to The first time you run it it asks you to authorise it and then it remembers that forever.

I’ve never looked at D-Bus before so I’m probably doing it all very wrong, but it appears to work. Look, I have scrobbles again! And after all it would not be Linux on the desktop if it didn’t require some sort of lash-up that would make Heath Robinson cry his way to the nearest Apple store to beg a Genius to install iTunes, right?

Anyway it turns out that there is a standard for this remote control and introspection of media players, called MPRIS, and quite a few of them support it. Even Spotify, apparently. So it probably wouldn’t be hard to adapt this script to scrobble from loads of different things even if they don’t have scrobbling extensions themselves.

Categories: LUG Community Blogs

Debian Bits: Debian turns 22!

Sun, 16/08/2015 - 22:59

Sorry for posting so late, we're very busy at DebConf15!

Happy 22nd birthday Debian!

Categories: LUG Community Blogs

Adam Trickett: Bog Roll: Weight Plateau

Thu, 13/08/2015 - 22:09

After nearly 22 weeks of continuous and even weight loss I've hit my weigh plateau and not changed my weight for over three weeks now.

There are three basic reasons for this:

  1. My energy intake now equals my energy use. As you lose weight your total metabolic demand falls, so the deliberate energy deficit gradually shrinks to nothing. This is normal.
  2. Calorie creep, over time it's easy to add a little extra to your diet, which means the energy deficit isn't as big as it should be. This is common.
  3. Laziness creep, over time it's easy to slow down and not stick to the exercise plan. This is also common.

The closer you are to your target weight the more likely, and the easier it is to give and stay put. In my case all three are probably happening, my BMR has probably fallen by 168 kcal / 702 kj, which is 400 g of milk or 30 g of almonds - which isn't much but if you eat a few extra nuts or an extra glass of milk, it adds up...

To correct this, I've made sure I don't eat too many nuts (they are good for me in moderation) and I've cut down on the milk in my porridge, substituting water. I've also trimmed my bread, as good though it is, wheat has ~ 360 kcal per 100 g. I'll also try to push harder on the bike and walk faster...

I'm currently stuck under 74 kg, with about 8 kg to go...

Categories: LUG Community Blogs

Steve Kemp: Making an old android phone useful again

Thu, 13/08/2015 - 15:44

I've got an HTC Desire, running Android 2.2. It is old enough that installing applications such as thsoe from my bank, etc, fails.

The process of upgrading the stock ROM/firmware seems to be:

  • Download an unsigned zip file, from a shady website/forum.
  • Boot the phone in recovery mode.
  • Wipe the phone / reset to default state.
  • Install the update, and hope it works.
  • Assume you're not running trojaned binaries.
  • Hope the thing still works.
  • Reboot into the new O/S.

All in all .. not ideal .. in any sense.

I wish there were a more "official" way to go. For the moment I guess I'll ignore the problem for another year. My nokia phone does look pretty good ..

Categories: LUG Community Blogs

Steve Kemp: A brief look at the weed file store

Mon, 10/08/2015 - 14:29

Now that I've got a citizen-ID, a pair of Finnish bank accounts, and have enrolled in a Finnish language-course (due to start next month) I guess I can go back to looking at object stores, and replicated filesystems.

To recap my current favourite, despite the lack of documentation, is the Camlistore project which is written in Go.

Looking around there are lots of interesting projects being written in Go, and so is my next one the seaweedfs, which despite its name is not a filesystem at all, but a store which is accessed via HTTP.

Installation is simple, if you have a working go-lang environment:

go get

Once that completes you'll find you have the executable bin/weed placed beneath your $GOPATH. This single binary is used for everything though it is worth noting that there are distinct roles:

  • A key concept in weed is "volumes". Volumes are areas to which files are written. Volumes may be replicated, and this replication is decided on a per-volume basis, rather than a per-upload one.
  • Clients talk to a master. The master notices when volumes spring into existance, or go away. For high-availability you can run multiple masters, and they elect the real master (via RAFT).

In our demo we'll have three hosts one, the master, two and three which are storage nodes. First of all we start the master:

root@one:~# mkdir / root@one:~# weed master -mdir / -defaultReplication=001

Then on the storage nodes we start them up:

root@two:~# mkdir /data; root@two:~# weed volume -dir=/data -max=1 -mserver=one.our.domain:9333

Then the second storage-node:

root@three:~# mkdir /data; root@three:~# weed volume -dir=/data -max=1 -mserver=one.our.domain:9333

At this point we have a master to which we'll talk (on port :9333), and a pair of storage-nodes which will accept commands over :8080. We've configured replication such that all uploads will go to both volumes. (The -max=1 configuration ensures that each volume-store will only create one volume each. This is in the interest of simplicity.)

Uploading content works in two phases:

  • First tell the master you wish to upload something, to gain an ID in response.
  • Then using the upload-ID actually upload the object.

We'll do that like so:

laptop ~ $ curl -X POST http://one.our.domain:9333/dir/assign {"fid":"1,06c3add5c3","url":"","publicUrl":"","count":1} client ~ $ curl -X PUT -F file=@/etc/passwd,06c3add5c3 {"name":"passwd","size":2137}

In the first command we call /dir/assign, and receive a JSON response which contains the IPs/ports of the storage-nodes, along with a "file ID", or fid. In the second command we pick one of the hosts at random (which are the IPs of our storage nodes) and make the upload using the given ID.

If the upload succeeds it will be written to both volumes, which we can see directly by running strings on the files beneath /data on the two nodes.

The next part is retrieving a file by ID, and we can do that by asking the master server where that ID lives:

client ~ $ curl http://one.our.domain:9333/dir/lookup?volumeId=1,06c3add5c3 {"volumeId":"1","locations":[ {"url":"","publicUrl":""}, {"url":"","publicUrl":""} ]}

Or, if we prefer we could just fetch via the master - it will issue a redirect to one of the volumes that contains the file:

client ~$ curl http://one.our.domain:9333/1,06c3add5c3 <a href=",06c3add5c3">Moved Permanently</a>

If you follow redirections then it'll download, as you'd expect:

client ~ $ curl -L http://one.our.domain:9333/1,06c3add5c3 root:x:0:0:root:/root:/bin/bash ..

That's about all you need to know to decide if this is for you - in short uploads require two requests, one to claim an identifier, and one to use it. Downloads require that your storage-volumes be publicly accessible, and will probably require a proxy of some kind to make them visible on :80, or :443.

A single "weed volume .." process, which runs as a volume-server can support multiple volumes, which are created on-demand, but I've explicitly preferred to limit them here. I'm not 100% sure yet whether it's a good idea to allow creation of multiple volumes or not. There are space implications, and you need to read about replication before you go too far down the rabbit-hole. There is the notion of "data centres", and "racks", such that you can pretend different IPs are different locations and ensure that data is replicated across them, or only within-them, but these choices will depend on your needs.

Writing a thin middleware/shim to allow uploads to be atomic seems simple enough, and there are options to allow exporting the data from the volumes as .tar files, so I have no undue worries about data-storage.

This system seems reliable, and it seems well designed, but people keep saying "I'm not using it in production because .. nobody else is" which is an unfortunate problem to have.

Anyway, I like it. The biggest omission is really authentication. All files are public if you know their IDs, but at least they're not sequential ..

Categories: LUG Community Blogs

Andy Smith: SSDs and Linux Native Command Queuing

Sun, 09/08/2015 - 08:10
Native Command Queueing

Native Command Queuing (NCQ) is an extension of the Serial ATA protocol that allows multiple requests to be sent to a drive, allowing the drive to order them in a way it considers optimal.

This is very handy for rotational media like conventional hard drives, because they have to move the head all over to do random IO, so in theory if they are allowed to optimise ordering then they may be able to do a better job of it. If the drive supports NCQ then it will advertise this fact to the operating system and Linux by default will enable it.

Queue depth

The maximum depth of the queue in SATA is 31 for practical purposes, and so if the drive supports NCQ then Linux will usually set the depth to 31. You can change the depth by writing a number between 1 and 31 to /sys/block/<device>/device/queue_depth. Writing 1 to the file effectively disables NCQ for that device.

NCQ and SSDs

So what about SSDs? They aren’t rotational media; any access is in theory the same as any other access, so no need to optimally order the commands, right?

The sad fact is, many SSDs even today have incompatibilities with SATA drivers and chipsets such that NCQ does not reliably work. There’s advice all over the place that NCQ can be disabled with no ill effect, because supposedly SSDs do not benefit from it. Some posts even go as far as to suggest that NCQ might be detrimental to performance with SSDs.

Well, let’s see what fio has to say about that.

The setup
  • Two Intel DC s3610 1.6TB SSDs in an MD RAID-10 on Debian 8.1.
  • noop IO scheduler.
  • fio operating on a 4GiB test file that is on an ext4 filesystem backed by LVM.
  • fio set to do a 70/30% mix of read vs write operations with 128 simultaneous IO operations in flight.

The goal of this is to simulate a busy highly parallel server load, such as you might see with a database.

The fio command line looks like this:

fio --randrepeat=1 \ --ioengine=libaio \ --direct=1 \ --gtod_reduce=1 \ --name=ncq \ --filename=test \ --bs=4k \ --iodepth=128 \ --size=4G \ --readwrite=randrw \ --rwmixread=70

Expected output will be something like this:

ncq: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128 fio-2.1.11 Starting 1 process Jobs: 1 (f=1): [m(1)] [100.0% done] [50805KB/21546KB/0KB /s] [12.8K/5386/0 iops] [eta 00m:00s] ncq1: (groupid=0, jobs=1): err= 0: pid=11272: Sun Aug 9 06:29:33 2015 read : io=2867.6MB, bw=44949KB/s, iops=11237, runt= 65327msec write: io=1228.5MB, bw=19256KB/s, iops=4813, runt= 65327msec cpu : usr=4.39%, sys=25.20%, ctx=732814, majf=0, minf=6 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1% issued : total=r=734099/w=314477/d=0, short=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=128   Run status group 0 (all jobs): READ: io=2867.6MB, aggrb=44949KB/s, minb=44949KB/s, maxb=44949KB/s, mint=65327msec, maxt=65327msec WRITE: io=1228.5MB, aggrb=19255KB/s, minb=19255KB/s, maxb=19255KB/s, mint=65327msec, maxt=65327msec   Disk stats (read/write): dm-0: ios=732755/313937, merge=0/0, ticks=4865644/3457248, in_queue=8323636, util=99.97%, aggrios=734101/314673, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00% md4: ios=734101/314673, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=364562/313849, aggrmerge=2519/1670, aggrticks=2422422/2049132, aggrin_queue=4471730, aggrutil=94.37% sda: ios=364664/313901, merge=2526/1618, ticks=2627716/2223944, in_queue=4852092, util=94.37% sdb: ios=364461/313797, merge=2513/1722, ticks=2217128/1874320, in_queue=4091368, util=91.68%

The figures we’re interested in are the iops= ones, in this case 11237 and 4813 for read and write respectively.


Here’s how different NCQ queue depths affected things. Click the graph image for the full size version.


On this setup anything below a queue depth of about 8 is disastrous to performance. The aberration at a queue depth of 19 is interesting. This is actually repeatable. I have no explanation for it.

Don’t believe anyone who tells you that NCQ is unimportant for SSDs unless you’ve benchmarked that and proven it to yourself. Disabling NCQ on an Intel DC s3610 appears to reduce its performance to around 25% of what it would be with even a queue depth of 8. Modern SSDs, especially enterprise ones, have a parallel architecture that allows them to get multiple things done at once. They expect NCQ to be enabled.

It’s easy to guess why 8 might be the magic number for the DC s3610:

The top of the PCB has eight NAND emplacements and Intel’s proprietary eight-channel PC29AS21CB0 controller.

The newer NVMe devices are even more aggressive with this; while the SATA spec stops at one queue with a depth of 32, NVMe specifies up to 65k queues with a depth of up to 65k each! Modern SSDs are designed with this in mind.

Categories: LUG Community Blogs

Steve Kemp: The differences in Finland start at home.

Thu, 30/07/2015 - 10:09

So we're in Finland, and the differences start out immediately.

We're renting a flat, in building ten, on a street. You'd think "10 Streetname" was a single building, but no. It is a pair of buildings: 10A, and 10B.

Both of the buildings have 12 flats in them, with 10A having 1-12, and 10B having 13-24.

There's a keypad at the main entrance, which I assumed was to let you press a button and talk to the people inside "Hello I'm the postmaster", but no. There is no intercom system, instead you type in a magic number and the door opens.

The magic number? Sounds like you want to keep that secret, since it lets people into the common-area? No. Everybody has it. The postman, the cleaners, the DHL delivery man, and all the ex-tenants. We invited somebody over recently and gave it out in advance so that they could knock on our flat-door.

Talking of cleaners: In the UK I lived in a flat and once a fortnight somebody would come and sweep the stair-well, since we didn't ever agree to do it ourselves. Here somebody turns up every day, be it to cut the grass, polish the hand-rail, clean the glass on the front-door, or mop the floors of the common area. Sounds awesome. But they cut the grass, right outside our window, at 7:30AM. On the dot. (Or use a leaf-blower, or something equally noisy.)

All this communal-care is paid for by the building-association, of which all flat-owners own shares. Sounds like something we see in England, or even like Americas idea of a Home-Owners-Association. (In Scotland you own your own flat, you don't own shares of an entity which owns the complete building. I guess there are pros and cons to both approaches.)

Moving onwards other things are often the same, but the differences when you spot them are odd. I'm struggling to think of them right now, somebody woke me up by cutting our grass for the second time this week (!)

Anyway I'm registered now with the Finnish government, and have a citizen-number, which will be useful, I've got an appointment booked to register with the police - which is something I had to do as a foreigner within the first three months - and today I've got an appointment with a local bank so that I can have a euro-bank-account.

Happily I did find a gym to join, the owner came over one Sunday to give me a tiny-tour, and then gave me a list of other gyms to try if his wasn't good enough - which was a nice touch - I joined a couple of days later, his gym is awesome.

(I'm getting paid in UK-pounds, to a UK-bank, so right now I'm getting local money by transferring to my wifes account here, but I want to do that to my own, and open a shared account for paying for rent, electricity, internet, water, & etc).

My flat back home is still not rented, because the nice property management company lost my keys. Yeah you can't make that up can you? With a bit of luck the second set of keys I mailed them will arrive soon and the damn thing can be occupied, while I'm not relying on that income I do wish to have it.

Categories: LUG Community Blogs

Alan Pope: Easily port mobile HTML5 games to Ubuntu Phone

Tue, 28/07/2015 - 13:36

Article also available in Spanish at thanks to Marcos Costales.

I really like playing games on my phone & tablet and wanted some more games to play on Ubuntu. With a little work it turns out it’s really pretty easy to ‘port’ games over to Ubuntu phone. I put the word ‘port’ in quotes simply because in some cases it’s not a tremendous amount of effort, so calling it a ‘port’ might make people think it’s more work than it is.

Update: A few people have asked why someone would want to even do this, and why not just bookmark a game in the browser. Sorry if that’s not clear. With this method the game is entirely cached offline on the customer phone. Having fully offline games is desirable in many situations including when travelling or in a location with spotty Internet access. Not all games are fully offline of course, this method wouldn’t help with a large on-line multi-player game like Clash of Clans for example. It would be great for many other titles though. This method also makes use of application confinement on Ubuntu so the app/game cannot access anything outside of the game data directory.

I worked with sturmflut from the Ubuntu Insiders on this over a few evenings and weekends. He wrote it up in his post Panda Madness.

We had some fun porting a few games and I wanted to share what we did so others can do the same. We created a simple template on github which can be used as a starting point, but I wanted to explain the process and the issues I had, so others can port apps/games.

If you have any questions feel free to leave me a comment, or if you’d rather talk privately you can get in contact in other ways.

Proof of concept

To prove that we could easily port existing games, we licensed a couple of games from Code Canyon. This is a marketplace where developers can license their games either for other developers to learn from, build upon or redistribute as-is. I started with a little game called Don’t Crash which is an HTML5 game written using Construct 2. I could have licensed other games, and other marketplaces are also available, but this seemed like a good low-cost way for me to test out this process.

Side note: Construct 2 by Scirra is a popular, powerful, point-and-click Windows-only tool for developing cross-platform HTML5 apps and games. It’s used by a lot of indie game developers to create games for desktop browsers and mobile devices alike. In development is Construct 3 which aims to be backwards compatible, and available on Linux too.

Before I licensed Don’t Crash I checked it worked satisfactorily on Ubuntu phone using the live preview feature on Code Canyon. I was happy it worked, so I paid and received a download containing the ‘source’ Construct 2 files.

If you’re a developer with your own game, then you can of course skip the above step, because you’ve already got the code to port.

Porting to Ubuntu

The absolute minimum needed to port a game is a few text files and the directory containing the game code. Sometimes a couple of tweaks are needed for things like permissions and lock rotation, but mostly it Just Works(TM).

I’m using an Ubuntu machine for all the packaging and testing, but in this instance I needed a Windows machine to export out the game runtime using Construct 2. Your requirements may vary, but for Ubuntu if you don’t have one, you could install it in a VM like VMWare or VirtualBox, then add the SDK tools as detailed at

This is the entire contents of the directory, with the game itself in the www/ folder.

alan@deep-thought:~/phablet/code/popey/licensed/html5_dontcrash⟫ ls -l total 52 -rw-rw-r-- 1 alan alan 171 Jul 25 00:51 app.desktop -rw-rw-r-- 1 alan alan 167 Jun 9 17:19 app.json -rw-rw-r-- 1 alan alan 32826 May 19 19:01 icon.png -rw-rw-r-- 1 alan alan 366 Jul 25 00:51 manifest.json drwxrwxr-x 4 alan alan 4096 Jul 24 23:55 www Creating the metadata Manifest

This contains the basic details about your app like name, description, author, contact email and so on. Here’s mine (called manifest.json) from the latest version of Don’t Crash. Most of it should be fairly self-explanitory. You can simply replace each of the fields with your app details.

{ "description": "Don't Crash!", "framework": "ubuntu-sdk-14.10-html", "hooks": { "dontcrash": { "apparmor": "app.json", "desktop": "app.desktop" } }, "maintainer": "Alan Pope ", "name": "dontcrash.popey", "title": "Don't Crash!", "version": "0.22" }

Note: “popey” is my developer namespace in the store, you’ll need to specify your namespace which you configure in your account page on the developer portal.

Security profile

Named app.json, this details what permissions my app needs in order to run:-

{ "template": "ubuntu-webapp", "policy_groups": [ "networking", "audio", "video", "webview" ], "policy_version": 1.2 } Desktop file

This defines how the app is launched, what the icon filename is, and some other details:-

[Desktop Entry] Name=Don't Crash Comment=Avoid the other cars Exec=webapp-container $@ www/index.html Terminal=false Type=Application X-Ubuntu-Touch=true Icon=./icon.png

Again, change the Name and Comment fields, and you’re mostly done here.

Building a click package

With those files created, and an icon.png thrown in, I can now build my click package for uploading to the store. Here’s that process in its entirety:-

alan@deep-thought:~/phablet/code/popey/licensed⟫ click build html5_dontcrash/ Now executing: click-review ./ ./ pass Successfully built package in './'.

Which on my laptop took about a second.

Note the “pass” is output from the click-review tool which sanity checks click packages immediately after building, to make sure there’s no errors likely to cause it to be rejected from the store.

Testing on an Ubuntu device

Testing the click package on a device is pretty easy. It’s just a case of copying the click package over from my Ubuntu machine via a USB cable using adb, then installing it.

adb push /tmp adb shell pkcon install-local --allow-untrusted /tmp/

Switch to the app scope and pull down to refresh, tap the icon and play the game.


Tweaking the app

At this point for some of the games I noticed some issues which I’ll highlight here in case others also have them:-

Local loading of files

Construct 2 moans that “Exported games won’t work until you upload them. (When running on the file:/// protocol, browsers block many features from working for security reasons.” in a javascript popup and the game doesn’t start. I just removed that chunk of js which does the check from the index.html and the game works fine in our browser.

Device orientation

With the most recent Over The Air (OTA) update of Ubuntu we enabled device orientation everywhere which means some games can rotate and become unplayable. We can lock games to be portrait or landscape in the desktop file (created above) by simply adding this line:-


Obviously changing “portrait” to “landscape” if your game is horizontally played. For Don’t Crash I didn’t do this because the developer had coded orientation detection in the game, and tells the player to rotate the device when it’s the wrong way round.

Twitter links

Some games we ported have Twitter links in the game so players can tweet their score. Unfortunately the mobile web version of Twitter doesn’t support intents so you can’t have a link which contains the content “Check out my score in Don’t Crash” embedded in it for example. So I just removed the Twitter links for now.


Our browser doesn’t support locally served cookies. Some games use this. For Heroine Dusk I ported from cookies to Local Storage which worked fine.

Uploading to the store

Uploading click packages to the Ubuntu store is fast and easy. Simply visit, sign up/in, click “New Application” and follow the upload steps.

That’s it! I look forward to seeing some more games in the store soon. Patches also welcome to the template on github.

Categories: LUG Community Blogs

Adam Trickett: Bog Roll: GCN/Hannah Grant Energy Bars

Sat, 25/07/2015 - 12:58

Today I tried to make some GCN/Hannah Grant energy bars. I fist had to convert from silly cups into sensible units*, and we were missing pumpkin seeds but we had everything else.

  • 4 ripe bananas - about 340 g. Blended to a fine mush
  • 200 g rolled oats
  • 100 g dried fruit - we used rains
  • 60 g linseed/flax seeds - our were golden
  • 60 g sunflower seeds - ours were kernels only
  • 60 g almonds - chopped
  • 60 g pecans - chopped, we also had some cashews in this mix
  • cinnamon - I substituted nutmeg as my better half doesn't like cinnamon
  • salt - skipped as I'm on a low salt diet

Mix together, spread in a baking tray - ours wasn't deep enough, it should be 2 - 3 cm thick and bake on 170°C for 20 to 25 minutes until golden brown. Allow to cool for 10 minutes before cutting into energy bar shaped pieces. Store in an airtight container in the fridge.

Before baking it looks a bit like a home made lard & seed cake for garden birds, which in may respects it is, albeit with a lot less fat and lot more expensive ingredients!

Mine is now cooling and we'll try this it afternoon!

How do you measure a cup full of banana? Weights are far easier to use.

Categories: LUG Community Blogs

Steve Kemp: We're in Finland now.

Sat, 25/07/2015 - 03:00

So we've recently spent our first week together in Helsinki, Finland.

Mostly this has been stress-free, but there are always oddities about living in new places, and moving to Europe didn't minimize them.

For the moment I'll gloss over the differences and instead document the computer problem I had. Our previous shared-desktop system had a pair of drives configured using software RAID. I pulled one of the drives to use in a smaller-cased system (smaller so it was easier to ship).

Only one drive of a pair being present make mdadm scream, via email, once per day, with reports of failure.

The output of cat /proc/mdstat looked like this:

md2 : active raid1 sdb6[0] [LVM-storage-area] 1903576896 blocks super 1.2 2 near-copies [2/1] [_U] md1 : active raid10 sdb5[1] [/root] 48794112 blocks super 1.2 2 near-copies [2/1] [_U] md0 : active raid1 sdb1[0] [/boot] 975296 blocks super 1.2 2 near-copies [2/1] [_U]

See the "_" there? That's the missing drive. I couldn't remove the drive as it wasn't present on-disk, so this failed:

mdadm --fail /dev/md0 /dev/sda1 mdadm --remove /dev/md0 /dev/sda1 # repeat for md1, md2.

Similarly removing all "detached" drives failed, so the only thing to do was to mess around re-creating the arrays with a single drive:

lvchange -a n shelob-vol mdadm --stop /dev/md2 mdadm --create /dev/md2 --level=1 --raid-devices=1 /dev/sdb6 --force ..

I did that on the LVM-storage area, and the /boot partition, but "/" is still to be updated. I'll use knoppix/similar to do it next week. That'll give me a "RAID" system which won't alert every day.

Thanks to the joys of re-creation the UUIDs of the devices changed, so /etc/mdadm/mdadm.conf needed updating. I realized that too late, when grub failed to show the menu, because it didn't find it's own UUID. Handy recipe for the future:

set prefix=(md/0)/grub/ insmod linux linux (md/0)/vmlinuz-3.16.0-0.bpo.4-amd64 root=/dev/md1 initrd (md/0)//boot/initrd.img-3.16.0-0.bpo.4-amd64 boot
Categories: LUG Community Blogs