News aggregator

Mick Morgan: strip exif data

Planet ALUG - Sat, 11/01/2014 - 21:52

I have a large collection of photographs on my computer. And each Christmas the collection grows ever larger. I use digiKam to manage that collection, but as I have mentioned before, storing family photographs as a collection of jpeg files seems counter intuitive to me. Photographs should be on display, or at least stored in physical albums that family members can browse at will. At a push, even an old shoebox will do.

So when this Christmas I copied the latest batch of images from my camera to my PC, I did a quick count of the files I hold – it came to nearly 5,500. Ok, so many of these are very similar, and this is simply a reflection of the ease (and very marginal cost) of photographs today compared with the old 35mm days, but even so, that is a lot of images. Disappointingly few of these ever see the light of day because whilst both my wife and I can happily view them on-line, I don’t print enough of them to make worthwhile albums. Sure, actually /taking/ photographs is cheap these days, but printing them at home on the sort of inkjet printer most people posess is rather expensive. Which is where online print companies such as photopanda, snapfish, photobox or jessops come in.

Most of these companies will provide high quality prints in the most popular 6″ x 4″ size for around 5 pence each – so a batch of 40 Christmas pictures is not going to break the bank. But one nice innovation of the digital era is that you can get your photos pre-printed into hard back albums for very reasonable prices. Better yet, my wife pointed me to a “special offer” (70% off) being run by a site she has used in the past. That was such a bargain that I decided to go back over my entire collection and create a “year book” for each of the eleven years of digital images I currently hold (yes, I don’t get out much).

However, I don’t much like the idea of posting a large batch of photographs to a site run by a commercial company, even when that company may have a much less cavalier approach to my privacy than does say, facebook. Once the photographs have been posted, they are outside my control and could end up anywhere. And of course I am not just concerned with the actual images, but the metadata that goes with those images. All electronic images created by modern cameras (or more usually these days, smartphones) contain EXIF data which at the minimum will give date and time information alongside the technical details about the shot (exposure, flash timing etc). In the case of nearly all smartphones, and increasingly with cameras themselves, the image will also contain geo-location details derived from the camera’s GPS system. I don’t take many images with my smartphone, and in any case, its GPS system is resolutely turned off, but my camera (a Panasonic TZ40) contains not just a GPS location system but a map display capability. Sometimes too much technology gets crammed into devices which don’t necessarily need them. As digicamhelp points out, many popular photo sharing sites such as Flickr or picasa helpfully allow viewers to examine EXIF data on-line. It is exactly this sort of capability which is so scarily exploited by ilektrojohn’s creepy tool.

So, before posting my deeply personal pictures of my cats to a commercial site I thought I would scrub the EXIF data. This is made easy with Phil Harvey’s excellent exiftool. This tool is platform independent so OSX and Windows users can take advantage of its capabilities – though of course it is much more flexible when used in conjunction with an OS which offers shell scripting.

Exiftool allows you to selectively edit, remove or replace any of the EXIF data stored in an image. But in my case I simply wanted to remove /all/ EXIF data. This is easy with the “-all= ” switch. Thus having chosen (and copied) the images I wanted to post to the commercial site it was a matter of a minute or two to recursively edit those files with find – thus:

find . -type f -iname ‘*.jpg’ -exec exiftool -all= {} \;

Highly recommended – particularly if you are in the habit of using photo sharing sites.

Categories: LUG Community Blogs

Steve Kemp: Some productive work

Planet HantsLUG - Sat, 11/01/2014 - 19:15

Having decided to take a fortnight off, between looking for a new job, I assumed I'd spend a while coding.

Happily my wife, who is a (medical) doctor, has been home recently so we've got to spend time together instead.

I'm currently pondering projects which will be small enough to be complete in a week, but large enough to be useful. Thus far I've just reimplemented RSS -> chat which I liked a lot at Bytemark.

I have my own chat-server setup, which doesn't have any users but myself. Instead it has a bunch of rooms setup, and different rooms get different messages.

I've now created a new "RSS" room, and a bunch of RSS feeds get announced there when new posts appear. It's a useful thing if you like following feeds, and happen to have a chat-room setup.

I use Prosody as my chat-server, and I use my http2xmpp code to implement a simple HTTP-POST to XMPP broadcast mechanism.

The new script is included as examples/rss-announcer and just polls RSS feeds - URLs which haven't been broadcast previously are posted to the HTTP-server, and thus get injected into the chatroom. A little convoluted, but simple to understand.

This time round I'm using Redis to keep track of which URLs have been seen already.

Beyond that I've been doing a bit of work for friends, and have recently setup an nginx server which will handle 3000+ simultaneous connections. Not too bad, but I'm sure we can make it do better - another server running on BigV which is nice to see :)

I'll be handling a few Squeeze -> Wheezy upgrades in the next week too, setting up backups, and doing some other related "consultation".

If I thought there was a big enough market locally I might consider doing that full-time, but I suspect that relying upon random work wouldn't work long-term.

Categories: LUG Community Blogs

Andrew Savory: Mobile modem

Planet ALUG - Fri, 10/01/2014 - 23:01

I was trying to get a handle on how much mobile data download speeds have improved over the years, so I did some digging through my archives. (The only thing I like more than a mail archive that spans decades is a website that spans decades. Great work, ALUG!) Here’s some totally arbitrary numbers to illustrate a point.

In response to A few Questions, this is what I wrote in May 2002:

[Nokia] 7110 works fine with IR (7110 has a full modem). The 7110 supports 14.4bps
connections, but I bet your telco doesn’t

That should have been 14.4kbps (14,400bps). In 2002 the phones were ahead of the network’s ability to deliver. In 2014, not much has changed.

In GPRS on Debian, this is what I wrote in November 2002:

I finally took the plunge and went for GPRS [..]  (up to 5x the speed of a dialup connection over a GSM mobile connection)

Remember when we did dialup over mobile connections? GSM Arena on the Sony-Ericsson T68i states 24-36 kbps. I’m assuming I got the lower end of that.

In 2003 I was using a Nokia 3650 GPRS connection. GSM Arena on the Nokia 3650 states 24-36 kbps. Let’s be generous and assume reality was right in the middle, at 30 kbps.

In 2004 I got a Nokia 6600, which according to GSM Arena could also do 24 – 36 kbps. It was a great phone, so let’s assume the upper bound for the 6600.

In 2008 I upgraded to 3G with the Nokia N80, and wrote:

3G data connections are dramatically better than GPRS

… but sadly I didn’t quantify how much better. According to GSM Arena, it was 384 kbps.

That’s a pretty good and pretty dramatic speed increase:

But then in 2009 I was using the Nokia N900 (and iPhone, HTC Hero, Google Nexus One, …). GSM Arena on the Nokia N900 states a theoretical 10Mbps … quite the upgrade, except O2 were limited to 3.6 mbps.

In 2012 I was using the Samsung Galaxy SII. GSM Arena on the Samsung Galaxy SII promises 21 mbps.

And now the Sony Xperia Z Ultra supports LTE at 42 MBPS and 150MBPS. Sadly, the networks don’t yet fully support those speeds, but if they did, the chart would be truly dramatic. 2003-2008 starts to look like a rounding error:

I don’t need to use a modem or infrared, either. Things have really improved over the last twelve years!

(This post is probably best read in conjunction with Tom’s analysis of Mobile phone sizes.)

Categories: LUG Community Blogs

Peter Cannon: Linux Outlaws Live 300

Planet WolvesLUG - Fri, 10/01/2014 - 12:07

Linux Outlaws celebrated 300 episodes on Sunday 17th February 2013. Initially I was not intending to go even though my co-conspirator Matthew Copperwaite said “Well if I’m going you’ve got to go!” fortunately for everyone Travelodge just happened to send me an email offering rooms for £15.00 plus Trainline was offer return tickets to Liverpool for £16.00 that meant with the £5.00 entrance fee I could go to the event for the incredible price of £36.00.

I set out on a crisp, sunny Sunday morning from the village halt where I was to board my first train terminating at Rugeley Trent Valley. I had three trains in total to catch, each with a 10 minute separation. Knowing the UK rail system this was going to be a nail-biting experience. My first train arrived on time and my daughters waived me off from the platform, I suspect to make sure I actually left the county more than anything? We diesel chugged through the countryside and I discovered from the conductor that all I had to do was get off the train and my next connection arrived on the same platform. Now this is where things get interesting, so stood in a group was about seven or eight middle aged men drinking cans and bottles of larger? I thought “10:45am is a bit early in the day for that kind of thing isn’t it?” my next train appeared, the 10:43 to Stafford. Oh yes now we know why the larger louts was on the platform don’t we? As I alighted my carriage I was greeted by red vested, omg I must have a drink, football fans. Just fecking great. Now I know zilch about football but I was thinking “Hopefully they’ll get off at Crewe?” Oh you soppy thing Pete. The 11:09 London Liverpool Virgin train pulled in, Virgin trains are the perfect colour for what greeted me! Yep, I was on the football special Liverpool vs Swansea, whoopty fooking do one hour of larger clutching fans traipsing up and down the corridor to visit the toilet or try to find the buffet car.

Now Matt and I have some traditions when we go to events, Matt’s are generally passport/forgetting things related mine is mobile phone related. I always run out of credit mid transit! The day before I put £10 on my PAYG phone, I remembered not to upload lots of photo’s but I did check and make comments on my  G+ account fairly regularly, and I did also try to tether my phone via USB to my netbook but just outside Runcorn I’d used up my credit. This always sends me into a panic as I think “What the hell do I do if Matt contacts me?” Willing the train to “Get a fooking hurry on” we eventually pulled into Lime Street where I popped into WH Smiths and stuck some more credit on and texted Matt to say I had arrived and would wait for him. One hour later the other half of The Dick Turpin Road Show arrived and we set off for The LEAF.

Now we had arranged for one or two people to join us but the two faced gits sloped off to some private Chinese meal based event that Matt and I was not invited to, I mean, is that any way to treat friends? Bout 15:15 with tear stained cheeks Matt and I dragged our dejected bodies off to our respective hotels with Matt telling me to get back to his room by 15:45. I dunno what he was thinking? I was in the Strand which was at least a 10-15 minute walk, I had to book in, get cleaned up, change and then yomp it over to the Travelodge Central like some fooking Marine come paratrooper! I arrived at his room and barely managed to tap his door through my wheezing frail carcass. Matt called out “Who’s there?” A fooking corpse that’s who! Upon entering there was one of the swine’s that stood us up Oliver Clark  “Hello Pete” he said, hoping I did not bare a grudge or believe in blood feuds “You can fook right off.” I responded.

After being offered apologies, excuses and the sacrifice of Oliver’s first man-child we all settled down to record EP46 of The Dick Turpin Road Show #lol300 (which will be released soon) the main theme running through this recording is Pete’s bitterness at being stood up, Mr Les Pounder by the way. Once we had finished we set off for the event, just past wetherspoons a local lad walked up to us and said “Hey lads wanna buy some champagne?” While Matt pretended to have and engrossing text to read on his phone and Oliver was gripped with a fit of giggles I was left to deal with the damn scally! I told him no thanks but this did not deter him “It’s good stuff mate, normally £50 but I can let you have it for £20″ I replied again that we was not interested and we quickly made our escape.

[This will remain unfinished sadly]

Categories: LUG Community Blogs

Dick Turpin: Translating Customers Comments.

Planet WolvesLUG - Fri, 10/01/2014 - 11:01
"I'm a friend of xyz" = "I want it really cheap or better still for free because I know xyz."

"It says here delivery included, how much is the delivery?" = "I'll come and collect it and save myself the [Delivery cost]"
Categories: LUG Community Blogs

Jono Bacon: Community Leadership Summit 2014 Announced!

Planet WolvesLUG - Fri, 10/01/2014 - 07:10

I am delighted to announce the Community Leadership Summit 2014, now in it’s sixth year! This year it takes place on the 18th and 19th July 2014, the weekend before OSCON at the Oregon Convention Center. Thanks again to O’Reilly for providing the venue.

For those of you who are unfamiliar with the CLS, it is an entirely free event designed to bring together community leaders and managers and the projects and organizations that are interested in growing and empowering a strong community. The event provides an unconference style schedule in which attendees can discuss, debate and explore topics. This is augmented with a range of scheduled talks, panel discussions, networking opportunities and more.

The heart of CLS is an event driven by the attendees, for the attendees.

The event provides an opportunity to bring together the leading minds in the field with new community builders to discuss topics such as governance, creating collaborative environments, conflict resolution, transparency, open infrastructure, social networking, commercial investment in community, engineering vs. marketing approaches to community leadership and much more.

The previous events have been hugely successful and a great way to connect together different people from different community backgrounds to share best practice and make community management an art and science better understood and shared by us all.

I will be providing more details about the event closer to the time, but in the meantime be sure to register!

Categories: LUG Community Blogs

Surrey LUG Bring-A-Box 11th January 2014

Surrey LUG - Thu, 09/01/2014 - 23:25
Start: 2014-01-11 11:00 End: 2014-01-11 17:00

We have regular sessions each month. Bring a 'box', bring a notebook, bring anything that might run Linux, or just bring yourself and enjoy socialising/learning/teaching or simply chilling out!

Our first meeting of 2014 is at the Red Hat offices in Farnborough, Hampshire on Saturday 11th January - thanks to Dominic Cleal for hosting us..

Categories: LUG Community Blogs

Aq: Pretending to type like a Hollywood hacker in Sublime Text 2

Planet WolvesLUG - Thu, 09/01/2014 - 17:34

Christian Heilmann has just drawn my attention to a neat trick for automating typing into a text editor, from William Bamberg at Mozilla. Basically, when you’re doing a screencast, popping up a screen full of code is disorienting and hard for your users to take in, but if you actually type the stuff live on air then everyone gets to see all your typos and your mic makes it sound like a herd of wildebeest sweeping majestically across your keyboard.

Bamberg’s solution is to have an AppleScript which reads the file of your choice and then sends keypresses to your editor to “type” the file in, and it’s a neat idea. However, that’s Mac-specific so I can’t use it, and it doesn’t (as Chris notes) work in Sublime Text 2 (my editor, and his) because ST2 does autoindenting and so on and that sods you up.

Conveniently, I needed a script to do precisely this for some screencasts I’m about to work on, so I thought: I shall write it as an ST2 plugin. And lo, I have done so. It’s only about 30 lines: in ST2, do Tools > New Plugin, then paste the Python from and save it as in the ST2 User folder (which should be default).

You then need a way of running it: I added a keybinding for it in Preferences > Key Bindings -- User so that file now looks like

[ { "keys": ["ctrl+shift+."], "command": "type_file_out" } ]

so I can press ctrl-shift-fullstop.

What it actually does is: when you run it, it removes all the text in the current editing tab, waits two seconds, and then types it back in, character by character. The two second wait is to give you a cut point for the screencast, so you enter or load the code you want into ST2, then start your screencast showing slides or whatever, switch to ST2, then press ctrl-shift-. and it’ll type the text back in. When you’re editing your screencast, cut the part between switching to ST2 and the 2 second break.

There’s probably a way of packaging this up so other people can download it with a click, but I don’t think I know how to do that.

Categories: LUG Community Blogs

Dick Turpin: Free upgrades for life!

Planet WolvesLUG - Thu, 09/01/2014 - 15:47
Me: "You need to make sure you are all running the same version of Office tbh."
Customer: "Yeah but I bet you have to pay for the upgrades?"

Am I really living an an alternate reality?
Categories: LUG Community Blogs

MJ Ray: Request for West Norfolk to Complete the PCC Consultation

Planet ALUG - Thu, 09/01/2014 - 14:19

Please excuse the intrusion to your usual software and co-op news items but vine seems broken and as part of my community and democratic interests, I’d like to share this short clip quoting Norfolk’s Deputy Police Commissioner Jenny McKibben about why Commissioner Stephen Bett believes it’s important to get views from the west of the county about next year’s police budget:

Personally, with a King’s Lynn + West Norfolk Bike Users Group hat on, I’d like it if people supported a 2% (£4/year average) tax increase to reduce the police’s funding cut (the grant from is being cut by 4%) so that we’re less likely to have future cuts to traffic policing. The consultation details and response form are on the PCC website.

Categories: LUG Community Blogs

Jono Bacon: Ask Me Anything on Monday

Planet WolvesLUG - Thu, 09/01/2014 - 02:36

On Monday 13th Jan starting at 6pm UTC (10am Pacific / 1pm Eastern) I will be doing an AMA on Reddit. For those unfamiliar with this – this is where you can ask me anything on Reddit, and the most popular questions and responses are up/down voted.

The post will go live about 30mins before that time so you can start adding questions.

I welcome questions about absolutely anything to do with Ubuntu, Canonical, community management, working in the Open Source industry, writing books, podcasting, free culture, heavy metal, moving from England to America, or anything else. Let’s have some fun!

For those of you who are unfamiliar with my work, I work at Canonical as the Ubuntu Community Manager, I am the author of The Art of Community by O’Reilly, founder of the annual Community Leadership Summit, and have spoken around the world about community leadership and encouraging people to get together to create awesome things.

Outside of my work, I co-founded the Bad Voltage, Lugradio, and Shot Of Jaq podcasts, founded the Creative Commons metal band Severed Fifth, wrote an archive of Creative Commons music, built the BBQ website BBQpad, write for various magazines (Linux Format / Ubuntu User), and have contributed to various Open Source projects.

I will follow up on Twitter/Google+ with a link to the thread when it is published.

Categories: LUG Community Blogs

Steve Kemp: Interesting times

Planet HantsLUG - Wed, 08/01/2014 - 17:04

In November I resigned from Bytemark.

In December I started working for a local company, here in Edinburgh, in a real office (rather than working from home).

Unfortunately today I resigned from that new job, meaning I'm currently unemployed.

I plan to take a 1-2 week vacation, then look for another job as a matter of some urgency. (I can live off savings for the next half-year, or so, if I need to, but I'd go crazy if I had nothing to do for that long.)

It is unfortunate to have to resign from a new job after only five-six weeks, but much more honest to do so now than pretend everything was OK and do it at the point I'd passed my probationary period (of three months).

The people were lovely, the office was lovely, the coffee machine was excellent, the work was interesting, but the nature of a large corporate job with the associated beaurocracy made it a less good fit for me than it looked on paper.

I shall pretend that the next week or two of down-time is our honeymoon ;)

Categories: LUG Community Blogs

Andrew Savory: Multi media

Planet ALUG - Tue, 07/01/2014 - 22:39

My movie collection is a bit of a mishmash, a bunch of different file formats all sat on a Drobo. In the early days I would create AVI, MKV or MP4 rips of my DVDs depending on how and where I wanted to watch them. Sometimes the rips would be split across multiple files. More recently I just copied the DVD wholesale, for conversion later. As a result, ensuring a consistent set of files to copy onto my phone or tablet is a bit of a pain.

With the arrival of the RaspBMC media server, I decided to clean everything up. Some constraints I set:

  • I want to avoid loss of quality from source material (so no re-encoding if possible, only copying).
  • I should be able to do everything from the command line so it can be automated (manipulating video files can be a slow process even without encoding).
  • I want to combine multiple DVDs where possible for easier viewing.
  • My end goal is to have MKV files for most things.
Here’s what I’ve got working so far. Bug fixes and improvements welcome.


AVI files

You can glue AVI files together (concatenate them) and then run mencoder over the joined up file to fix up the indexes:

brew install mplayer cat part1.avi part2.avi > tmp.avi && \ /usr/local/bin/mencoder -forceidx -oac copy -ovc copy tmp.avi -o whole.avi

This forces mencoder to rebuild the index of the avi, which allows players to seek through the file. It encodes with the “copy” audio and video codec, i.e. no encoding, just streamed copying.


MKV files

MKV is Matroska, an open source open standard video container format. The process is similar to AVI files, but the mkvmerge tool does everything for you: 

brew install mkvtoolnix /usr/local/bin/mkvmerge -o whole.mkv part1.mkv +part2.mkv

This takes the two parts and joins them together. Again, no re-encoding, just copying.


DVD rips

I started using RipIt to back up my DVDs; it can automatically encode DVDs, but once I got my Drobo I opted to keep the originals, so I always have the option to re-encode on a case-by-case basis for the target device without losing the best quality original.

I don’t need to touch most of the DVD copies, but a number of my DVDs are split across several disks, for example Starship Troopers and The Lord of the Rings.

One option would be to encode each DVD at the highest possible quality and then merge the AVI or MKV using the mechanisms above, but I want to avoid encoding if possible.

It turns out that the VOB files on a DVD are just MPEG files (see What’s on a DVD? for more details), so there’s no need to convert to AVI or MP4. We can glue them together as we did with the AVIs, then package them as MKV. The basic method is:

cat *.VOB > movie.vob

The problem is that we need to be selective about the VOB files that are included; there’s no point including DVD menu and setup screen animations, for example. A dirty hack might be to select only the VOB files bigger than a certain threshold size, and just hope that the movie is divided into logical chunks. Something like this, run in a movie directory:

find -s . -name '*.VOB' -size +50M

There’s a catch: the first VOB (vts_XX_0.vob) always contains a menu, so we need to skip those, and we don’t want the menu/copyright message (video_ts.vob):

find -s . \( -iname '*.VOB' ! -iname 'VTS_*_0.VOB' ! -iname 'VIDEO_TS.VOB' \) -size +50M 

We can then use ffmpeg to copy the output of find (a list of our VOB files) into an MKV file. So far we’re assuming we only want the first audio stream (usually English), and I haven’t investigated how best to handle subtitles yet. The command is:

ffmpeg -i - -vcodec copy -acodec copy foo.mkv

There’s a couple of issues with this:

So our final command is:

find -s . \( -iname '*.VOB' ! -iname 'VTS_*_0.VOB' ! -iname 'VIDEO_TS.VOB' \) -size +50M -exec cat {} \; \ | ffmpeg -fflags +genpts -i - -f matroska -vcodec copy -acodec copy -c:s copy foo.mkv

The output should be an mkv file roughly the same size as the constituent .dvdmedia directories. You can test it using mkvinfo foo.mkv, which should output information on the mkv file. For some reason, using ‘file foo.mkv’ does not recognise it as an mkv file, only as data.


Putting it all together

Now we know how to handle several individual file formats, we can script the whole process.

The next step is to trawl through a disk full of movies and to normalise them into one format. At this point, we’re well into XKCD territory (The General Problem, and Is It Worth The Time?), so that’s left as an exercise for the reader



Categories: LUG Community Blogs
Syndicate content