Planet ALUG

Syndicate content
Planet ALUG - http://planet.alug.org.uk/
Updated: 1 hour 13 min ago

Mick Morgan: this is what a scary man looks like

Thu, 09/02/2017 - 15:23

No, I mean the one on the right – the one Trump is pointing at.

General John Kelly is just one of Trump’s controversial appointments (and not necessarily the worst) and I guess that by writing this now, I have finally nailed down the lid on the coffin of my ever returning to the US. Pity. I had promised my wife that I would take her to San Francisco in the near future so that she could see for herself why I like it. I’ve visited the USA several times in the past, but only on business and never with my lady. Now it would seem that I cannot go, because I will not submit her, nor myself, to the indignity of being treated like a criminal simply because I wish to enter the country.

Today, El Reg reports that General Kelly has said that he wants the right to demand passwords for social media and financial accounts from some visa applicants so that immigration and homeland securty officers can vet Twitter, Facebook or online banking accounts.

Kelly is reported to have said:

“We want to say ‘what kind of sites do you visit and give us your passwords,’ so we can see what they do. We want to get on their social media with passwords – what do you do, what do you say. If they don’t want to cooperate then they don’t come in. If they truly want to come to America they’ll cooperate, if not then ‘next in line’.”

Now as El Reg points out:

“By “they”, Kelly was referring to refugees and visa applicants from the seven Muslim countries subject to President Trump’s anti-immigration executive order, which was signed last month.”

But it goes on:

“Given the White House’s tough stance on immigration, we can imagine the scope of this “enhanced vetting” creeping from that initial subset to cover visitors of other nationalities. Just simply wait for the president to fall out with another country.”

Or for individuals to draw attention to themselves by being publicly critical of some of the more worrying developments in the USA…..

My own experience of US immigration, even whilst travelling under an A2 Visa, is such that I would most certainly not wish to enter the country if I were to be treated with anything like the hostility I know could be possible. Unfortunately that also means that I might have a problem should I ever wish to fly anywhere else in the world which necessitates a stopover in the US.

The reason I think Kelly may be truly scary? He is reported to have told Representative Kathleen Rice under questioning that:

“I work for one man, his name is Donald Trump, and he told me ‘Kelly, secure the border,’ and that’s what I’m going to do,”

In typical El Reg commentard style, some responders have been less than subtle about this response, evoking obvious references to Godwin’s Law, but one poster, called Jim-234 notes:

“This is a truly stupid plan that is bound to fail on so many levels and will do nothing but upset decent people and open them up to hacking & identity theft while doing nothing to actually stop people who want to cause harm. It reeks of lazy ignorant fools who want to be seen to do something rather than actually do something that works…..

“This is just going to be security theater and bothering everyone and invading their privacy for no net effect at all. As soon as it goes live, all the bad guys will know they need a clean profile online, there will probably even be special paid services to make your online profile all nice and minty fresh, probably even with posting and messaging “good” stuff to make sure you look nice online.”

Jim-234 concludes:

“They want to start demanding your passwords for your phones & laptops?

.. well pretty soon all they will find is factory reset phones, laptops with a never used OS and a new booming business for Chinese, Russian and European data centers of “whole system data backups”.

The only good news is that if this goes live, everyone will probably start scrubbing their Facebook profiles to be about as informative as Zuckerberg’s page… so maybe then Facebook will finally go the way of MySpace.”

Depressingly, I see the same tendency in the UK for security theatre because politicians think “we must be seen to be doing something” in order to make the people feel safer. As the saying goes, “the road to hell is paved with good intentions”.

And what about when the intentions themselves are not good?

Categories: LUG Community Blogs

Jonathan McDowell: GnuK on the Maple Mini

Tue, 07/02/2017 - 18:34

Last weekend, as a result of my addiction to buying random microcontrollers to play with, I received some Maple Minis. I bought the Baite clone direct from AliExpress - so just under £3 each including delivery. Not bad for something that’s USB capable, is based on an ARM and has plenty of IO pins.

I’m not entirely sure what my plan is for the devices, but as a first step I thought I’d look at getting GnuK up and running on it. Only to discover that chopstx already has support for the Maple Mini and it was just a matter of doing a ./configure --vidpid=234b:0000 --target=MAPLE_MINI --enable-factory-reset ; make. I’d hoped to install via the DFU bootloader already on the Mini but ended up making it unhappy so used SWD by following the same steps with OpenOCD as for the FST-01/BusPirate. (SWCLK is D21 and SWDIO is D22 on the Mini). Reset after flashing and the device is detected just fine:

usb 1-1.1: new full-speed USB device number 73 using xhci_hcd usb 1-1.1: New USB device found, idVendor=234b, idProduct=0000 usb 1-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 usb 1-1.1: Product: Gnuk Token usb 1-1.1: Manufacturer: Free Software Initiative of Japan usb 1-1.1: SerialNumber: FSIJ-1.2.3-87155426

And GPG is happy:

$ gpg --card-status Reader ...........: 234B:0000:FSIJ-1.2.3-87155426:0 Application ID ...: D276000124010200FFFE871554260000 Version ..........: 2.0 Manufacturer .....: unmanaged S/N range Serial number ....: 87155426 Name of cardholder: [not set] Language prefs ...: [not set] Sex ..............: unspecified URL of public key : [not set] Login data .......: [not set] Signature PIN ....: forced Key attributes ...: rsa2048 rsa2048 rsa2048 Max. PIN lengths .: 127 127 127 PIN retry counter : 3 3 3 Signature counter : 0 Signature key ....: [none] Encryption key....: [none] Authentication key: [none] General key info..: [none]

While GnuK isn’t the fastest OpenPGP smart card implementation this certainly seems to be one of the cheapest ways to get it up and running. (Plus the fact that chopstx already runs on the Mini provides me with a useful basis for other experimentation.)

Categories: LUG Community Blogs

Chris Lamb: The ChangeLog #237: Reproducible Builds and Secure Software

Sat, 04/02/2017 - 20:39

I recently appeared on the Changelog podcast to talk about the Reproducible Builds project:


Whilst I am an avid podcast listener, this was actually my first appearance on one. It was an curious and somewhat disconcerting feeling to be "just" talking to Adam and Jerod in the moment yet knowing all the time that anything and everything I said would be distributed more widely in the future.

Categories: LUG Community Blogs

Chris Lamb: Free software activities in January 2017

Tue, 31/01/2017 - 08:54

Here is my monthly update covering what I have been doing in the free software world (previous month):

  • Created github-sync, a tool to mirror arbitrary repositories onto GitHub.
  • Submitted two pull requests to the word-wrap Chrome browser extension that adds the ability to wrap text via the right-click context menu:
    • Support dynamically-added <textarea> elements in "rich" Javascript applications such as mail clients, etc. (#2)
    • Avoid an error message if no "editable" has been selected yet. (#1)
  • Submitted a pull request to wordwarvi (a "retro-styled old school side-scrolling shooter") to ensure the build is reproducible. (#5)
  • Filed a pull request with the yard Ruby documentation tool to ensure the generated output is reproducible. (#1048)
  • Made some improvements to travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds on every code change:
    • Merged a pull request from Evgeni Golov to allow for skipped tests. (#39)
    • Add logging when running autopkgtests. (commit)
  • Merged a pull request from jwilk for python-fadvise my Python interface to the posix_fadvise(2) interface to predeclare an pattern for accessing data. (#6)
  • Filed an issue against the redis key-value database regarding build failures on non-x86 architectures. (#3768)
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users.

The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced — either maliciously or accidentally — during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

(I have previously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.)

This month I:

I also made the following changes to our tooling:

diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • Comparators:
    • Display magic file type when we know the file format but can't find file-specific details. (Closes: #850850).
    • Ensure our "APK metadata" file appears first, fixing non-deterministic tests. (998b288)
    • Fix APK extration with absolute filenames. (Closes: #850485).
    • Don't error if directory containing ELF debug symbols already exists. (Closes: #850807).
    • Support comparing .ico files (Closes: #850730).
    • If we don't have a tool (eg. apktool), don't blow up when attempting to unpack it.
  • Output formats:
    • Add Markdown output format. (Closes: #848141).
    • Add RestructuredText output format.
    • Use an optimised indentation routine throughout all presenters.
    • Move text presenter to use the Visitor pattern.
    • Correctly escape value of href="" elements (re. #849411).
  • Tests:
    • Prevent FTBFS by loading fixtures as UTF-8 in case surrounding terminal is not Unicode-aware. (Closes: #852926).
    • Skip tests if binutils can't handle the object file format. (Closes: #851588).
    • Actually compare the output of text/ReST/Markdown formats to fixtures.
    • Add tests for: Comparing two empty directories, HTML output, image.ICOImageFile, --html-dir, --text-color & no arguments (beyond the filenames) emits the text output.
  • Profiling:
    • Count the number of calls, not just the total time.
    • Skip as much profiling overhead when not enabled for a ~2% speedup.
  • Misc:
    • Alias an expensive Config() lookup for a 10% optimisation.
    • Avoid expensive regex creation until we actually need it, speeding up diff parsing by 2X.
    • Use Pythonic logging functions based on __name__, etc.
    • Drop milliseconds from logging output.

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Store files directly onto S3.
  • Drop big unique_together index to save disk space.
  • Show SHA256 checksums where space permits.

Debian LTS

This month I have been paid to work 12.75 hours on Debian Long Term Support (LTS). In that time I did the following:

  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 773-1 for python-crypto fixing a vulnerability where calling AES.new with an invalid parameter could crash the Python interpreter.
  • Issued DLA 777-1 for libvncserver addressing two heap-based buffer overflow attacks based on invalid FramebufferUpdate data.
  • Issued DLA 778-1 for pcsc-lite correcting a use-after-free vulnerability.
  • Issued DLA 795-1 for hesiod which fixed a weak SUID check as well as removed the hard-coding of a fallback domain if the configuration file could not be found.
  • Issued DLA 810-1 for libarchive fixing a heap buffer overflow.
Uploads
  • python-django:
    • 1:1.10.5-1 — New upstream stable release.
    • 1:1.11~alpha1-1 — New upstream experimental release.
  • gunicorn (19.6.0-10) — Moved debian/README.Debian to debian/NEWS so that the recent important changes will be displayed to users when upgrading to stretch.
  • redis:
    • 3:3.2.6-2 & 4:4.0-rc2-2 — Tidy patches and rename RunTimeDirectory to RuntimeDirectory in .service files. (Closes: #850534)
    • 3:3.2.6-3 — Remove a duplicate redis-server binary by symlinking /usr/bin/redis-check-rdb. This was found by the dedup service.
    • 3:3.2.6-4 — Expand the documentation in redis-server.service and redis-sentinel.service regarding the default hardening options and how, in most installations, they can be increased.
    • 3:3.2.6-5, 3:3.2.6-6, 4:4.0-rc2-3 & 4:4.0-rc2-4 — Add taskset calls to try and avoid build failures due to parallelism in upstream test suite.

I also made the following non-maintainer uploads:

  • cpio:
    • 2.12+dfsg-1 — New upstream release (to experimental), refreshing all patches, etc.
    • 2.12+dfsg-2 — Add missing autoconf to Build-Depends.
  • xjump (2.7.5-6.2) — Make the build reproducible by passing -n to gzip calls in debian/rules. (Closes: #777354)
  • magicfilter (1.2-64.1) — Make the build reproducible by passing -n to gzip calls in debian/rules. (Closes: #777478)
Debian bugs filed RC bugs

I also filed 16 FTBFS bugs against bzr-git, coq-highschoolgeometry, eclipse-anyedit, eclipse-gef, libmojolicious-plugin-assetpack-perl, lua-curl, node-liftoff, node-liftoff, octave-msh, pcb2gcode, qtile, rt-authen-externalauth, ruby-hamster, ruby-sshkit, tika & txfixtures.

FTP Team

As a Debian FTP assistant I ACCEPTed 35 packages: chromium-browser, debichem, flask-limiter, golang-github-golang-leveldb, golang-github-nebulouslabs-demotemutex, golang-github-nwidger-jsoncolor, libatteanx-endpoint-perl, libproc-guard-perl, libsub-quote-perl, libtest-mojibake-perl, libytnef, linux, lua-sql, node-graceful-readlink, node-invariant, node-rollup, node-socket.io-parser, node-timed-out, olefile, packaging-tutorial, pgrouting, pyparallel, python-coards, python-django-tagging, python-graphviz, python-irc, python-mechanicalsoup, python-persistent, python-scandir, python-stopit, r-cran-zelig, ruby-ast, ruby-whitequark-parser, sagetex & u-boot-menu.

Categories: LUG Community Blogs

Mick Morgan: variable substitution – redux

Mon, 30/01/2017 - 14:40

Back in October last year, I posted a note about the usage of variable substitution in lighttpd’s configuration files. In fact I got that post very slightly wrong (now corrected) in that I showed the test I applied in the file as: “$HTTP[“remoteip”] !~ “12.34.56.78″”. (Note the “!~” when I should have used “!=”). This works, in that it would limit access, but it is subtly wrong because it does not limit access in quite the way I intended. I only noticed this when I later came to change the variable assignment to allow access from three separate IP addresses (on which more later) rather than just one.

The “!~” operator is a perl style regular expression “not” match whilst the “!=” operator is the more strict string not equal match. This matters. My construct using the perl regex not wouldn’t actually just limit access solely to remote address 12.34.56.78 but would also allow in addresses of the form n12n.n34n.n56n.n78n where “n” is any other valid numeral (or none). So for example, my construct would have allowed in connections from 125.134.56.178 or 212.34.156.78 or 121.34.156.78 etc. That is not what I wanted at all.

The (correct) assignment and test now looks like this:

var.IP = “12\.34\.56\.78|23\.45\.67\.78|34\.56\.78\.90”

$HTTP[“remoteip”] !~ var.IP {
$HTTP[“url”] =~ “^/wp-admin/” {
url.access-deny = (“”)
}

Which says, allow connections from address 12.34.56.78 or 23.45.67.89 or 34.56.78.90 but no others.

For reference, the BNF like notation used in the basic configuration for lighty is given on the redmine wiki.

Categories: LUG Community Blogs

Mick Morgan: not welcome here

Mon, 30/01/2017 - 13:45

US President Trump has said that refugees and travellers from seven, mainly majority Muslim, countries are barred from entry to the US. Notwithstanding our own dear PM’s invitation to Trump, some 1.2 million brits have so far signed a Parliamentary petition to “Prevent Donald Trump from making a State Visit to the United Kingdom“.

I do not actually agree with the wording of the petition. As a republican at heart I really don’t care whether the Queen would be “embarrassed” by meeting Trump. Besides, she has met many, arguably worse, leaders in her time (think Robert Mugabe, or Nicolae Ceaușescu). The point here is that to extend an invitation to Trump so quickly, and whilst he is advocating such distateful and divisive policies gives the distinct impression that the UK endorses those policies. We do not, and should be seen to be highly critical of those policies. In her visit to the US last week, Theresa May made much of the UK’s ability as a close friend and partner of the USA to feel free to criticise that partner. She has done no such thing. In my view that is shameful.

When I signed the petition there were around 600,000 other signatories. The total is still climbing. That is encouraging. No doubt the petition will be ignored though.

(Postscript. El Reg has a discussion raging about the petition. Apparently I am a “virtue signaller”. Oh well, I don’t feel bad about it. After all, I’m a blogger so by definition I’m already self interested and vain.)

Categories: LUG Community Blogs

Jonathan McDowell: BelFOSS 2017

Sun, 29/01/2017 - 22:18

On Friday I attended the second BelFOSS conference. I’d spoken about my involvement with Debian at the conference last year, which seemed to be well received. This year I’d planned to just be a normal attendee, but ended up roped in at a late stage to be part of a panel discussing various licensing issues. I had a thoroughly enjoyable day - there were many great speakers, and plenty of opportunity for interesting chats with other attendees.

The conference largely happens through the tireless efforts of Jonny McCullagh, though of course many people are involved in bringing it together. It’s a low budget single day conference which has still managed to fill its single track attendee capacity both years, and attract more than enough speakers. Last year Red Hat and LPI turned up, this year Matt Curry from Allstate’s Arizona office appeared, but in general it’s local speakers talking to a local audience. This is really good to see - I don’t think Jonny would object at all if he managed to score a `big name’ speaker, but one of his aims is to get students interested and aware of Free Software, and I think it helps a lot that the conference allows them to see that it’s actively in use in lots of aspects of the industry here in Northern Ireland.

Here’s hoping that BelFOSS becomes an annual fixture in the NI tech calendar!

Categories: LUG Community Blogs

Jonathan McDowell: Experiments with 1-Wire

Tue, 24/01/2017 - 20:49

As previously mentioned, at the end of last year I got involved with a project involving the use of 1-Wire. In particular a DS28E15 device, intended to be used as a royalty tracker for a licensed piece of hardware IP. I’d no previous experience with 1-Wire (other than knowing it’s commonly used for driving temperature sensors), so I took it as an opportunity to learn a bit more about it.

The primary goal was to program a suitable shared key into the DS28E15 device that would also be present in the corresponding hardware device. A Maxim programmer had been ordered, but wasn’t available in stock so had to be back ordered. Of course I turned to my trusty Bus Pirate, which claimed 1-Wire support. However it failed to recognise the presence of the device at all. After much head scratching I finally listened to a co-worker who had suggested it was a clock speed issue - the absence of any option to select the 1-Wire speed in the Bus Pirate or any mention of different speeds in the documentation I had read had made me doubt it was an issue. Turns out that the Bus Pirate was talking “standard” 1-Wire and the DS28E15 only talks “overdrive” 1-Wire, to the extent that it won’t even announce its presence if the reset pulse conforms to the standard, rather than overdrive, reset time period. Lesson learned: listen to your co-workers sooner.

A brief period of yak shaving led to adding support to the Bus Pirate for the overdrive mode (since landed in upstream), and resulted in a search request via the BP interface correctly finding the device and displaying its ROM ID. This allowed exploration of the various commands the authenticator supports, to verify that the programming sequence operated as expected. These allow for setting the shared secret, performing a SHA256 MAC against this secret and a suitable nonce, and retrieving the result.

Next problem: the retrieved SHA256 MAC did not match the locally computed value. Initially endianness issues were suspected, but trying the relevant permutations did not help. Some searching found an implementation of SHA256 for the DS28E15 that showed differences between a standard SHA256 computation and what the authenticator performs. In particular SHA256 normally adds the current working state (a-g) to the current hash value (h0-h7) at the end of every block. The authenticator does this for all but the final block, where instead the hash value is set to the working state. I haven’t been able to find any documentation from Maxim that this is how things are calculated, nor have I seen any generic implementation of SHA256 which supports this mode. However rolling my own C implementation based on the code I found and using it to compare the results retrieved from the device confirms that this is what’s happening.

So at this point we’re done, right? Wait for the proper programming hardware to turn up, write the key to the devices, profit? Well, no. There was a bit of a saga involving the programmer (actually programmers, one with at least some documentation that allowed the creation of a Python tool to allow setting the key and reading + recording the ROM ID for tracking, and one with no programming documentation that came with a fancy GUI for manually doing the programming), but more importantly it was necessary to confirm that the programmed device interacted with the hardware correctly.

Initial testing with the hardware was unsuccessful. Again endianness issues were considered and permutations tried, but without success. A simple key constructed to avoid such issues was tried, and things worked fine. There was a hardware simulation of both components available, so it was decided to run that and obtain a capture of the traffic between them. As the secret key was known this would then allow the random nonce to be captured, and the corresponding (correct) hash value. Tests could then be performed in software to determine what the issue was & how to generate the same hash for verification.

Two sets of analyzer software were tried, OpenBench LogicSniffer (OLS) and sigrok. As it happened both failed to correctly decode the bitstream detected as 1-Wire, but were able to show the captured data graphically, allowing for decoding by eye. A slight patch to OLS to relax the timing constraints allowed it to successfully decode the full capture and provided the appropriate data for software reproduction. The end issue? A 256 bit number (as defined in VHDL) is not the same as 32 element byte array… Obvious when you know what the issue is!

So? What did I learn, other than a lot about 1-Wire? Firstly, don’t offhandedly discount suggestions that you don’t think make sense. Secondly, having a tool (in this case the Bus Pirate) that lets you easily play with a protocol via a simple interface is invaluable in understanding it. Thirdly, don’t trust manufacturers to be doing something in a normal fashion when they claim to be using a well defined technology. Fourthly, be conscious about all of the different ways bitstreams can be actually processed in memory. It’s not just endianness. Finally, spending the time to actually understand what’s going on up front can really help when things don’t work as you’d expect later on - without the yak shaving to support Overdrive on the BP I wouldn’t have been able to so quickly use the simulation capture to help diagnose the issue.

Categories: LUG Community Blogs

Steve Engledow (stilvoid): Angst

Sun, 22/01/2017 - 02:06

I had planned to spend this evening playing games; something I really enjoy doing but rarely set aside any time for. However, while we were eating dinner, I put some music on and it got me in the mood for playing some guitar. Over the course of dinner and playing with my son afterwards, that developed into wanting to write and record some music. I used to write electronic nonsense sometimes but this evening, I fancied trying my hand at some metal.

The first 90 minutes was - as almost every time I get the rare combination of an urge to do something musical and time to do it in - spent trying to remember how my setup worked, which bits of software I needed to install, and how to get the right combination of inputs and outputs I want. I eventually got it sussed and decided I'd better write it down for my own future reference.

Hardware
  1. Plug the USB audio interface from the V-Amp3 into the laptop.
  2. Plug external audio sources into the audio interface's input. (e.g. the V-Amp or a synth).
  3. Plug some headphones into the headphone socket of the audio interface.
  4. Switch on the audio interface's monitoring mode ;) (this kept me going for a little while; it's a small switch)
Software
  1. The following packages need to be installed at a minimum:

    • qjackctl
    • qsynth
    • soundfont-fluidsynth
    • vkeybd
    • ardour
    • hydrogen
  2. Use pavucontrol or similar to disable the normal audio system and just use the USB audio interface.

  3. Qjackctl needs the following snippets in its config for when jack comes up and goes down, respectively:

    • pacmd suspend true

      This halts pulseaudio so that jack can take over

    • pacmd suspend false

      This starts puseaudio back up again

  4. Use the connection tool in Jack to hook hydrogen's and qsynth's outputs to ardour's input. Use the ALSA tab to connect vkeybd to qsynth.

  5. When starting Ardour and Hydrogen, make sure they're both configured to use Jack for MIDI. Switch Ardour's clock from Internal to JACK.

For posterity, here's this evening's output.

Categories: LUG Community Blogs

Jonathan McDowell: Cloning a USB LED device

Sat, 14/01/2017 - 11:53

A month or so ago I got involved in a discussion on IRC about notification methods for a headless NAS. One of the options considered was some sort of USB attached LED. DealExtreme had a cheap “Webmail notifier”, which was already supported by mainline kernels as a “Riso Kagaku” device but it had been sold out for some time.

This seemed like a fun problem to solve with a tinyAVR and V-USB. I had my USB relay board so I figured I could use that to at least get some code to the point that the kernel detected it as the right device, and the relay output could be configured as one of the colours to ensure it was being driven in roughly the right manner. The lack of a full lsusb dump (at least when I started out) made things a bit harder, plus the fact that the Riso uses an output report unlike the relay code, which uses a control message. However I had the kernel source for the driver and with a little bit of experimentation had something which would cause the driver to be loaded and the appropriate files in /sys/class/leds/ to be created. The relay was then successfully activated when the red LED was supposed to be on.

hid-led 0003:1294:1320.0001: hidraw0: USB HID v1.01 Device [MAIL MAIL ] on usb-0000:00:14.0-6.2/input0 hid-led 0003:1294:1320.0001: Riso Kagaku Webmail Notifier initialized

I subsequently ordered some Digispark clones and modified the code to reflect the pins there (my relay board used pins 1+2 for USB, the Digispark uses pins 3+4). I then soldered a tricolour LED to the board, plugged it in and had a clone of the Riso Kaguku device for about £1.50 in parts (no doubt much cheaper in bulk). Very chuffed.

In case it’s useful to someone, the code is released under GPLv3+ and is available at https://the.earth.li/gitweb/?p=riso-kagaku-clone.git;a=summary or on GitHub at https://github.com/u1f35c/riso-kagaku-clone. I’m seeing occasional issues on an older Dell machine that only does USB2 with enumeration, but it generally is fine once it gets over that.

(FWIW, Jon, who started the original discussion, ended up with a BlinkStick Nano which is a neater device with 2 LEDs but still based on an Tiny85.)

Categories: LUG Community Blogs

Jonathan McDowell: 2016 in 50 Words

Fri, 06/01/2017 - 08:03

Idea via Roger. Roughly chronological order. Some things were obvious inclusions but it was interesting to go back and look at the year to get to the full 50 words.

Speaking at BelFOSS. Earthlings birthday. ATtiny hacking. Speaking at ISCTSJ. Dublin Anomaly. Co-habiting. DebConf. Peak Lion. Laura’s wedding. Christmas + picnic. Engagement. Car accident. Car write off. Tennent’s Vital. Dissertation. OMGWTFBBQ. BSides. New job. Rachel’s wedding. Digital Privacy talk. Graduation. All The Christmas Dinners. IMDB Top 250. Shay leaving drinks.

(This also serves as a test to see if I’ve correctly updated Planet Debian to use https and my new Hackergotchi that at least looks a bit more like I currently do.)

Categories: LUG Community Blogs

Jonathan McDowell: IMDB Top 250: Complete. Sort of.

Sat, 31/12/2016 - 16:01

Back in 2010, inspired by Juliet, I set about doing 101 things in 1001 days. I had various levels of success, but one of the things I did complete was the aim of watching half of the IMDB Top 250. I didn’t stop at that point, but continued to work through it at a much slower pace until I realised that through the Queen’s library I had access to quite a few DVDs of things I was missing, and that it was perfectly possible to complete the list by the end of 2016. So I did.

I should point out that I didn’t set out to watch the list because I’m some massive film buff. It was more a mixture of watching things that I wouldn’t otherwise choose to, and also watching things I knew were providing cultural underpinnings to films I had already watched and enjoyed. That said, people have asked for some sort of write up when I was done. So here are some random observations, which are almost certainly not what they were looking for.

My favourite film is not in the Top 250

First question anyone asks is “What’s your favourite film?”. That depends a lot on what I’m in the mood for really, but fairly consistently my answer is The Hunt for Red October. This has never been in the Top 250 that I’ve noticed. Which either says a lot about my taste in films, or the Top 250, or both. Das Boot was in the list and I would highly recommend it (but then I like all submarine movies it seems).

The Shawshank Redemption is overrated

I can’t recall a time when The Shawshank Redemption was not top of the list. It’s a good film, and I’ve watched it many times, but I don’t think it’s good enough to justify its seemingly unbroken run. I don’t have a suggestion for a replacement, however.

The list is constantly changing

I say I’ve completed the Top 250, but that’s working from a snapshot I took back in 2010. Today the site is telling me I’ve watched 215 of the current list. Last night it was 214 and I haven’t watched anything in between. Some of those are films released since 2010 (in particular new releases often enter high and then fall out of the list over a month or two), but the current list has films as old as 1928 (The Passion of Joan of Arc) that weren’t there back in 2010. So keeping up to date is not simply a matter of watching new releases.

The best way to watch the list is terrestrial TV

There were various methods I used to watch the list. Some I’d seen in the cinema when they came out (or was able to catch that way anyway - the QFT showed Duck Soup, for example). Netflix and Amazon Video had some films, but overall a very disappointing percentage. The QUB Library, as previously mentioned, had a good number of DVDs on the list (especially the older things). I ended up buying a few (Dial M for Murder on 3D Bluray was well worth it; it’s beautifully shot and unobtrusively 3D), borrowed a few from friends and ended up finishing off the list by a Lovefilm one month free trial. The single best source, however, was UK terrestrial TV. Over the past 6 years Freeview (the free-to-air service here) had the highest percentage of the list available. Of course this requires some degree of organisation to make sure you don’t miss things.

Films I enjoyed

Not necessarily my favourite, but things I wouldn’t have necessarily watched and was pleasantly surprised by. No particular order, and I’m leaving out a lot of films I really enjoyed but would have got around to watching anyway.

  • Clint Eastwood films - Gran Torino and Million Dollar Baby were both excellent but neither would have appealed to me at first glance. I hated Unforgiven though.
  • Jimmy Stewart. I’m not a fan of It’s a Wonderful Life (which I’d already watched because it’s Lister’s favourite film), but Harvey is obviously the basis of lots of imaginary friend movies and Rear Window explained a Simpsons episode (there were a lot of Simpsons episodes explained by watching the list).
  • Spaghetti Westerns. I wouldn’t have thought they were my thing, but I really enjoyed the Sergio Leone films (A Fistful of Dollars etc.). You can see where Tarantino gets a lot of his inspiration.
  • Foreign language films. I wouldn’t normally seek these out. And in general it seems I cannot get on with Italian films (except Life is Beautiful), but Amores Perros, Amelie and Ikiru were all better than expected.
  • Kind Hearts and Coronets. For some reason I didn’t watch this until almost the end; I think the title always put me off. Turned out to be very enjoyable.
Films I didn’t enjoy

I’m sure these mark me out as not being a film buff, but there are various things I would have turned off if I’d caught them by accident rather than setting out to watch them.

I’ve kept the full list available, if you’re curious.

Categories: LUG Community Blogs

Chris Lamb: Free software activities in December 2016

Sat, 31/12/2016 - 10:40

Here is my monthly update covering what I have been doing in the free software world (previous month):

  • Celebrated my 10-year anniversary of contributing to Debian. An excerpt of this post was quoted on LWN.
  • Made a number of improvements to AptFS, my FUSE-based filesystem that provides a view on unpacked Debian source packages as regular folders, including move from the popen2 Python module to subprocess and correcting the parsing of package lists.
  • Corrected an UnboundLocalError exception in the Finnish social security number generator in faker, a tool to generate test data in Python applications. (#441)
  • Made a small change to travis.debian.net (my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds on every code change) to fix an issue with malformed YAML.
  • Added the ability to specify the clone target to gbp-import-dsc etc. in git-buildpackage, a tool to build Debian packages using Git. (commit)
  • Filed three issues against the Redis key-value database:
    • Tests fail on the alpha architecture due to "memory efficiency". (#3666)
    • Please update hiredis (#3687)
    • Correct "whenever" typo. (#3652)
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users.

The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced — either maliciously or accidentally — during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

This month:

I also made the following changes to our tooling:

diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • Optimisations:
    • Avoid unnecessary string manipulation writing --text output (~20x speedup).
    • Avoid n iterations over archive files (~8x speedup).
    • Don't analyse .deb s twice when comparing .changes files (2x speedup).
    • Avoid shelling out to colordiff by implementing color support directly.
    • Memoize calls to distutils.spawn.find_executable to avoid excessive stat(1) syscalls.
  • Progress bar:
    • Show current file / ELF section under analysis etc. in progress bar.
    • Move the --status-fd output to use JSON and to include the current filename.
  • Code tidying:
    • Split out the try.diffoscope.org client so that it can be released separately on PyPI.
    • Completely rework the diffoscope and diffoscope.comparators modules, grouping similar utilities into their own modules, etc.
  • Miscellaneous:
    • Update dex_expected_diffs test to ensure compatibility with enjarify ≥ 1.0.3.
    • Ensure that running from Git will always use that checkout's Python modules.
    • Add a simple profiling framework.

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Makefile.PL: Change NAME argument to a Perl package name.
  • Ensure our binaries are available in autopkgtest tests.

try.diffoscope.org

trydiffoscope is a web-based version of the diffoscope in-depth and content-aware diff utility. Continued thanks to Bytemark for sponsoring the hardware.

  • Show progress bar and position in queue, etc. (#25 & #26)
  • Promote command-line client with PyPI instructions.
  • Increase comparison time limit to 90 seconds.

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Added support for version 0.2 .buildinfo files. (#15)

Debian Debian LTS

This month I have been paid to work 13½ hours on Debian Long Term Support (LTS). In that time I did the following:

  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 733-1 for openafs, fixing an information leak vulnerability. Due to incomplete initialization or clearing of reused memory, directory objects could contain 'dead' directory entry information.
  • Issued DLA 734-1 for mapserver closing an information leakage vulnerability.
  • Issued DLA 737-1 for roundcube preventing arbitrary remote code execution by sending a specially crafted email.
  • Issued DLA 738-1 for spip patching a cross-site scripting (XSS) vulnerability.
  • Issued DLA 740-1 for libgsf fixing a null pointer deference exploit via a crafted .tar file.
Debian Uploads
  • redis:
    • 3.2.5-5 — Add RunTimeDirectory=redis to systemd .service files.
    • 3.2.5-6 — Add missing Depends on lsb-base for /lib/lsb/init-functions usage in redis-sentinel's initscript.
    • 3.2.6-1 — New upstream release.
    • 4.0-1 & 4.0-rc2-1 — New upstream experimental releases.
  • aptfs: 0.9-1 & 0.10-1 — New upstream releases.
Debian bugs filed

I filed 29 FTBFS bugs against a7xpg, conntrack-tools, factory-boy, faker, glimpse, gunroar, hexchat-otr, jackson-datatype-guava, jalview, jquery, kodi-pvr-mythtv, leap-cli, libbio-graphics-perl, libparanoid-perl, libsass-python, metastudent-data, node-temporary, node-yargs, python-requests-unixsocket, python-restless, ruby-bunny, ruby-github-markup, ruby-rabl, sagenb-export, seaborn, soapdenovo2, titanion, ufw & vagrant-cachier.

I additionally filed 2 bugs for packages that access the internet during build against fence-agents & lua-geoip.

Debian FTP Team

As a Debian FTP assistant I ACCEPTed 107 packages: android-platform-libcore, compiz, debian-edu, dehydrated, dh-cargo, gnome-shell-extension-pixelsaver, golang-1.8, golang-github-btcsuite-btcd-btcec, golang-github-elithrar-simple-scrypt, golang-github-pelletier-go-toml, golang-github-restic-chunker, golang-github-weaveworks-mesh, golang-google-genproto, igmpproxy, jimfs, kpmcore, libbio-coordinate-perl, libdata-treedumper-oo-perl, libdate-holidays-de-perl, libpgobject-type-bytestring-perl, libspecio-library-path-tiny-perl, libterm-table-perl, libtext-hogan-perl, lighttpd, linux, linux-signed, llmnrd, lua-geoip, lua-sandbox-extensions, lua-systemd, node-cli-cursor, node-command-join, node-death, node-detect-indent, node-domhandler, node-duplexify, node-end-of-stream, node-first-chunk-stream, node-from2, node-glob-stream, node-has-binary, node-inquirer, node-interpret, node-is-negated-glob, node-is-unc-path, node-lazy-debug-legacy, node-lazystream, node-load-grunt-tasks, node-merge-stream, node-object-assign-sorted, node-orchestrator, node-pkg-up, node-resolve-from, node-resolve-pkg, node-rx, node-sorted-object, node-stream-shift, node-streamtest, node-string.prototype.codepointat, node-strip-bom-stream, node-through2-filter, node-to-absolute-glob, node-unc-path-regex, node-vinyl, openzwave, openzwave-controlpanel, pcb-rnd, pd-upp, pg-partman, postgresql-common, pybigwig, python-acora, python-cartopy, python-codegen, python-efilter, python-flask-sockets, python-intervaltree, python-jsbeautifier, python-portpicker, python-pretty-yaml, python-protobix, python-sigmavirus24-urltemplate, python-sqlsoup, python-tinycss, python-watson-developer-cloud, python-zc.customdoctests, python-zeep, r-cran-dbitest, r-cran-dynlm, r-cran-mcmcpack, r-cran-memoise, r-cran-modelmetrics, r-cran-plogr, r-cran-prettyunits, r-cran-progress, r-cran-withr, ruby-clean-test, ruby-gli, ruby-json-pure, ruby-parallel, rustc, sagemath, sbuild, scram, sidedoor, toolz & yabasic.

I additionally filed 4 RC bugs against packages that had incomplete debian/copyright files against jimfs, compiz, python-efilter & ruby-json-pure.

Categories: LUG Community Blogs

Chris Lamb: My favourite books of 2016

Fri, 30/12/2016 - 18:51

Whilst I managed to read almost sixty books in 2016 here are ten of my favourites in no particular order.

Disappointments this year include Stewart Lee's Content Provider (nothing like his stand-up), Christopher Hitchens' And Yet (his best essays are already published) and Heinlein's Stranger in a Strange Land (great exposition, bizarre conclusion).

The worst book I finished, by far, was Mark Edward's Follow You Home.






Animal QC

Gary Bell, QC

Subtitled My Preposterous Life, this rags-to-riches story about a working-class boy turned eminent lawyer would be highly readable as a dry and factual account but I am compelled to include it here for its extremely entertaining style of writing.

Full of unsurprising quotes that take one unaware: would you really expect a now-Queen's Counsel to "heartily suggest that if you find yourself suffering from dysentery in foreign climes you do not medicate it with lobster thermidor and a bottle of Ecuadorian red?"

A real good yarn.

So You've Been Publically Shamed

Jon Ronson

The author was initially recommended to me by Brad but I believe I started out with the wrong book. In fact, I even had my doubts about this one, prematurely judging from the title that it was merely cashing-in on a fairly recent internet phenomenon — like his more recent shallow take on Trump and the alt-Right — but in the end I read Publically Shamed thrice in quick succession.

I would particularly endorse the audiobook version: Ronson's deadpan drawl suits his writing perfectly.

The Obstacle is the Way

Ryan Holiday

Whilst everyone else appears to be obligated to include Ryan's recent Ego is the Enemy in their Best of 2016 lists I was actually taken by his earlier "introduction by stealth" to stoic philosophy.

Certainly not your typical self-help book, this is "a manual to turn to in troubling times".

Returning to this work at least three times over the year — even splashing out on the audiobook at some point — I feel like I learned a great deal, although it is now difficult to pinpoint exactly what. Perhaps another read in 2017 is thus in order…

Layer Cake

J.J. Connolly

To judge a book in comparison to the film is to do both a disservice, but reading the book of Layer Cake really underscored just how well the film played to the strengths of that medium.

All of the aspects that would not have worked had been carefully excised from the screenplay, ironically leaving more rewarding "layers" for readers attempting the book. A parallel adaption here might be No Country for Old Men - I would love to read (or write) a comparative essay between these two adaptions although McCarthy's novel is certainly the superior source material.

Lying

Sam Harris

I've absorbed a lot of Sam Harris's œuvre this year in the form of his books but moreover via his compelling podcast. I'm especially fond of Waking Up on spirituality without religion and would rank that as my favourite work of his.

Lying is a comparatively short read, more of a long essay in fact, where he argues that we can radically simplify our lives by merely telling the truth in situations where others invariably lie. Whilst it would take a brave soul to adopt his approach his case is superlatively well-argued and a delight to read.

Letters from a Stoic

Seneca

Great pleasure is to be found not only in keeping up an old and established friendship but also in beginning and building up a new one.

Reading this in a beautifully svelte hardback, I tackled a randomly-chosen letter per day rather than attempting to read it cover-to-cover. Breaking with a life-long tradition, I even decided to highlight sections in pen so I could return to them at ease.

I hope it's not too hackneyed to claim I gained a lot from "building up" a relationship with this book. Alas, it is one of those books that is too easy to recommend given that it might make one appear wise and learned, but if you find yourself in a slump, either in life or in your reading habits, it certainly has my approval.

Solo: A James Bond Novel

William Boyd

I must have read all of the canonical Fleming novels as a teenager and Solo really rewards anyone who has done so. It would certainly punish anyone expecting a Goldeneye or at least be a little too foreign to be enjoyed.

Indeed, its really a pastiche of these originals, both in terms of the time period, general tone (Bond is more somber; more vulnerable) and in various obsessions of Fleming's writing, such as the overly-detailed description of the gambling and dining tables. In this universe, 007's restaurant expenses probably contributed signifcantly to the downfall of the British Empire, let alone his waistline.

Bond flicking through a ornithological book at one point was a cute touch…

The Subtle Art of Not Giving A F*ck

Mark Manson

Certainly a wildcard to include here and not without its problems, The Subtle Art… is a curious manifesto on how to approach life. Whilst Manson expouses an age-old philosophy of grounding yourself and ignoring the accumulation of flatscreen TVs, etc. he manages to do so in a fresh and provocative "21st-centry gonzo" style.

Highly entertaining, at one point the author posits an alternative superhero ("Disappointment Panda") that dishes out unsolicited and uncomfortable truths to strangers before simply walking away: "You know, if you make more money, that’s not going to make your kids love you," or: "What you consider friendship is really just your constant attempts to impress people."

Ouch.

The Fourth Protocol

Frederick Forsyth

I have a crystal-clear memory from my childhood of watching a single scene from a film in the dead of night: Pierce Brosnan sets a nuclear device to detonate after he can get away but a double-crossing accomplice surreptitiously brings the timetable forward in order that the bomb also disposes of him…

Anyway, at some point whilst reading The Fourth Protocol it dawned on me that this was that book. I might thus be giving the book more credit due to this highly satisfying connection but I think it stands alone as a superlative political page-turner and is still approachable outside the machinations of the Cold War.

The Partner

John Grisham

After indulging in a bit too much non-fiction and an aborted attempt at The Ministry of Fear, I turned to a few so-called lower-brow writers such as Jeffrey Archer, etc.

However, it was The Partner that turned out to be a real page-turner for somewhat undefinable reasons. Alas, it appears the rest of the author's output is unfortunately in the same vein (laywers, etc.) so I am hesitant to immediately begin others but judging from various lists online I am glad I approached this one first.

Shogun: The First Novel of the Asian saga

James Clavell

Despite its length, I simply couldn't resist returning to Shogun this year although it did fatigue me to the point that I have still yet to commence on its sequel, Tai-Pan.

Like any good musical composition, one is always rewarded by returning to a book and I took great delight in uncovering more symbolism throughout (such as noticing that one of the first words Blackthorne learns in Japanese is "truth") but also really savouring the tragic arcs that run throughout the novel, some beautiful phrases ("The day seemed to lose its warmth…") and its wistful themes of inevitability and karma.

Categories: LUG Community Blogs

Jonathan McDowell: The terrible PIC ecosystem

Sun, 25/12/2016 - 00:51

I recently had call to play with some 1-Wire devices at work (more of which in a future post). It was taking a while for the appropriate programmer to turn up, so of course I pulled out my trusty BusPirate. It turned out the devices in question would only talk in overdrive mode, while the Bus Pirate could only offer standard mode. So I set about trying to figure out how to add the appropriate support.

This is is a huge endorsement for test equipment with Free Software firmware. Rather than giving up I was able to go and grab the current firmware, which has been adopted by the community since Dangerous Prototypes have discontinued development. What let me down was the ecosystem around the PIC24FJ64GA002.

My previous recent experience with microcontrollers has been with the ATTiny range and the STM32. Getting up and running with both of these was fairly easy - the tool chains necessary were already present in Debian, so all it took was a simple apt invocation to install everything I needed to compile code and program it to the devices.

Not so with the PIC series, which surprised me. There seems to be some basic support for the earlier PIC16 range, but for later chips there’s nothing that works out of the box with Debian. Investigation revealed that this was because there’s nothing maintained that enabled Free development for the PIC range. The accepted solution is the closed MPLAB X. Now, in one sense fair play to Microchip for making this available. But in another, shame on you. I can’t imagine ever choosing to build something based on a chip that only had a closed source tool chain available. I want things I can use in Makefiles and properly script, that are available in my distro of choice and that generally work in the same fashion as the tool chains I’m used to. I understand there might be some benefit in a closed compiler in terms of performance (and have HPC friends who would never trust a benchmark provided using GCC), but in general that’s not the space I move in. Nor does it seem to be the sort of attitude you should be taking if you are trying to attract the hobbyist and small production run market.

Any yet this seems common amongst hardware manufacturers. People whose core business is selling physical items, where the software is only relevant in terms of being able to use those items, seem to consider the software to be precious. Instead of opening up programming specifications and allowing a more widespread use of the hardware, increasing sales. I understand there are some cases where this isn’t practical, but the default attitude is definitely one of being closed rather than open, which is a terrible shame.

Anyway. I do have some Bus Pirate 1-Wire overdrive support now working (pending some testing to ensure standard mode still works), but I am glad I never spent a lot of time getting involved with PICs now.

Categories: LUG Community Blogs

Mick Morgan: Merry Christmas 2016

Sat, 24/12/2016 - 19:08

As is now traditional :-) I post today to wish everyone a very merry christmas.

Today is trivia’s birthday – indeed it is trivia’s 10th birthday so I have been writing here for a decade. Good grief. If I had known then what I know now trivia might have been still born. As it is we are both still here – more importantly so is everyone else I really care about.

Here’s to the next 10 years. And I might actually write some more next year.

Best Wishes

Mick

Categories: LUG Community Blogs

Chris Lamb: 10 years of Debian

Mon, 19/12/2016 - 11:27

Today marks the 10-year anniversary of my first contribution to Debian GNU/Linux.

I will not recount the full history here but my first experience with Debian was a happy accident. I had sent off for a 5-CD set of Red Hat from The Linux Emporium only to discover I lacked the required 12MB of RAM. Annoyed, I reached for the Debian "potato" CD that was included gratis in my order due to it being outdated at the time…

Fast-forwarding a few years, whilst my first contribution was trivial, it was Thomas Bushnell's infectious enthusiasm that led me to contribute more, eventually becoming a Google Summer of Code student under Daniel Baumann, and finally becoming an official Debian Developer in September 2008 with Thomas Viehmann as my Application Manager. (Some things may never change, however I still struggle with the bug tracker's control@ interface.)

The response I got to my patch always reminds me of the irrational power of providing attibution. I've always liked to tell myself I'm above such vanities but perhaps the truly mature approach would be to accept that ego is part of the human condition and—as a community—take steps to avoid handicapping ourselves by underestimating the value of "trivialities" such as having one's name listed.

I've since been fascinated by the number of maintainers who do not attribute patches in changelogs, especially from newcomers or when the changes are non-trivial — a handful in particular have stung me fairly deeply.

I would certainly concede that it adds nothing technical and can even be distracting, but it seems a reasonable concession that dramatically increases the chance of future efforts or, frankly, is simply a kindly gesture of thanks and good will. Given our level of technical expertise, I fear we regularly suffer from not having sufficient empathy for newcomers or first-time users who lack the context or orientation that we possess.

Anyway, here's to another ten…

Categories: LUG Community Blogs

Jonathan McDowell: Timezones + static blog generation

Sun, 18/12/2016 - 23:28

So, it turns out when you move to static blog generation and do the generation on your laptop, which is usually in the timezone you’re currently physically located, it can cause URLs to change. Especially if you’re prone to blogging late at night, which can result in even just a shift to DST changing things. I’ve forced jekyll to UTC by adding timezone: 'UTC' to the config, and ensuring all the posts now have timezones for when they were written (a lot of the imported ones didn’t), so hopefully things should be stable from here on.

Categories: LUG Community Blogs

Jonathan McDowell: No longer a student. Again.

Mon, 12/12/2016 - 22:27

(image courtesy of XKCD)

Last week I graduated with a Masters in Legal Science (now taught as an MLaw) from Queen’s University Belfast. I’m pleased to have achieved a Distinction, as well an award for Outstanding Achievement in the Dissertation (which was on the infringement of privacy by private organisations due to state mandated surveillance and retention laws - pretty topical given the unfortunate introduction of the Investigatory Powers Act 2016). However, as previously stated, I had made the decision that I was happier building things, and wanted to return to the world of technology. I talked to a bunch of interesting options, got to various stages in the hiring process with each of them, and happily accepted a role with Titan IC Systems which started at the beginning of September.

Titan have produced a hardware accelerated regular expression processor (hence the XKCD reference); the RXP in its FPGA variant (what I get to play with) can handle pattern matching against 40Gb/s of traffic. Which is kinda interesting, as it lends itself to a whole range of applications from network scanning to data mining to, well, anything where you want to sift through a large amount of data checking against a large number of rules. However it’s brand new technology for me to get up to speed with (plus getting back into a regular working pattern rather than academentia), and the combination of that and spending most of the summer post DebConf wrapping up the dissertation has meant I haven’t had as much time to devote other things as I’d have liked. However I’ve a few side projects at various stages of completion and will try to manage more regular updates.

Categories: LUG Community Blogs

Chris Lamb: Free software activities in November 2016

Wed, 30/11/2016 - 21:18

Here is my monthly update covering what I have been doing in the free software world (previous month):

  • Started work on a Python API to the UK Postbox mail scanning and forwarding service. (repo)
  • Lots of improvements to buildinfo.debian.net, my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them, including making GPG signatures mandatory (#7), updating jenkins.debian.net to sign them and moving to SSL.
  • Improved the Django client to the KeyError error tracking software, enlarging the test coverage and additionally adding support for grouping errors using a context manager.
  • Made a number of improvements to travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds on every code change:
    • Install build-dependencies with debugging output. Thanks to @waja. (#31)
    • Install Lintian by default. Thanks to @freeekanayaka. (#33).
    • Call mktemp with --dry-run to avoid having to delete it later. (commit)
  • Submitted a pull request to Wheel (a utility to package Python libraries) to make the output of METADATA files reproducible. (#73)
  • Submitted some miscellaneous documentation updates to the Tails operating system. (patches)
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users.

The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced — either maliciously or accidentally — during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.


This month:


My work in the Reproducible Builds project was also covered in our weekly reports (#80, #81, #82 #83).


Toolchain issues

I submitted the following patches to fix reproducibility-related toolchain issues with Debian:


strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.


jenkins.debian.net

jenkins.debian.net runs our comprehensive testing framework.

  • buildinfo.debian.net has moved to SSL. (ac3b9e7)
  • Submit signing keys to keyservers after generation. (bdee6ff)
  • Various cosmetic changes, including
    • Prefer if X not in Y over if not X in Y. (bc23884)
    • No need for a dictionary; let's just use a set. (bf3fb6c)
    • Avoid DRY violation by using a for loop. (4125ec5)

I also submitted 9 patches to fix specific reproducibility issues in apktool, cairo-5c, lava-dispatcher, lava-server, node-rimraf, perlbrew, qsynth, tunnelx & zp.

Debian
Debian LTS

This month I have been paid to work 11 hours on Debian Long Term Support (LTS). In that time I did the following:

  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 697-1 for bsdiff fixing an arbitrary write vulnerability.
  • Issued DLA 705-1 for python-imaging correcting a number of memory overflow issues.
  • Issued DLA 713-1 for sniffit where a buffer overflow allowed a specially-crafted configuration file to provide a root shell.
  • Issued DLA 723-1 for libsoap-lite-perl preventing a Billion Laughs XML expansion attack.
  • Issued DLA 724-1 for mcabber fixing a roster push attack.
Uploads
  • redis:
    • 3.2.5-2 — Tighten permissions of /var/{lib,log}/redis. (#842987)
    • 3.2.5-3 & 3.2.5-4 — Improve autopkgtest tests and install upstream's MANIFESTO and README.md documentation.
  • gunicorn (19.6.0-9) — Adding autopkgtest tests.
  • libfiu:
    • 0.94-1 — Add autopkgtest tests.
    • 0.95-1, 0.95-2 & 0.95-3 — New upstream release and improve autopkgtest coverage.
  • python-django (1.10.3-1) — New upstream release.
  • aptfs (0.8-3, 0.8-4 & 0.8-5) — Adding and subsequently improving the autopkgtext tests.


I performed the following QA uploads:



Finally, I also made the following non-maintainer uploads:

  • libident (0.22-3.1) — Move from obsolete Source-Version substvar to binary:Version. (#833195)
  • libpcl1 (1.6-1.1) — Move from obsolete Source-Version substvar to binary:Version. (#833196)
  • pygopherd (2.0.18.4+nmu1) — Move from obsolete Source-Version substvar to ${source:Version}. (#833202)
Debian bugs filed RC bugs

I also filed 59 FTBFS bugs against arc-gui-clients, asyncpg, blhc, civicrm, d-feet, dpdk, fbpanel, freeciv, freeplane, gant, golang-github-googleapis-gax-go, golang-github-googleapis-proto-client-go, haskell-cabal-install, haskell-fail, haskell-monadcatchio-transformers, hg-git, htsjdk, hyperscan, jasperreports, json-simple, keystone, koji, libapache-mod-musicindex, libcoap, libdr-tarantool-perl, libmath-bigint-gmp-perl, libpng1.6, link-grammar, lua-sql, mediatomb, mitmproxy, ncrack, net-tools, node-dateformat, node-fuzzaldrin-plus, node-nopt, open-infrastructure-system-images, open-infrastructure-system-images, photofloat, ppp, ptlib, python-mpop, python-mysqldb, python-passlib, python-protobix, python-ttystatus, redland, ros-message-generation, ruby-ethon, ruby-nokogiri, salt-formula-ceilometer, spykeviewer, sssd, suil, torus-trooper, trash-cli, twisted-web2, uftp & wide-dhcpv6.

FTP Team

As a Debian FTP assistant I ACCEPTed 70 packages: bbqsql, coz-profiler, cross-toolchain-base, cross-toolchain-base-ports, dgit-test-dummy, django-anymail, django-hstore, django-html-sanitizer, django-impersonate, django-wkhtmltopdf, gcc-6-cross, gcc-defaults, gnome-shell-extension-dashtodock, golang-defaults, golang-github-btcsuite-fastsha256, golang-github-dnephin-cobra, golang-github-docker-go-events, golang-github-gogits-cron, golang-github-opencontainers-image-spec, haskell-debian, kpmcore, libdancer-logger-syslog-perl, libmoox-buildargs-perl, libmoox-role-cloneset-perl, libreoffice, linux-firmware-raspi3, linux-latest, node-babel-runtime, node-big.js, node-buffer-shims, node-charm, node-cliui, node-core-js, node-cpr, node-difflet, node-doctrine, node-duplexer2, node-emojis-list, node-eslint-plugin-flowtype, node-everything.js, node-execa, node-grunt-contrib-coffee, node-grunt-contrib-concat, node-jquery-textcomplete, node-js-tokens, node-json5, node-jsonfile, node-marked-man, node-os-locale, node-sparkles, node-tap-parser, node-time-stamp, node-wrap-ansi, ooniprobe, policycoreutils, pybind11, pygresql, pysynphot, python-axolotl, python-drizzle, python-geoip2, python-mockupdb, python-pyforge, python-sentinels, python-waiting, pythonmagick, r-cran-isocodes, ruby-unicode-display-width, suricata & voctomix-outcasts.

I additionally filed 4 RC bugs against packages that had incomplete debian/copyright files against node-cliui, node-core-js, node-cpr & node-grunt-contrib-concat.

Categories: LUG Community Blogs