Planet ALUG

Syndicate content
Planet ALUG - http://planet.alug.org.uk/
Updated: 1 hour 51 min ago

Mick Morgan: pwned

Sat, 18/03/2017 - 12:55

I recently received a spam email to one of my email addresses. In itself this is annoying, but not particularly interesting or that unusual (despite my efforts to avoid such nuisances). What was unusual was the form of the address because it contained a username I have not used in a long time, and only on one specific site.

The address took the form “username” <realaddress@realdomain> and the email invited me to hook up with a “hot girl” who “was missing me”. The return address was at a Russian domain.

Intrigued as to how this specific UID and address had appeared in my inbox I checked Troy Hunt’s haveibeenpwned database and found that, sure enough, the site I had signed up to with that UID had been compromised. I have since both changed the password on that site (too late of course because it would seem that the password database was stored insecurely) and deleted the account (which I haven’t used in years anyway). I don’t /think/ that I have used that particular UID/password combination anywhere else, but I’m checking nonetheless.

The obvious lesson here is that a) password re-use is a /very/ bad idea and b) even old unused accounts can later cause you difficulty if you don’t manage them actively.

But you knew that anyway. Didn’t you?

Categories: LUG Community Blogs

Jonathan McDowell: Rational thoughts on the GitHub ToS change

Thu, 02/03/2017 - 18:13

I woke this morning to Thorsten claiming the new GitHub Terms of Service could require the removal of Free software projects from it. This was followed by joeyh removing everything from github. I hadn’t actually been paying attention, so I went looking for some sort of summary of whether I should be worried and ended up reading the actual ToS instead. TL;DR version: No, I’m not worried and I don’t think you should be either.

First, a disclaimer. I’m not a lawyer. I have some legal training, but none of what I’m about to say is legal advice. If you’re really worried about the changes then you should engage the services of a professional.

The gist of the concerns around GitHub’s changes are that they potentially circumvent any license you have applied to your code, either converting GPL licensed software to BSD style (and thus permitting redistribution of binary forms without source) or making it illegal to host software under certain Free software licenses on GitHub due to being unable to meet the requirements of those licenses as a result of GitHub’s ToS.

My reading of the GitHub changes is that they are driven by a desire to ensure that GitHub are legally covered for the things they need to do with your code in order to run their service. There are sadly too many people who upload code there without a license, meaning that technically no one can do anything with it. Don’t do this people; make sure that any project you put on GitHub has some sort of license attached to it (don’t write your own - it’s highly likely one of Apache/BSD/GPL will suit your needs) so people know whether they can make use of it or not. “I don’t care” is not a valid reason not to do this.

Section D, relating to user generated content, is the one causing the problems. It’s possibly easiest to walk through each subsection in order.

D1 says GitHub don’t take any responsibility for your content; you make it, you’re responsible for it, they’re not accepting any blame for harm your content does nor for anything any member of the public might do with content you’ve put on GitHub. This seems uncontentious.

D2 reaffirms your ownership of any content you create, and requires you to only post 3rd party content to GitHub that you have appropriate rights to. So I can’t, for example, upload a copy of ‘Friday’ by Rebecca Black.

Thorsten has some problems with D3, where GitHub reserve the right to remove content that violates their terms or policies. He argues this could cause issues with licenses that require unmodified source code. This seems to be alarmist, and also applies to any random software mirror. The intent of such licenses is in general to ensure that the pristine source code is clearly separate from 3rd party modifications. Removal of content that infringes GitHub’s T&Cs is not going to cause an issue.

D4 is a license grant to GitHub, and I think forms part of joeyh’s problems with the changes. It affirms the content belongs to the user, but grants rights to GitHub to store and display the content, as well as make copies such as necessary to provide the GitHub service. They explicitly state that no right is granted to sell the content at all or to distribute the content outside of providing the GitHub service.

This term would seem to be the minimum necessary for GitHub to ensure they are allowed to provide code uploaded to them for download, and provide their web interface. If you’ve actually put a Free license on your code then this isn’t necessary, but from GitHub’s point of view I can understand wanting to make it explicit that they need these rights to be granted. I don’t believe it provides a method of subverting the licensing intent of Free software authors.

D5 provides more concern to Thorsten. It seems he believes that the ability to fork code on GitHub provides a mechanism to circumvent copyleft licenses. I don’t agree. The second paragraph of this subsection limits the license granted to the user to be the ability to reproduce the content on GitHub - it does not grant them additional rights to reproduce outside of GitHub. These rights, to my eye, enable the forking and viewing of content within GitHub but say nothing about my rights to check code out and ignore the author’s upstream license.

D6 clarifies that if you submit content to a GitHub repo that features a license you are licensing your contribution under these terms, assuming you have no other agreement in place. This looks to be something that benefits projects on GitHub receiving contributions from users there; it’s an explicit statement that such contributions are under the project license.

D7 confirms the retention of moral rights by the content owner, but states they are waived purely for the purposes of enabling GitHub to provide service, as stated under D4. In particular this right is revocable so in the event they do something you don’t like you can instantly remove all of their rights. Thorsten is more worried about the ability to remove attribution and thus breach CC-BY or some BSD licenses, but GitHub’s whole model is providing attribution for changesets and tracking such changes over time, so it’s hard to understand exactly where the service falls down on ensuring the provenance of content is clear.

There are reasons to be wary of GitHub (they’ve taken a decentralised revision control system and made a business model around being a centralised implementation of it, and they store additional metadata such as PRs that aren’t as easily extracted), but I don’t see any indication that the most recent changes to their Terms of Service are something to worry about. The intent is clearly to provide GitHub with the legal basis they need to provide their service, rather than to provide a means for them to subvert the license intent of any Free software uploaded.

Categories: LUG Community Blogs

Brett Parker (iDunno): Using the Mythic Beasts IPv4 -&gt; IPv6 Proxy for Websites on a v6 only Pi and getting the right REMOTE_ADDR

Wed, 01/03/2017 - 18:35

So, more because I was intrigued than anything else, I've got a pi3 from Mythic Beasts, they're supplied with IPv6 only connectivity and the file storage is NFS over a private v4 network. The proxy will happily redirect requests to either http or https to the Pi, but this results (without turning on the Proxy Protocol) with getting remote addresses in your logs of the proxy servers, which is not entirely useful.

I've cheated a bit, because the turning on of ProxyProtocol for the hostedpi.com addresses is currently not exposed to customers (it's on the list!), to do it without access to Mythic's backends use your own domainname (I've also got https://pi3.sommitrealweird.co.uk/ mapped to this Pi).

So, first step first, we get our RPi and we make sure that we can login to it via ssh (I'm nearly always on a v6 connection anyways, so this was a simple case of sshing to the v6 address of the Pi). I then installed haproxy and apache2 on the Pi and went about configuring them, with apache2 I changed it to listen to localhost only and on ports 8080 and 4443, I hadn't at this point enabled the ssl module so, really, the change for 4443 didn't kick in. Here's my /etc/apache2/ports.conf file:

# If you just change the port or add more ports here, you will likely also # have to change the VirtualHost statement in # /etc/apache2/sites-enabled/000-default.conf Listen [::1]:8080 <IfModule ssl_module> Listen [::1]:4443 </IfModule> <IfModule mod_gnutls.c> Listen [::1]:4443 </IfModule> # vim: syntax=apache ts=4 sw=4 sts=4 sr noet

I then edited /etc/apache2/sites-available/000-default.conf to change the VirtualHost line to [::1]:8080.

So, with that in place, now we deploy haproxy infront of it, the basic /etc/haproxy/haproxy.cfg config is:

global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy stats socket /run/haproxy/admin.sock mode 660 level admin stats timeout 30s user haproxy group haproxy daemon # Default SSL material locations ca-base /etc/ssl/certs crt-base /etc/ssl/private # Default ciphers to use on SSL-enabled listening sockets. # For more information, see ciphers(1SSL). This list is from: # https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/ ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS ssl-default-bind-options no-sslv3 defaults log global mode http option httplog option dontlognull timeout connect 5000 timeout client 50000 timeout server 50000 errorfile 400 /etc/haproxy/errors/400.http errorfile 403 /etc/haproxy/errors/403.http errorfile 408 /etc/haproxy/errors/408.http errorfile 500 /etc/haproxy/errors/500.http errorfile 502 /etc/haproxy/errors/502.http errorfile 503 /etc/haproxy/errors/503.http errorfile 504 /etc/haproxy/errors/504.http frontend any_http option httplog option forwardfor acl is_from_proxy src 2a00:1098:0:82:1000:3b:1:1 2a00:1098:0:80:1000:3b:1:1 tcp-request connection expect-proxy layer4 if is_from_proxy bind :::80 default_backend any_http backend any_http server apache2 ::1:8080

Obviously after that you then do:

systemctl restart apache2 systemctl restart haproxy

Now you have a proxy protocol'd setup from the proxy servers, and you can still talk directly to the Pi over ipv6, you're not yet logging the right remote ips, but we're a step closer. Next enable mod_remoteip in apache2:

a2enmod remoteip

And add a file, /etc/apache2/conf-available/remoteip-logformats.conf containing:

LogFormat "%v:%p %a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" remoteip_vhost_combined

And edit the /etc/apache2/sites-available/000-default.conf to change the CustomLog line to use remoteip_vhost_combined rather than combined as the LogFormat and add the relevant RemoteIP settings:

RemoteIPHeader X-Forwarded-For RemoteIPTrustedProxy ::1 CustomLog ${APACHE_LOG_DIR}/access.log remoteip_vhost_combined

Now, enable the config and restart apache2:

a2enconf remoteip-logformats systemctl restart apache2

Now you'll get the right remote ip in the logs (cool, huh!), and, better still, the environment that gets pushed through to cgi scripts/php/whatever is now also correct.

So, you can now happily visit http://www.<your-pi-name>.hostedpi.com/, e.g. http://www.srwpi.hostedpi.com/.

Next up, you'll want something like dehydrated - I grabbed the packaged version from debian's jessie-backports repository - so that you can make yourself some nice shiny SSL certificates (why wouldn't you, after all!), once you've got dehydrated installed, you'll probably want to tweak it a bit, I have some magic extra files that I use, I also suggest getting the dehydrated-apache2 package, which just makes it all much easier too.

/etc/dehydrated/conf.d/mail.sh:

CONTACT_EMAIL="my@email.address"

/etc/dehydrated/conf.d/domainconfig.sh:

DOMAINS_D="/etc/dehydrated/domains.d"

/etc/dehydrated/domains.d/srwpi.hostedpi.com:

HOOK="/etc/dehydrated/hooks/srwpi"

/etc/dehydrated/hooks/srwpi:

#!/bin/sh action="$1" domain="$2" case $action in deploy_cert) privkey="$3" cert="$4" fullchain="$5" chain="$6" cat "$privkey" "$fullchain" > /etc/ssl/private/srwpi.pem chmod 640 /etc/ssl/private/srwpi.pem ;; *) ;; esac

/etc/dehydrated/hooks/srwpi has the execute bit set (chmod +x /etc/dehydrated/hooks/srwpi), and is really only there so that the certificate can be used easily in haproxy.

And finally the file /etc/dehydrated/domains.txt:

www.srwpi.hostedpi.com srwpi.hostedpi.com

Obviously, use your own pi name in there, or better yet, one of your own domain names that you've mapped to the proxies.

Run dehydrated in cron mode (it's noisy, but meh...):

dehydrated -c

That s then generated you some shiny certificates (hopefully). For now, I'll just tell you how to do it through the /etc/apache2/sites-available/default-ssl.conf file, just edit that file and change the SSLCertificateFile and SSLCertificateKeyFile to point to /var/lib/dehydrated/certs/www.srwpi.hostedpi.com/fullchain.pem and /var/llib/dehydrated/certs/ww.srwpi.hostedpi.com/privkey.pem files, do the edit for the CustomLog as you did for the other default site, and change the VirtualHost to be [::1]:443 and enable the site:

a2ensite default-ssl a2enmod ssl

And restart apache2:

systemctl restart apache2

Now time to add some bits to haproxy.cfg, usefully this is only a tiny tiny bit of extra config:

frontend any_https option httplog option forwardfor acl is_from_proxy src 2a00:1098:0:82:1000:3b:1:1 2a00:1098:0:80:1000:3b:1:1 tcp-request connection expect-proxy layer4 if is_from_proxy bind :::443 ssl crt /etc/ssl/private/srwpi.pem default_backend any_https backend any_https server apache2 ::1:4443 ssl ca-file /etc/ssl/certs/ca-certificates.crt

Restart haproxy:

systemctl restart haproxy

And we're all done! REMOTE_ADDR will appear as the correct remote address in the logs, and in the environment.

Categories: LUG Community Blogs

Brett Parker (iDunno): Ooooooh! Shiny!

Wed, 01/03/2017 - 15:12

Yay! So, it's a year and a bit on from the last post (eeep!), and we get the news of the Psion Gemini - I wants one, that looks nice and shiny and just the right size to not be inconvenient to lug around all the time, and far better for ssh usage than the onscreen keyboard on my phone!

Categories: LUG Community Blogs

Chris Lamb: Free software activities in February 2017

Tue, 28/02/2017 - 22:09

Here is my monthly update covering what I have been doing in the free software world (previous month):

  • Submitted a number of pull requests to the Django web development framework:
    • Add a --mode=unified option to the "diffsettings" management command. (#8113)
    • Fix a crash in setup_test_environment() if ALLOWED_HOSTS is a tuple. (#8101)
    • Use Python 3 "shebangs" now that the master branch is Python 3 only. (#8105)
    • URL namespacing warning should consider nested namespaces. (#8102)
  • Created an experimental patch against the Python interpreter in order to find reproducibility-related assumptions in dict handling in arbitrary Python code. (#29431)
  • Filed two issues against dh-virtualenv, a tool to package Python virtualenv environments in Debian packages:
    • Fix "upgrage-pip" typo in usage documentation. (#195)
    • Missing DH_UPGRADE_SETUPTOOLS equivalent for dh_virtualenv (#196)
  • Fixed a large number of spelling corrections in Samba, a free-software re-implementation of the Windows networking protocols.
  • Reviewed and merged a pull request by @jheld for django-slack (my library to easily post messages to the Slack group-messaging utility) to support per-message backends and channels. (#63)
  • Created a pull request for django-two-factor-auth, a complete Two-Factor Authentication (2FA) framework for projects using the Django web development framework to drop use of the @lazy_property decorator to ensure compatibility with Django 1.11. (#195)
  • Filed, triaged and eventually merged a change from @evgeni to fix an autopkgtest-related issue in travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds on every code change) travis.debian.net. (#41)
  • Submitted a pull request against social-core — a library to allow Python applications to authenticate against third-party web services such as Facebook, Twitter, etc. — to use the more-readable X if Y else Z construction over Y and X or Z. (#44)
  • Filed an issue against freezegun (a tool to make it easier to write Python tests involving times) to report that dateutils was missing from requirements.txt. (#173)
  • Submitted a pull request against the Hypothesis "QuickCheck"-like testing framework to make the build reproducible. (#440)
  • Fixed an issue reported by @davidak in trydiffoscope (a web-based version of the diffoscope in-depth and content-aware diff utility) where the maximum upload size was incorrectly calculated. (#22)
  • Created a pull request for the Mars Simulation Project to remove some embedded timestamps from the changelog.gz and mars-sim.1.gz files in order to make the build reproducible. (#24)
  • Filed a bug against the cpio archiving utility to report that the testsuite fails when run in the UTC +1300 timezone. (Thread)
  • Submitted a pull request against the "pnmixer" system-tray volume mixer in order to make the build reproducible. (#153)
  • Sent a patch to Testfixtures (a collection of helpers and mock objects that are useful when writing Python unit tests or doctests) to make the build reproducible. (#56)
  • Created a pull request for the "Cloud" Sphinx documentation theme in order to make the output reproducible. (#22)
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users.

The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced — either maliciously or accidentally — during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

(I have been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.)

This month I:

I also made the following changes to our tooling:

diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • New features:
    • Add a machine-readable JSON output format. (Closes: #850791).
    • Add an --exclude option. (Closes: #854783).
    • Show results from debugging packages last. (Closes: #820427).
    • Extract archive members using an auto-incrementing integer avoiding the need to sanitise filenames. (Closes: #854723).
    • Apply --max-report-size to --text output. (Closes: #851147).
    • Specify <html lang="en"> in the HTML output. (re. #849411).
  • Bug fixes:
    • Fix errors when comparing directories with non-directories. (Closes: #835641).
    • Device and RPM fallback comparisons require xxd. (Closes: #854593).
    • Fix tests that call xxd on Debian Jessie due to change of output format. (Closes: #855239).
    • Add missing Recommends for comparators. (Closes: #854655).
    • Importing submodules (ie. parent.child) will attempt to import parent. (Closes: #854670).
    • Correct logic of module_exists ensuring we correctly skip the debian.deb822 tests when python3-debian is not installed. (Closes: #854745).
    • Clean all temporary files in the signal handler thread instead of attempting to pass the exception back to the main thread. (Closes: #852013).
    • Fix behaviour of setting report maximums to zero (ie. no limit).
  • Optimisations:
    • Don't uselessly run xxd(1) on non-directories.
    • No need to track libarchive directory locations.
    • Optimise create_limited_print_func.
  • Tests:
    • When comparing two empty directories, ensure that the mtime of the directory is consistent to avoid non-deterministic failures.
    • Ensure we can at least import the "deb_fallback" and "rpm_fallback" modules.
    • Add test for symlink differing in destination.
    • Add tests for --progress, --status-fd and profiling output options as well as the Deb{Changes,Buildinfo,Dsc} and RPM fallback comparisons.
    • Add get_data and @skip_unless_module_exists test helpers.
    • Mark impossible-to-reach code to improve test coverage.

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Drop raw_text fields now as we've moved these to Amazon S3.
  • Drop storage of Installed-Build-Depends and subsequently-orphaned Binary package instances to recover diskspace.

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Print log entry when fixing a file. (Closes: #777239).
  • Run our entire testsuite in autopkgtests, not just the first test. (Closes: #852517).
  • Don't test for stat(2)'s blksize and block attributes. (Closes: #854937).
  • Use error() from Dh_Lib.pm over "manual" die().


Debian Patches contributed Debian LTS

This month I have been paid to work 13 hours on Debian Long Term Support (LTS). In that time I did the following:

  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 817-1 for libphp-phpmailer, correcting a local file disclosure vulnerability where insufficient parsing of HTML messages could potentially be used by attacker to read a local file.
  • Issued DLA 826-1 for wireshark which fixes a denial of service vulnerability in wireshark, where a malformed NATO Ground Moving Target Indicator Format ("STANAG 4607") capture file could cause a memory exhausion/infinite loop.
Uploads
  • python-django (1:1.11~beta1-1) — New upstream beta release.
  • redis (3:3.2.8-1) — New upstream release.
  • gunicorn (19.6.0-11) — Use ${misc:Pre-Depends} to populate Pre-Depends for dpkg-maintscript-helper.
  • dh-virtualenv (1.0-1~bpo8+1) — Upload to jessie-backports.

I sponsored the following uploads:

I also performed the following QA uploads:

  • dh-kpatches (0.99.36+nmu4) — Make kernel kernel builds reproducible.

Finally, I made the following non-maintainer uploads:

  • cpio (2.12+dfsg-3) — Remove rmt.8.gz to prevent a piuparts error.
  • dot-forward (1:0.71-2.2) — Correct a FTBFS; we don't install anything to /usr/sbin, so use GNU Make's $(wildcard ..) over the shell's own * expansion.
Debian bugs filed

I also filed 15 FTBFS bugs against binaryornot, chaussette, examl, ftpcopy, golang-codegangsta-cli, hiro, jarisplayer, libchado-perl, python-irc, python-stopit, python-stopit, python-stopit, python-websockets, rubocop & yash.

FTP Team

As a Debian FTP assistant I ACCEPTed 116 packages: autobahn-cpp, automat, bglibs, bitlbee, bmusb, bullet, case, certspotter, checkit-tiff, dash-el, dash-functional-el, debian-reference, el-x, elisp-bug-hunter, emacs-git-messenger, emacs-which-key, examl, genwqe-user, giac, golang-github-cloudflare-cfssl, golang-github-docker-goamz, golang-github-docker-libnetwork, golang-github-go-openapi-spec, golang-github-google-certificate-transparency, golang-github-karlseguin-ccache, golang-github-karlseguin-expect, golang-github-nebulouslabs-bolt, gpiozero, gsequencer, jel, libconfig-mvp-slicer-perl, libcrush, libdist-zilla-config-slicer-perl, libdist-zilla-role-pluginbundle-pluginremover-perl, libevent, libfunction-parameters-perl, libopenshot, libpod-weaver-section-generatesection-perl, libpodofo, libprelude, libprotocol-http2-perl, libscout, libsmali-1-java, libtest-abortable-perl, linux, linux-grsec, linux-signed, lockdown, lrslib, lua-curses, lua-torch-cutorch, mariadb-10.1, mini-buildd, mkchromecast, mocker-el, node-arr-exclude, node-brorand, node-buffer-xor, node-caller, node-duplexer3, node-ieee754, node-is-finite, node-lowercase-keys, node-minimalistic-assert, node-os-browserify, node-p-finally, node-parse-ms, node-plur, node-prepend-http, node-safe-buffer, node-text-table, node-time-zone, node-tty-browserify, node-widest-line, npd6, openoverlayrouter, pandoc-citeproc-preamble, pydenticon, pyicloud, pyroute2, pytest-qt, pytest-xvfb, python-biomaj3, python-canonicaljson, python-cgcloud, python-gffutils, python-h5netcdf, python-imageio, python-kaptan, python-libtmux, python-pybedtools, python-pyflow, python-scrapy, python-scrapy-djangoitem, python-signedjson, python-unpaddedbase64, python-xarray, qcumber, r-cran-urltools, radiant, repo, rmlint, ruby-googleauth, ruby-os, shutilwhich, sia, six, slimit, sphinx-celery, subuser, swarmkit, tmuxp, tpm2-tools, vine, wala & x265.

I additionally filed 8 RC bugs against packages that had incomplete debian/copyright files against: checkit-tiff, dash-el, dash-functional-el, libcrush, libopenshot, mkchromecast, pytest-qt & x265.

Categories: LUG Community Blogs

Mick Morgan: this is what a scary man looks like

Thu, 09/02/2017 - 15:23

No, I mean the one on the right – the one Trump is pointing at.

General John Kelly is just one of Trump’s controversial appointments (and not necessarily the worst) and I guess that by writing this now, I have finally nailed down the lid on the coffin of my ever returning to the US. Pity. I had promised my wife that I would take her to San Francisco in the near future so that she could see for herself why I like it. I’ve visited the USA several times in the past, but only on business and never with my lady. Now it would seem that I cannot go, because I will not submit her, nor myself, to the indignity of being treated like a criminal simply because I wish to enter the country.

Today, El Reg reports that General Kelly has said that he wants the right to demand passwords for social media and financial accounts from some visa applicants so that immigration and homeland securty officers can vet Twitter, Facebook or online banking accounts.

Kelly is reported to have said:

“We want to say ‘what kind of sites do you visit and give us your passwords,’ so we can see what they do. We want to get on their social media with passwords – what do you do, what do you say. If they don’t want to cooperate then they don’t come in. If they truly want to come to America they’ll cooperate, if not then ‘next in line’.”

Now as El Reg points out:

“By “they”, Kelly was referring to refugees and visa applicants from the seven Muslim countries subject to President Trump’s anti-immigration executive order, which was signed last month.”

But it goes on:

“Given the White House’s tough stance on immigration, we can imagine the scope of this “enhanced vetting” creeping from that initial subset to cover visitors of other nationalities. Just simply wait for the president to fall out with another country.”

Or for individuals to draw attention to themselves by being publicly critical of some of the more worrying developments in the USA…..

My own experience of US immigration, even whilst travelling under an A2 Visa, is such that I would most certainly not wish to enter the country if I were to be treated with anything like the hostility I know could be possible. Unfortunately that also means that I might have a problem should I ever wish to fly anywhere else in the world which necessitates a stopover in the US.

The reason I think Kelly may be truly scary? He is reported to have told Representative Kathleen Rice under questioning that:

“I work for one man, his name is Donald Trump, and he told me ‘Kelly, secure the border,’ and that’s what I’m going to do,”

In typical El Reg commentard style, some responders have been less than subtle about this response, evoking obvious references to Godwin’s Law, but one poster, called Jim-234 notes:

“This is a truly stupid plan that is bound to fail on so many levels and will do nothing but upset decent people and open them up to hacking & identity theft while doing nothing to actually stop people who want to cause harm. It reeks of lazy ignorant fools who want to be seen to do something rather than actually do something that works…..

“This is just going to be security theater and bothering everyone and invading their privacy for no net effect at all. As soon as it goes live, all the bad guys will know they need a clean profile online, there will probably even be special paid services to make your online profile all nice and minty fresh, probably even with posting and messaging “good” stuff to make sure you look nice online.”

Jim-234 concludes:

“They want to start demanding your passwords for your phones & laptops?

.. well pretty soon all they will find is factory reset phones, laptops with a never used OS and a new booming business for Chinese, Russian and European data centers of “whole system data backups”.

The only good news is that if this goes live, everyone will probably start scrubbing their Facebook profiles to be about as informative as Zuckerberg’s page… so maybe then Facebook will finally go the way of MySpace.”

Depressingly, I see the same tendency in the UK for security theatre because politicians think “we must be seen to be doing something” in order to make the people feel safer. As the saying goes, “the road to hell is paved with good intentions”.

And what about when the intentions themselves are not good?

Categories: LUG Community Blogs

Jonathan McDowell: GnuK on the Maple Mini

Tue, 07/02/2017 - 18:34

Last weekend, as a result of my addiction to buying random microcontrollers to play with, I received some Maple Minis. I bought the Baite clone direct from AliExpress - so just under £3 each including delivery. Not bad for something that’s USB capable, is based on an ARM and has plenty of IO pins.

I’m not entirely sure what my plan is for the devices, but as a first step I thought I’d look at getting GnuK up and running on it. Only to discover that chopstx already has support for the Maple Mini and it was just a matter of doing a ./configure --vidpid=234b:0000 --target=MAPLE_MINI --enable-factory-reset ; make. I’d hoped to install via the DFU bootloader already on the Mini but ended up making it unhappy so used SWD by following the same steps with OpenOCD as for the FST-01/BusPirate. (SWCLK is D21 and SWDIO is D22 on the Mini). Reset after flashing and the device is detected just fine:

usb 1-1.1: new full-speed USB device number 73 using xhci_hcd usb 1-1.1: New USB device found, idVendor=234b, idProduct=0000 usb 1-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 usb 1-1.1: Product: Gnuk Token usb 1-1.1: Manufacturer: Free Software Initiative of Japan usb 1-1.1: SerialNumber: FSIJ-1.2.3-87155426

And GPG is happy:

$ gpg --card-status Reader ...........: 234B:0000:FSIJ-1.2.3-87155426:0 Application ID ...: D276000124010200FFFE871554260000 Version ..........: 2.0 Manufacturer .....: unmanaged S/N range Serial number ....: 87155426 Name of cardholder: [not set] Language prefs ...: [not set] Sex ..............: unspecified URL of public key : [not set] Login data .......: [not set] Signature PIN ....: forced Key attributes ...: rsa2048 rsa2048 rsa2048 Max. PIN lengths .: 127 127 127 PIN retry counter : 3 3 3 Signature counter : 0 Signature key ....: [none] Encryption key....: [none] Authentication key: [none] General key info..: [none]

While GnuK isn’t the fastest OpenPGP smart card implementation this certainly seems to be one of the cheapest ways to get it up and running. (Plus the fact that chopstx already runs on the Mini provides me with a useful basis for other experimentation.)

Categories: LUG Community Blogs

Chris Lamb: The ChangeLog #237: Reproducible Builds and Secure Software

Sat, 04/02/2017 - 20:39

I recently appeared on the Changelog podcast to talk about the Reproducible Builds project:


Whilst I am an avid podcast listener, this was actually my first appearance on one. It was an curious and somewhat disconcerting feeling to be "just" talking to Adam and Jerod in the moment yet knowing all the time that anything and everything I said would be distributed more widely in the future.

Categories: LUG Community Blogs

Chris Lamb: Free software activities in January 2017

Tue, 31/01/2017 - 08:54

Here is my monthly update covering what I have been doing in the free software world (previous month):

  • Created github-sync, a tool to mirror arbitrary repositories onto GitHub.
  • Submitted two pull requests to the word-wrap Chrome browser extension that adds the ability to wrap text via the right-click context menu:
    • Support dynamically-added <textarea> elements in "rich" Javascript applications such as mail clients, etc. (#2)
    • Avoid an error message if no "editable" has been selected yet. (#1)
  • Submitted a pull request to wordwarvi (a "retro-styled old school side-scrolling shooter") to ensure the build is reproducible. (#5)
  • Filed a pull request with the yard Ruby documentation tool to ensure the generated output is reproducible. (#1048)
  • Made some improvements to travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds on every code change:
    • Merged a pull request from Evgeni Golov to allow for skipped tests. (#39)
    • Add logging when running autopkgtests. (commit)
  • Merged a pull request from jwilk for python-fadvise my Python interface to the posix_fadvise(2) interface to predeclare an pattern for accessing data. (#6)
  • Filed an issue against the redis key-value database regarding build failures on non-x86 architectures. (#3768)
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users.

The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced — either maliciously or accidentally — during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

(I have previously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.)

This month I:

I also made the following changes to our tooling:

diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • Comparators:
    • Display magic file type when we know the file format but can't find file-specific details. (Closes: #850850).
    • Ensure our "APK metadata" file appears first, fixing non-deterministic tests. (998b288)
    • Fix APK extration with absolute filenames. (Closes: #850485).
    • Don't error if directory containing ELF debug symbols already exists. (Closes: #850807).
    • Support comparing .ico files (Closes: #850730).
    • If we don't have a tool (eg. apktool), don't blow up when attempting to unpack it.
  • Output formats:
    • Add Markdown output format. (Closes: #848141).
    • Add RestructuredText output format.
    • Use an optimised indentation routine throughout all presenters.
    • Move text presenter to use the Visitor pattern.
    • Correctly escape value of href="" elements (re. #849411).
  • Tests:
    • Prevent FTBFS by loading fixtures as UTF-8 in case surrounding terminal is not Unicode-aware. (Closes: #852926).
    • Skip tests if binutils can't handle the object file format. (Closes: #851588).
    • Actually compare the output of text/ReST/Markdown formats to fixtures.
    • Add tests for: Comparing two empty directories, HTML output, image.ICOImageFile, --html-dir, --text-color & no arguments (beyond the filenames) emits the text output.
  • Profiling:
    • Count the number of calls, not just the total time.
    • Skip as much profiling overhead when not enabled for a ~2% speedup.
  • Misc:
    • Alias an expensive Config() lookup for a 10% optimisation.
    • Avoid expensive regex creation until we actually need it, speeding up diff parsing by 2X.
    • Use Pythonic logging functions based on __name__, etc.
    • Drop milliseconds from logging output.

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Store files directly onto S3.
  • Drop big unique_together index to save disk space.
  • Show SHA256 checksums where space permits.

Debian LTS

This month I have been paid to work 12.75 hours on Debian Long Term Support (LTS). In that time I did the following:

  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 773-1 for python-crypto fixing a vulnerability where calling AES.new with an invalid parameter could crash the Python interpreter.
  • Issued DLA 777-1 for libvncserver addressing two heap-based buffer overflow attacks based on invalid FramebufferUpdate data.
  • Issued DLA 778-1 for pcsc-lite correcting a use-after-free vulnerability.
  • Issued DLA 795-1 for hesiod which fixed a weak SUID check as well as removed the hard-coding of a fallback domain if the configuration file could not be found.
  • Issued DLA 810-1 for libarchive fixing a heap buffer overflow.
Uploads
  • python-django:
    • 1:1.10.5-1 — New upstream stable release.
    • 1:1.11~alpha1-1 — New upstream experimental release.
  • gunicorn (19.6.0-10) — Moved debian/README.Debian to debian/NEWS so that the recent important changes will be displayed to users when upgrading to stretch.
  • redis:
    • 3:3.2.6-2 & 4:4.0-rc2-2 — Tidy patches and rename RunTimeDirectory to RuntimeDirectory in .service files. (Closes: #850534)
    • 3:3.2.6-3 — Remove a duplicate redis-server binary by symlinking /usr/bin/redis-check-rdb. This was found by the dedup service.
    • 3:3.2.6-4 — Expand the documentation in redis-server.service and redis-sentinel.service regarding the default hardening options and how, in most installations, they can be increased.
    • 3:3.2.6-5, 3:3.2.6-6, 4:4.0-rc2-3 & 4:4.0-rc2-4 — Add taskset calls to try and avoid build failures due to parallelism in upstream test suite.

I also made the following non-maintainer uploads:

  • cpio:
    • 2.12+dfsg-1 — New upstream release (to experimental), refreshing all patches, etc.
    • 2.12+dfsg-2 — Add missing autoconf to Build-Depends.
  • xjump (2.7.5-6.2) — Make the build reproducible by passing -n to gzip calls in debian/rules. (Closes: #777354)
  • magicfilter (1.2-64.1) — Make the build reproducible by passing -n to gzip calls in debian/rules. (Closes: #777478)
Debian bugs filed RC bugs

I also filed 16 FTBFS bugs against bzr-git, coq-highschoolgeometry, eclipse-anyedit, eclipse-gef, libmojolicious-plugin-assetpack-perl, lua-curl, node-liftoff, node-liftoff, octave-msh, pcb2gcode, qtile, rt-authen-externalauth, ruby-hamster, ruby-sshkit, tika & txfixtures.

FTP Team

As a Debian FTP assistant I ACCEPTed 35 packages: chromium-browser, debichem, flask-limiter, golang-github-golang-leveldb, golang-github-nebulouslabs-demotemutex, golang-github-nwidger-jsoncolor, libatteanx-endpoint-perl, libproc-guard-perl, libsub-quote-perl, libtest-mojibake-perl, libytnef, linux, lua-sql, node-graceful-readlink, node-invariant, node-rollup, node-socket.io-parser, node-timed-out, olefile, packaging-tutorial, pgrouting, pyparallel, python-coards, python-django-tagging, python-graphviz, python-irc, python-mechanicalsoup, python-persistent, python-scandir, python-stopit, r-cran-zelig, ruby-ast, ruby-whitequark-parser, sagetex & u-boot-menu.

Categories: LUG Community Blogs

Mick Morgan: variable substitution – redux

Mon, 30/01/2017 - 14:40

Back in October last year, I posted a note about the usage of variable substitution in lighttpd’s configuration files. In fact I got that post very slightly wrong (now corrected) in that I showed the test I applied in the file as: “$HTTP[“remoteip”] !~ “12.34.56.78″”. (Note the “!~” when I should have used “!=”). This works, in that it would limit access, but it is subtly wrong because it does not limit access in quite the way I intended. I only noticed this when I later came to change the variable assignment to allow access from three separate IP addresses (on which more later) rather than just one.

The “!~” operator is a perl style regular expression “not” match whilst the “!=” operator is the more strict string not equal match. This matters. My construct using the perl regex not wouldn’t actually just limit access solely to remote address 12.34.56.78 but would also allow in addresses of the form n12n.n34n.n56n.n78n where “n” is any other valid numeral (or none). So for example, my construct would have allowed in connections from 125.134.56.178 or 212.34.156.78 or 121.34.156.78 etc. That is not what I wanted at all.

The (correct) assignment and test now looks like this:

var.IP = “12\.34\.56\.78|23\.45\.67\.78|34\.56\.78\.90”

$HTTP[“remoteip”] !~ var.IP {
$HTTP[“url”] =~ “^/wp-admin/” {
url.access-deny = (“”)
}

Which says, allow connections from address 12.34.56.78 or 23.45.67.89 or 34.56.78.90 but no others.

For reference, the BNF like notation used in the basic configuration for lighty is given on the redmine wiki.

Categories: LUG Community Blogs

Mick Morgan: not welcome here

Mon, 30/01/2017 - 13:45

US President Trump has said that refugees and travellers from seven, mainly majority Muslim, countries are barred from entry to the US. Notwithstanding our own dear PM’s invitation to Trump, some 1.2 million brits have so far signed a Parliamentary petition to “Prevent Donald Trump from making a State Visit to the United Kingdom“.

I do not actually agree with the wording of the petition. As a republican at heart I really don’t care whether the Queen would be “embarrassed” by meeting Trump. Besides, she has met many, arguably worse, leaders in her time (think Robert Mugabe, or Nicolae Ceaușescu). The point here is that to extend an invitation to Trump so quickly, and whilst he is advocating such distateful and divisive policies gives the distinct impression that the UK endorses those policies. We do not, and should be seen to be highly critical of those policies. In her visit to the US last week, Theresa May made much of the UK’s ability as a close friend and partner of the USA to feel free to criticise that partner. She has done no such thing. In my view that is shameful.

When I signed the petition there were around 600,000 other signatories. The total is still climbing. That is encouraging. No doubt the petition will be ignored though.

(Postscript. El Reg has a discussion raging about the petition. Apparently I am a “virtue signaller”. Oh well, I don’t feel bad about it. After all, I’m a blogger so by definition I’m already self interested and vain.)

Categories: LUG Community Blogs

Jonathan McDowell: BelFOSS 2017

Sun, 29/01/2017 - 22:18

On Friday I attended the second BelFOSS conference. I’d spoken about my involvement with Debian at the conference last year, which seemed to be well received. This year I’d planned to just be a normal attendee, but ended up roped in at a late stage to be part of a panel discussing various licensing issues. I had a thoroughly enjoyable day - there were many great speakers, and plenty of opportunity for interesting chats with other attendees.

The conference largely happens through the tireless efforts of Jonny McCullagh, though of course many people are involved in bringing it together. It’s a low budget single day conference which has still managed to fill its single track attendee capacity both years, and attract more than enough speakers. Last year Red Hat and LPI turned up, this year Matt Curry from Allstate’s Arizona office appeared, but in general it’s local speakers talking to a local audience. This is really good to see - I don’t think Jonny would object at all if he managed to score a `big name’ speaker, but one of his aims is to get students interested and aware of Free Software, and I think it helps a lot that the conference allows them to see that it’s actively in use in lots of aspects of the industry here in Northern Ireland.

Here’s hoping that BelFOSS becomes an annual fixture in the NI tech calendar!

Categories: LUG Community Blogs

Jonathan McDowell: Experiments with 1-Wire

Tue, 24/01/2017 - 20:49

As previously mentioned, at the end of last year I got involved with a project involving the use of 1-Wire. In particular a DS28E15 device, intended to be used as a royalty tracker for a licensed piece of hardware IP. I’d no previous experience with 1-Wire (other than knowing it’s commonly used for driving temperature sensors), so I took it as an opportunity to learn a bit more about it.

The primary goal was to program a suitable shared key into the DS28E15 device that would also be present in the corresponding hardware device. A Maxim programmer had been ordered, but wasn’t available in stock so had to be back ordered. Of course I turned to my trusty Bus Pirate, which claimed 1-Wire support. However it failed to recognise the presence of the device at all. After much head scratching I finally listened to a co-worker who had suggested it was a clock speed issue - the absence of any option to select the 1-Wire speed in the Bus Pirate or any mention of different speeds in the documentation I had read had made me doubt it was an issue. Turns out that the Bus Pirate was talking “standard” 1-Wire and the DS28E15 only talks “overdrive” 1-Wire, to the extent that it won’t even announce its presence if the reset pulse conforms to the standard, rather than overdrive, reset time period. Lesson learned: listen to your co-workers sooner.

A brief period of yak shaving led to adding support to the Bus Pirate for the overdrive mode (since landed in upstream), and resulted in a search request via the BP interface correctly finding the device and displaying its ROM ID. This allowed exploration of the various commands the authenticator supports, to verify that the programming sequence operated as expected. These allow for setting the shared secret, performing a SHA256 MAC against this secret and a suitable nonce, and retrieving the result.

Next problem: the retrieved SHA256 MAC did not match the locally computed value. Initially endianness issues were suspected, but trying the relevant permutations did not help. Some searching found an implementation of SHA256 for the DS28E15 that showed differences between a standard SHA256 computation and what the authenticator performs. In particular SHA256 normally adds the current working state (a-g) to the current hash value (h0-h7) at the end of every block. The authenticator does this for all but the final block, where instead the hash value is set to the working state. I haven’t been able to find any documentation from Maxim that this is how things are calculated, nor have I seen any generic implementation of SHA256 which supports this mode. However rolling my own C implementation based on the code I found and using it to compare the results retrieved from the device confirms that this is what’s happening.

So at this point we’re done, right? Wait for the proper programming hardware to turn up, write the key to the devices, profit? Well, no. There was a bit of a saga involving the programmer (actually programmers, one with at least some documentation that allowed the creation of a Python tool to allow setting the key and reading + recording the ROM ID for tracking, and one with no programming documentation that came with a fancy GUI for manually doing the programming), but more importantly it was necessary to confirm that the programmed device interacted with the hardware correctly.

Initial testing with the hardware was unsuccessful. Again endianness issues were considered and permutations tried, but without success. A simple key constructed to avoid such issues was tried, and things worked fine. There was a hardware simulation of both components available, so it was decided to run that and obtain a capture of the traffic between them. As the secret key was known this would then allow the random nonce to be captured, and the corresponding (correct) hash value. Tests could then be performed in software to determine what the issue was & how to generate the same hash for verification.

Two sets of analyzer software were tried, OpenBench LogicSniffer (OLS) and sigrok. As it happened both failed to correctly decode the bitstream detected as 1-Wire, but were able to show the captured data graphically, allowing for decoding by eye. A slight patch to OLS to relax the timing constraints allowed it to successfully decode the full capture and provided the appropriate data for software reproduction. The end issue? A 256 bit number (as defined in VHDL) is not the same as 32 element byte array… Obvious when you know what the issue is!

So? What did I learn, other than a lot about 1-Wire? Firstly, don’t offhandedly discount suggestions that you don’t think make sense. Secondly, having a tool (in this case the Bus Pirate) that lets you easily play with a protocol via a simple interface is invaluable in understanding it. Thirdly, don’t trust manufacturers to be doing something in a normal fashion when they claim to be using a well defined technology. Fourthly, be conscious about all of the different ways bitstreams can be actually processed in memory. It’s not just endianness. Finally, spending the time to actually understand what’s going on up front can really help when things don’t work as you’d expect later on - without the yak shaving to support Overdrive on the BP I wouldn’t have been able to so quickly use the simulation capture to help diagnose the issue.

Categories: LUG Community Blogs

Steve Engledow (stilvoid): Angst

Sun, 22/01/2017 - 02:06

I had planned to spend this evening playing games; something I really enjoy doing but rarely set aside any time for. However, while we were eating dinner, I put some music on and it got me in the mood for playing some guitar. Over the course of dinner and playing with my son afterwards, that developed into wanting to write and record some music. I used to write electronic nonsense sometimes but this evening, I fancied trying my hand at some metal.

The first 90 minutes was - as almost every time I get the rare combination of an urge to do something musical and time to do it in - spent trying to remember how my setup worked, which bits of software I needed to install, and how to get the right combination of inputs and outputs I want. I eventually got it sussed and decided I'd better write it down for my own future reference.

Hardware
  1. Plug the USB audio interface from the V-Amp3 into the laptop.
  2. Plug external audio sources into the audio interface's input. (e.g. the V-Amp or a synth).
  3. Plug some headphones into the headphone socket of the audio interface.
  4. Switch on the audio interface's monitoring mode ;) (this kept me going for a little while; it's a small switch)
Software
  1. The following packages need to be installed at a minimum:

    • qjackctl
    • qsynth
    • soundfont-fluidsynth
    • vkeybd
    • ardour
    • hydrogen
  2. Use pavucontrol or similar to disable the normal audio system and just use the USB audio interface.

  3. Qjackctl needs the following snippets in its config for when jack comes up and goes down, respectively:

    • pacmd suspend true

      This halts pulseaudio so that jack can take over

    • pacmd suspend false

      This starts puseaudio back up again

  4. Use the connection tool in Jack to hook hydrogen's and qsynth's outputs to ardour's input. Use the ALSA tab to connect vkeybd to qsynth.

  5. When starting Ardour and Hydrogen, make sure they're both configured to use Jack for MIDI. Switch Ardour's clock from Internal to JACK.

For posterity, here's this evening's output.

Categories: LUG Community Blogs

Jonathan McDowell: Cloning a USB LED device

Sat, 14/01/2017 - 11:53

A month or so ago I got involved in a discussion on IRC about notification methods for a headless NAS. One of the options considered was some sort of USB attached LED. DealExtreme had a cheap “Webmail notifier”, which was already supported by mainline kernels as a “Riso Kagaku” device but it had been sold out for some time.

This seemed like a fun problem to solve with a tinyAVR and V-USB. I had my USB relay board so I figured I could use that to at least get some code to the point that the kernel detected it as the right device, and the relay output could be configured as one of the colours to ensure it was being driven in roughly the right manner. The lack of a full lsusb dump (at least when I started out) made things a bit harder, plus the fact that the Riso uses an output report unlike the relay code, which uses a control message. However I had the kernel source for the driver and with a little bit of experimentation had something which would cause the driver to be loaded and the appropriate files in /sys/class/leds/ to be created. The relay was then successfully activated when the red LED was supposed to be on.

hid-led 0003:1294:1320.0001: hidraw0: USB HID v1.01 Device [MAIL MAIL ] on usb-0000:00:14.0-6.2/input0 hid-led 0003:1294:1320.0001: Riso Kagaku Webmail Notifier initialized

I subsequently ordered some Digispark clones and modified the code to reflect the pins there (my relay board used pins 1+2 for USB, the Digispark uses pins 3+4). I then soldered a tricolour LED to the board, plugged it in and had a clone of the Riso Kaguku device for about £1.50 in parts (no doubt much cheaper in bulk). Very chuffed.

In case it’s useful to someone, the code is released under GPLv3+ and is available at https://the.earth.li/gitweb/?p=riso-kagaku-clone.git;a=summary or on GitHub at https://github.com/u1f35c/riso-kagaku-clone. I’m seeing occasional issues on an older Dell machine that only does USB2 with enumeration, but it generally is fine once it gets over that.

(FWIW, Jon, who started the original discussion, ended up with a BlinkStick Nano which is a neater device with 2 LEDs but still based on an Tiny85.)

Categories: LUG Community Blogs

Jonathan McDowell: 2016 in 50 Words

Fri, 06/01/2017 - 08:03

Idea via Roger. Roughly chronological order. Some things were obvious inclusions but it was interesting to go back and look at the year to get to the full 50 words.

Speaking at BelFOSS. Earthlings birthday. ATtiny hacking. Speaking at ISCTSJ. Dublin Anomaly. Co-habiting. DebConf. Peak Lion. Laura’s wedding. Christmas + picnic. Engagement. Car accident. Car write off. Tennent’s Vital. Dissertation. OMGWTFBBQ. BSides. New job. Rachel’s wedding. Digital Privacy talk. Graduation. All The Christmas Dinners. IMDB Top 250. Shay leaving drinks.

(This also serves as a test to see if I’ve correctly updated Planet Debian to use https and my new Hackergotchi that at least looks a bit more like I currently do.)

Categories: LUG Community Blogs

Jonathan McDowell: IMDB Top 250: Complete. Sort of.

Sat, 31/12/2016 - 16:01

Back in 2010, inspired by Juliet, I set about doing 101 things in 1001 days. I had various levels of success, but one of the things I did complete was the aim of watching half of the IMDB Top 250. I didn’t stop at that point, but continued to work through it at a much slower pace until I realised that through the Queen’s library I had access to quite a few DVDs of things I was missing, and that it was perfectly possible to complete the list by the end of 2016. So I did.

I should point out that I didn’t set out to watch the list because I’m some massive film buff. It was more a mixture of watching things that I wouldn’t otherwise choose to, and also watching things I knew were providing cultural underpinnings to films I had already watched and enjoyed. That said, people have asked for some sort of write up when I was done. So here are some random observations, which are almost certainly not what they were looking for.

My favourite film is not in the Top 250

First question anyone asks is “What’s your favourite film?”. That depends a lot on what I’m in the mood for really, but fairly consistently my answer is The Hunt for Red October. This has never been in the Top 250 that I’ve noticed. Which either says a lot about my taste in films, or the Top 250, or both. Das Boot was in the list and I would highly recommend it (but then I like all submarine movies it seems).

The Shawshank Redemption is overrated

I can’t recall a time when The Shawshank Redemption was not top of the list. It’s a good film, and I’ve watched it many times, but I don’t think it’s good enough to justify its seemingly unbroken run. I don’t have a suggestion for a replacement, however.

The list is constantly changing

I say I’ve completed the Top 250, but that’s working from a snapshot I took back in 2010. Today the site is telling me I’ve watched 215 of the current list. Last night it was 214 and I haven’t watched anything in between. Some of those are films released since 2010 (in particular new releases often enter high and then fall out of the list over a month or two), but the current list has films as old as 1928 (The Passion of Joan of Arc) that weren’t there back in 2010. So keeping up to date is not simply a matter of watching new releases.

The best way to watch the list is terrestrial TV

There were various methods I used to watch the list. Some I’d seen in the cinema when they came out (or was able to catch that way anyway - the QFT showed Duck Soup, for example). Netflix and Amazon Video had some films, but overall a very disappointing percentage. The QUB Library, as previously mentioned, had a good number of DVDs on the list (especially the older things). I ended up buying a few (Dial M for Murder on 3D Bluray was well worth it; it’s beautifully shot and unobtrusively 3D), borrowed a few from friends and ended up finishing off the list by a Lovefilm one month free trial. The single best source, however, was UK terrestrial TV. Over the past 6 years Freeview (the free-to-air service here) had the highest percentage of the list available. Of course this requires some degree of organisation to make sure you don’t miss things.

Films I enjoyed

Not necessarily my favourite, but things I wouldn’t have necessarily watched and was pleasantly surprised by. No particular order, and I’m leaving out a lot of films I really enjoyed but would have got around to watching anyway.

  • Clint Eastwood films - Gran Torino and Million Dollar Baby were both excellent but neither would have appealed to me at first glance. I hated Unforgiven though.
  • Jimmy Stewart. I’m not a fan of It’s a Wonderful Life (which I’d already watched because it’s Lister’s favourite film), but Harvey is obviously the basis of lots of imaginary friend movies and Rear Window explained a Simpsons episode (there were a lot of Simpsons episodes explained by watching the list).
  • Spaghetti Westerns. I wouldn’t have thought they were my thing, but I really enjoyed the Sergio Leone films (A Fistful of Dollars etc.). You can see where Tarantino gets a lot of his inspiration.
  • Foreign language films. I wouldn’t normally seek these out. And in general it seems I cannot get on with Italian films (except Life is Beautiful), but Amores Perros, Amelie and Ikiru were all better than expected.
  • Kind Hearts and Coronets. For some reason I didn’t watch this until almost the end; I think the title always put me off. Turned out to be very enjoyable.
Films I didn’t enjoy

I’m sure these mark me out as not being a film buff, but there are various things I would have turned off if I’d caught them by accident rather than setting out to watch them.

I’ve kept the full list available, if you’re curious.

Categories: LUG Community Blogs

Chris Lamb: Free software activities in December 2016

Sat, 31/12/2016 - 10:40

Here is my monthly update covering what I have been doing in the free software world (previous month):

  • Celebrated my 10-year anniversary of contributing to Debian. An excerpt of this post was quoted on LWN.
  • Made a number of improvements to AptFS, my FUSE-based filesystem that provides a view on unpacked Debian source packages as regular folders, including move from the popen2 Python module to subprocess and correcting the parsing of package lists.
  • Corrected an UnboundLocalError exception in the Finnish social security number generator in faker, a tool to generate test data in Python applications. (#441)
  • Made a small change to travis.debian.net (my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds on every code change) to fix an issue with malformed YAML.
  • Added the ability to specify the clone target to gbp-import-dsc etc. in git-buildpackage, a tool to build Debian packages using Git. (commit)
  • Filed three issues against the Redis key-value database:
    • Tests fail on the alpha architecture due to "memory efficiency". (#3666)
    • Please update hiredis (#3687)
    • Correct "whenever" typo. (#3652)
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users.

The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced — either maliciously or accidentally — during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

This month:

I also made the following changes to our tooling:

diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • Optimisations:
    • Avoid unnecessary string manipulation writing --text output (~20x speedup).
    • Avoid n iterations over archive files (~8x speedup).
    • Don't analyse .deb s twice when comparing .changes files (2x speedup).
    • Avoid shelling out to colordiff by implementing color support directly.
    • Memoize calls to distutils.spawn.find_executable to avoid excessive stat(1) syscalls.
  • Progress bar:
    • Show current file / ELF section under analysis etc. in progress bar.
    • Move the --status-fd output to use JSON and to include the current filename.
  • Code tidying:
    • Split out the try.diffoscope.org client so that it can be released separately on PyPI.
    • Completely rework the diffoscope and diffoscope.comparators modules, grouping similar utilities into their own modules, etc.
  • Miscellaneous:
    • Update dex_expected_diffs test to ensure compatibility with enjarify ≥ 1.0.3.
    • Ensure that running from Git will always use that checkout's Python modules.
    • Add a simple profiling framework.

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Makefile.PL: Change NAME argument to a Perl package name.
  • Ensure our binaries are available in autopkgtest tests.

try.diffoscope.org

trydiffoscope is a web-based version of the diffoscope in-depth and content-aware diff utility. Continued thanks to Bytemark for sponsoring the hardware.

  • Show progress bar and position in queue, etc. (#25 & #26)
  • Promote command-line client with PyPI instructions.
  • Increase comparison time limit to 90 seconds.

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Added support for version 0.2 .buildinfo files. (#15)

Debian Debian LTS

This month I have been paid to work 13½ hours on Debian Long Term Support (LTS). In that time I did the following:

  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 733-1 for openafs, fixing an information leak vulnerability. Due to incomplete initialization or clearing of reused memory, directory objects could contain 'dead' directory entry information.
  • Issued DLA 734-1 for mapserver closing an information leakage vulnerability.
  • Issued DLA 737-1 for roundcube preventing arbitrary remote code execution by sending a specially crafted email.
  • Issued DLA 738-1 for spip patching a cross-site scripting (XSS) vulnerability.
  • Issued DLA 740-1 for libgsf fixing a null pointer deference exploit via a crafted .tar file.
Debian Uploads
  • redis:
    • 3.2.5-5 — Add RunTimeDirectory=redis to systemd .service files.
    • 3.2.5-6 — Add missing Depends on lsb-base for /lib/lsb/init-functions usage in redis-sentinel's initscript.
    • 3.2.6-1 — New upstream release.
    • 4.0-1 & 4.0-rc2-1 — New upstream experimental releases.
  • aptfs: 0.9-1 & 0.10-1 — New upstream releases.
Debian bugs filed

I filed 29 FTBFS bugs against a7xpg, conntrack-tools, factory-boy, faker, glimpse, gunroar, hexchat-otr, jackson-datatype-guava, jalview, jquery, kodi-pvr-mythtv, leap-cli, libbio-graphics-perl, libparanoid-perl, libsass-python, metastudent-data, node-temporary, node-yargs, python-requests-unixsocket, python-restless, ruby-bunny, ruby-github-markup, ruby-rabl, sagenb-export, seaborn, soapdenovo2, titanion, ufw & vagrant-cachier.

I additionally filed 2 bugs for packages that access the internet during build against fence-agents & lua-geoip.

Debian FTP Team

As a Debian FTP assistant I ACCEPTed 107 packages: android-platform-libcore, compiz, debian-edu, dehydrated, dh-cargo, gnome-shell-extension-pixelsaver, golang-1.8, golang-github-btcsuite-btcd-btcec, golang-github-elithrar-simple-scrypt, golang-github-pelletier-go-toml, golang-github-restic-chunker, golang-github-weaveworks-mesh, golang-google-genproto, igmpproxy, jimfs, kpmcore, libbio-coordinate-perl, libdata-treedumper-oo-perl, libdate-holidays-de-perl, libpgobject-type-bytestring-perl, libspecio-library-path-tiny-perl, libterm-table-perl, libtext-hogan-perl, lighttpd, linux, linux-signed, llmnrd, lua-geoip, lua-sandbox-extensions, lua-systemd, node-cli-cursor, node-command-join, node-death, node-detect-indent, node-domhandler, node-duplexify, node-end-of-stream, node-first-chunk-stream, node-from2, node-glob-stream, node-has-binary, node-inquirer, node-interpret, node-is-negated-glob, node-is-unc-path, node-lazy-debug-legacy, node-lazystream, node-load-grunt-tasks, node-merge-stream, node-object-assign-sorted, node-orchestrator, node-pkg-up, node-resolve-from, node-resolve-pkg, node-rx, node-sorted-object, node-stream-shift, node-streamtest, node-string.prototype.codepointat, node-strip-bom-stream, node-through2-filter, node-to-absolute-glob, node-unc-path-regex, node-vinyl, openzwave, openzwave-controlpanel, pcb-rnd, pd-upp, pg-partman, postgresql-common, pybigwig, python-acora, python-cartopy, python-codegen, python-efilter, python-flask-sockets, python-intervaltree, python-jsbeautifier, python-portpicker, python-pretty-yaml, python-protobix, python-sigmavirus24-urltemplate, python-sqlsoup, python-tinycss, python-watson-developer-cloud, python-zc.customdoctests, python-zeep, r-cran-dbitest, r-cran-dynlm, r-cran-mcmcpack, r-cran-memoise, r-cran-modelmetrics, r-cran-plogr, r-cran-prettyunits, r-cran-progress, r-cran-withr, ruby-clean-test, ruby-gli, ruby-json-pure, ruby-parallel, rustc, sagemath, sbuild, scram, sidedoor, toolz & yabasic.

I additionally filed 4 RC bugs against packages that had incomplete debian/copyright files against jimfs, compiz, python-efilter & ruby-json-pure.

Categories: LUG Community Blogs

Chris Lamb: My favourite books of 2016

Fri, 30/12/2016 - 18:51

Whilst I managed to read almost sixty books in 2016 here are ten of my favourites in no particular order.

Disappointments this year include Stewart Lee's Content Provider (nothing like his stand-up), Christopher Hitchens' And Yet (his best essays are already published) and Heinlein's Stranger in a Strange Land (great exposition, bizarre conclusion).

The worst book I finished, by far, was Mark Edward's Follow You Home.






Animal QC

Gary Bell, QC

Subtitled My Preposterous Life, this rags-to-riches story about a working-class boy turned eminent lawyer would be highly readable as a dry and factual account but I am compelled to include it here for its extremely entertaining style of writing.

Full of unsurprising quotes that take one unaware: would you really expect a now-Queen's Counsel to "heartily suggest that if you find yourself suffering from dysentery in foreign climes you do not medicate it with lobster thermidor and a bottle of Ecuadorian red?"

A real good yarn.

So You've Been Publically Shamed

Jon Ronson

The author was initially recommended to me by Brad but I believe I started out with the wrong book. In fact, I even had my doubts about this one, prematurely judging from the title that it was merely cashing-in on a fairly recent internet phenomenon — like his more recent shallow take on Trump and the alt-Right — but in the end I read Publically Shamed thrice in quick succession.

I would particularly endorse the audiobook version: Ronson's deadpan drawl suits his writing perfectly.

The Obstacle is the Way

Ryan Holiday

Whilst everyone else appears to be obligated to include Ryan's recent Ego is the Enemy in their Best of 2016 lists I was actually taken by his earlier "introduction by stealth" to stoic philosophy.

Certainly not your typical self-help book, this is "a manual to turn to in troubling times".

Returning to this work at least three times over the year — even splashing out on the audiobook at some point — I feel like I learned a great deal, although it is now difficult to pinpoint exactly what. Perhaps another read in 2017 is thus in order…

Layer Cake

J.J. Connolly

To judge a book in comparison to the film is to do both a disservice, but reading the book of Layer Cake really underscored just how well the film played to the strengths of that medium.

All of the aspects that would not have worked had been carefully excised from the screenplay, ironically leaving more rewarding "layers" for readers attempting the book. A parallel adaption here might be No Country for Old Men - I would love to read (or write) a comparative essay between these two adaptions although McCarthy's novel is certainly the superior source material.

Lying

Sam Harris

I've absorbed a lot of Sam Harris's œuvre this year in the form of his books but moreover via his compelling podcast. I'm especially fond of Waking Up on spirituality without religion and would rank that as my favourite work of his.

Lying is a comparatively short read, more of a long essay in fact, where he argues that we can radically simplify our lives by merely telling the truth in situations where others invariably lie. Whilst it would take a brave soul to adopt his approach his case is superlatively well-argued and a delight to read.

Letters from a Stoic

Seneca

Great pleasure is to be found not only in keeping up an old and established friendship but also in beginning and building up a new one.

Reading this in a beautifully svelte hardback, I tackled a randomly-chosen letter per day rather than attempting to read it cover-to-cover. Breaking with a life-long tradition, I even decided to highlight sections in pen so I could return to them at ease.

I hope it's not too hackneyed to claim I gained a lot from "building up" a relationship with this book. Alas, it is one of those books that is too easy to recommend given that it might make one appear wise and learned, but if you find yourself in a slump, either in life or in your reading habits, it certainly has my approval.

Solo: A James Bond Novel

William Boyd

I must have read all of the canonical Fleming novels as a teenager and Solo really rewards anyone who has done so. It would certainly punish anyone expecting a Goldeneye or at least be a little too foreign to be enjoyed.

Indeed, its really a pastiche of these originals, both in terms of the time period, general tone (Bond is more somber; more vulnerable) and in various obsessions of Fleming's writing, such as the overly-detailed description of the gambling and dining tables. In this universe, 007's restaurant expenses probably contributed signifcantly to the downfall of the British Empire, let alone his waistline.

Bond flicking through a ornithological book at one point was a cute touch…

The Subtle Art of Not Giving A F*ck

Mark Manson

Certainly a wildcard to include here and not without its problems, The Subtle Art… is a curious manifesto on how to approach life. Whilst Manson expouses an age-old philosophy of grounding yourself and ignoring the accumulation of flatscreen TVs, etc. he manages to do so in a fresh and provocative "21st-centry gonzo" style.

Highly entertaining, at one point the author posits an alternative superhero ("Disappointment Panda") that dishes out unsolicited and uncomfortable truths to strangers before simply walking away: "You know, if you make more money, that’s not going to make your kids love you," or: "What you consider friendship is really just your constant attempts to impress people."

Ouch.

The Fourth Protocol

Frederick Forsyth

I have a crystal-clear memory from my childhood of watching a single scene from a film in the dead of night: Pierce Brosnan sets a nuclear device to detonate after he can get away but a double-crossing accomplice surreptitiously brings the timetable forward in order that the bomb also disposes of him…

Anyway, at some point whilst reading The Fourth Protocol it dawned on me that this was that book. I might thus be giving the book more credit due to this highly satisfying connection but I think it stands alone as a superlative political page-turner and is still approachable outside the machinations of the Cold War.

The Partner

John Grisham

After indulging in a bit too much non-fiction and an aborted attempt at The Ministry of Fear, I turned to a few so-called lower-brow writers such as Jeffrey Archer, etc.

However, it was The Partner that turned out to be a real page-turner for somewhat undefinable reasons. Alas, it appears the rest of the author's output is unfortunately in the same vein (laywers, etc.) so I am hesitant to immediately begin others but judging from various lists online I am glad I approached this one first.

Shogun: The First Novel of the Asian saga

James Clavell

Despite its length, I simply couldn't resist returning to Shogun this year although it did fatigue me to the point that I have still yet to commence on its sequel, Tai-Pan.

Like any good musical composition, one is always rewarded by returning to a book and I took great delight in uncovering more symbolism throughout (such as noticing that one of the first words Blackthorne learns in Japanese is "truth") but also really savouring the tragic arcs that run throughout the novel, some beautiful phrases ("The day seemed to lose its warmth…") and its wistful themes of inevitability and karma.

Categories: LUG Community Blogs

Jonathan McDowell: The terrible PIC ecosystem

Sun, 25/12/2016 - 00:51

I recently had call to play with some 1-Wire devices at work (more of which in a future post). It was taking a while for the appropriate programmer to turn up, so of course I pulled out my trusty BusPirate. It turned out the devices in question would only talk in overdrive mode, while the Bus Pirate could only offer standard mode. So I set about trying to figure out how to add the appropriate support.

This is is a huge endorsement for test equipment with Free Software firmware. Rather than giving up I was able to go and grab the current firmware, which has been adopted by the community since Dangerous Prototypes have discontinued development. What let me down was the ecosystem around the PIC24FJ64GA002.

My previous recent experience with microcontrollers has been with the ATTiny range and the STM32. Getting up and running with both of these was fairly easy - the tool chains necessary were already present in Debian, so all it took was a simple apt invocation to install everything I needed to compile code and program it to the devices.

Not so with the PIC series, which surprised me. There seems to be some basic support for the earlier PIC16 range, but for later chips there’s nothing that works out of the box with Debian. Investigation revealed that this was because there’s nothing maintained that enabled Free development for the PIC range. The accepted solution is the closed MPLAB X. Now, in one sense fair play to Microchip for making this available. But in another, shame on you. I can’t imagine ever choosing to build something based on a chip that only had a closed source tool chain available. I want things I can use in Makefiles and properly script, that are available in my distro of choice and that generally work in the same fashion as the tool chains I’m used to. I understand there might be some benefit in a closed compiler in terms of performance (and have HPC friends who would never trust a benchmark provided using GCC), but in general that’s not the space I move in. Nor does it seem to be the sort of attitude you should be taking if you are trying to attract the hobbyist and small production run market.

Any yet this seems common amongst hardware manufacturers. People whose core business is selling physical items, where the software is only relevant in terms of being able to use those items, seem to consider the software to be precious. Instead of opening up programming specifications and allowing a more widespread use of the hardware, increasing sales. I understand there are some cases where this isn’t practical, but the default attitude is definitely one of being closed rather than open, which is a terrible shame.

Anyway. I do have some Bus Pirate 1-Wire overdrive support now working (pending some testing to ensure standard mode still works), but I am glad I never spent a lot of time getting involved with PICs now.

Categories: LUG Community Blogs