Planet ALUG

Syndicate content
Planet ALUG -
Updated: 19 min 58 sec ago

Mick Morgan: do not click here

4 hours 33 min ago

I have just noticed that the getsafeonline campaign’s website contains this wonderfully ironic side bar graphic.

Go on, you know you want to.

Categories: LUG Community Blogs

Daniel Silverstone (Kinnison): Gitano - Approaching Release - Deprecated commands

13 hours 30 min ago

As mentioned previously I am working toward getting Gitano into Stretch. Last time we spoke about lace, on which a colleague and friend of mine (Richard Maw) did a large pile of work. This time I'm going to discuss deprecation approaches and building more capability out of fewer features.

First, a little background -- Gitano is written in Lua which is a deliberately small language whose authors spend more time thinking about what they can remove from the language spec than they do what they could add in. I first came to Lua in the 3.2 days, a little before 4.0 came out. (The authors provide a lovely timeline in case you're interested.) With each of the releases of Lua which came after 3.2, I was struck with how the authors looked to take a number of features which the language had, and collapse them into more generic, more powerful, smaller, fewer features.

This approach to design stuck with me over the subsequent decade, and when I began Gitano I tried to have the smallest number of core features/behaviours, from which could grow the power and complexity I desired. Gitano is, at its core, a set of files in a single format (clod) stored in a consistent manner (Git) which mediate access to a resource (Git repositories). Some of those files result in emergent properties such as the concept of the 'owner' of a repository (though that can simply be considered the value of the project.owner property for the repository). Indeed the concept of the owner of a repository is a fiction generated by the ACL system with a very small amount of collusion from the core of Gitano. Yet until recently Gitano had a first class command set-owner which would alter that one configuration value.

[gitano] set-description ---- Set the repo's short description (Takes a repo) [gitano] set-head ---- Set the repo's HEAD symbolic reference (Takes a repo) [gitano] set-owner ---- Sets the owner of a repository (Takes a repo)

Those of you with Gitano installations may see the above if you ask it for help. Yet you'll also likely see:

[gitano] config ---- View and change configuration for a repository (Takes a repo)

The config command gives you access to the repository configuration file (which, yes, you could access over git instead, but the config command can be delegated in a more fine-grained fashion without having to write hooks). Given the config command has all the functionality of the three specific set-* commands shown above, it was time to remove the specific commands.


If you had automation which used the set-description, set-head, or set-owner commands then you will want to switch to the config command before you migrate your server to the current or any future version of Gitano.

In brief, where you had:

ssh git@gitserver set-FOO repo something

You now need:

ssh git@gitserver config repo set project.FOO something

It looks a little more wordy but it is consistent with the other features that are keyed from the project configuration, such as:

ssh git@gitserver config repo set cgitrc.section Fooble Section Name

And, of course, you can see what configuration is present with:

ssh git@gitserver config repo show

Or look at a specific value with:

ssh git@gitserver config repo show specific.key

As always, you can get more detailed (if somewhat cryptic) help with:

ssh git@gitserver help config

Next time I'll try and touch on the new PGP/GPG integration support.

Categories: LUG Community Blogs

Mick Morgan: NFC? NFW

Sat, 22/10/2016 - 20:48

As is our custom on a Saturday, this morning my wife and I went out to a local cafe for breakfast. We know the proprietress so I was chatting to her whilst paying for the meal. Part way through the chat, the cafe proprietress tore off the receipt from the POS terminal and removed my debit card and handed it back to me.

Me: “Hang on, I haven’t entered my PIN. Are you sure that has been paid?”
CP: “Yes, it says here it’s paid.”
Me: “I have NOT authorised that transaction. It cannot be paid.”
CP: “Oh, don’t worry, we accept “swipe to pay” it probably just authorised that as you put your card in the terminal.”
Me: “That cannot happen. That card is not “swipe to pay” enabled. And I haven’t authorised any payment yet.”
CP (looking at receipt): “It says here “Contactless Sale” and the payment has been authorised”.
Me: “Show me that receipt.”

Sure enough, the receipt showed a “Contactless Sale” for the amount of the breakfast, however, the card type shown, and the last four digits of the card quoted were not those of my debit card. But I did recognise the card type as one I hold in my wallet so I checked that. Sure enough, that card has the WiFi symbol on it and the last four digits matched that on the Cafe receipt. So the POS terminal had taken the payment from a card in my wallet and not the card I had actually inserted.

That should not happen. And the fact that it did worries the hell out of me.

At the time the payment was taken, my wallet holding the other card was in my left hand (I had just removed my debit card from it with my right hand because I am right handed). So I placed that wallet on the counter beside me so that I could pick up the POS terminal in my left hand allowing me push my debit card in with my right hand and then enter my PIN. Replaying that action afterwards I am absolutely certain that at no time was my wallet anywhere nearer than a foot or more away from the POS terminal. Moreover that terminal had a card inserted – my debit card – and it should have been waiting for my PIN authorisation. So what happened?

I don’t know. And as I said above, that worries me.

I have checked both Wikipedia for details of the standards used in passive NFC of the type used in contactless payment and the “Security FAQ” for contactless payments on the Smart Card Alliance site (warning, PDF). Both those references tell me what I thought I already knew – NFC is only supposed to work at ranges of up to 2-4 inches (or 10 cm). No way was my wallet ever anywhere near 10 cm from that POS terminal. The closest it could have been was at least a foot away.

If this can happen to me, then I am certain it must have happened to others. Possibly to others who have been charged for someone else’s transaction simply because their NFC enabled card happens to be within range of the POS in question. In such cases, neither the actual customer nor the unwitting person really charged for the transaction would be any the wiser at the time of the transaction. Nor would the retailer know or care because they have a receipt for a contactless sale.

I’ll bet there have been some interesting conversations between such unwitting payers and their banks when the payment was noticed and then disputed.

Meanwhile, I’m going to find out whether I can get a card without the NFC capability to replace the card I unwittingly used to pay for breakfast. No way do I want this to happen again.

Categories: LUG Community Blogs

Mick Morgan: variable substitution in lighttpd

Wed, 19/10/2016 - 17:25

I’ve been a lighty user for many years now, having junked apache when it became obviously overweight for my target devices (the slugs in particular). Trivia is, of course, powered by lighty as are all my other websites.

Lighty’s configuration file syntax is reasonably simple to understand, and is well documented on the Redmine wiki. The guys at have also put together quite a nice introduction to lighty. If you haven’t tried it, and find that apache is becoming too much of a resource hog for you, I’d recommend that you give lighty a run.

I use lighty’s access control mechanisms to prevent random bots and bad guys from reaching trivia’s administrative functions and I do this in much the same way as I limit access to my ssh and openvpn daemons – I restrict access to the fixed IP address assigned to my router by my ISP. So in the lighty virtual host configuration file I use the following construct:

$HTTP[“remoteip”] !~ “” {
$HTTP[“url”] =~ “^/wp-admin/” {
url.access-deny = (“”)

That says: if the remote IP address is not, then deny access to the wp-admin directory.

Now I have several virtual hosts running and I also protect several directories. I also use a similar construct to redirect all my own access to my websites to port 443 so that I can always be certain that my own connection is encrypted and my authentication credentials will be protected. This means, of course, that I have several entries of the form: “if this IP address, then take this action” dotted around my configuration files. Not good. A recent change of ISP meant that my IP address has changed and I needed to edit my configuration files or find myself locked out. The most important files to change were my iptables rules so that I could still get ssh access on all my VMs. This didn’t take long because I have all the important configuration details (ssh IP addresses and ports, openvpn port, DNS addresses etc.) defined at the head of the bash script. One change is all that is necessary and bash variable substitution takes care of the rest. But my lighty configuration files were a different matter and I had to check carefully to ensure that I didn’t miss an important change. That’s just daft. Surely lighty allows for variable assignment and substitution. And of course it does, I just hadn’t checked before now.

The syntax looks like this:

At the head of the configuration file make an entry of the form:

# set our fixed remote ip address used in access control

IP = “”

and then change the earlier configuration lines to:

$HTTP[“remoteip”] !~ IP {
$HTTP[“url”] =~ “^/wp-admin/” {
url.access-deny = (“”)

Simple, and I feel a complete idiot for not noticing this before.

Categories: LUG Community Blogs

MJ Ray: Rinse and repeat

Tue, 18/10/2016 - 05:28

Forgive me, reader, for I have sinned. It has been over a year since my last blog post. Life got busy. Paid work. Another round of challenges managing my chronic illness. Cycle campaigning. Fun bike rides. Friends. Family. Travels. Other social media to stroke. I’m still reading some of the planets where this blog post should appear and commenting on some, so I’ve not felt completely cut off, but I am surprised how many people don’t allow comments on their blogs any more (or make it too difficult for me with reCaptcha and the like).

The main motive for this post is to test some minor upgrades, though. Hi everyone. How’s it going with you? I’ll probably keep posting short updates in the future.

Go in peace to love and serve the web.

Categories: LUG Community Blogs

Daniel Silverstone (Kinnison): Gitano - Approaching Release - Access Control Changes

Sat, 15/10/2016 - 04:11

As mentioned previously I am working toward getting Gitano into Stretch. A colleague and friend of mine (Richard Maw) did a large pile of work on Lace to support what we are calling sub-defines. These let us simplify Gitano's ACL files, particularly for individual projects.

In this posting, I'd like to cover what has changed with the access control support in Gitano, so if you've never used it then some of this may make little sense. Later on, I'll be looking at some better user documentation in conjunction with another friend of mine (Lars Wirzenius) who has promised to help produce a basic administration manual before Stretch is totally frozen.


With a more modern lace (version 1.3 or later) there is a mechanism we are calling 'sub-defines'. Previously if you wanted to write a ruleset which said something like "Allow Steve to read my repository" you needed:

define is_steve user exact steve allow "Steve can read my repo" is_steve op_read

And, as you'd expect, if you also wanted to grant read access to Jeff then you'd need yet set of defines:

define is_jeff user exact jeff define is_steve user exact steve define readers anyof is_jeff is_steve allow "Steve and Jeff can read my repo" readers op_read

This, while flexible (and still entirely acceptable) is wordy for small rulesets and so we added sub-defines to create this syntax:

allow "Steve and Jeff can read my repo" op_read [anyof [user exact jeff] [user exact steve]]

Of course, this is generally neater for simpler rules, if you wanted to add another user then it might make sense to go for:

define readers anyof [user exact jeff] [user exact steve] [user exact susan] allow "My friends can read my repo" op_read readers

The nice thing about this sub-define syntax is that it's basically usable anywhere you'd use the name of a previously defined thing, they're compiled in much the same way, and Richard worked hard to get good error messages out from them just in case.

No more auto_user_XXX and auto_group_YYY

As a result of the above being implemented, the support Gitano previously grew for automatically defining users and groups has been removed. The approach we took was pretty inflexible and risked compilation errors if a user was deleted or renamed, and so the sub-define approach is much much better.

If you currently use auto_user_XXX or auto_group_YYY in your rulesets then your upgrade path isn't bumpless but it should be fairly simple:

  1. Upgrade your version of lace to 1.3
  2. Replace any auto_user_FOO with [user exact FOO] and similarly for any auto_group_BAR to [group exact BAR].
  3. You can now upgrade Gitano safely.
No more 'basic' matches

Since Gitano first gained support for ACLs using Lace, we had a mechanism called 'simple match' for basic inputs such as groups, usernames, repo names, ref names, etc. Simple matches looked like user FOO or group !BAR. The match syntax grew more and more arcane as we added Lua pattern support refs ~^refs/heads/${user}/. When we wanted to add proper PCRE regex support we added a syntax of the form: user pcre ^/.+?... where pcre could be any of: exact, prefix, suffix, pattern, or pcre. We had a complex set of rules for exactly what the sigils at the start of the match string might mean in what order, and it was getting unwieldy.

To simplify matters, none of the "backward compatibility" remains in Gitano. You instead MUST use the what how with match form. To make this slightly more natural to use, we have added a bunch of aliases: is for exact, starts and startswith for prefix, and ends and endswith for suffix. In addition, kind of match can be prefixed with a ! to invert it, and for natural looking rules not is an alias for !is.

This means that your rulesets MUST be updated to support the more explicit syntax before you update Gitano, or else nothing will compile. Fortunately this form has been supported for a long time, so you can do this in three steps.

  1. Update your gitano-admin.git global ruleset. For example, the old form of the defines used to contain define is_gitano_ref ref ~^refs/gitano/ which can trivially be replaced with: define is_gitano_ref prefix refs/gitano/
  2. Update any non-zero rulesets your projects might have.
  3. You can now safely update Gitano

If you want a reference for making those changes, you can look at the Gitano skeleton ruleset which can be found at or in /usr/share/gitano if Gitano is installed on your local system.

Next time, I'll likely talk about the deprecated commands which are no longer in Gitano, and how you'll need to adjust your automation to use the new commands.

Categories: LUG Community Blogs

Daniel Silverstone (Kinnison): Gitano - Approaching Release - Changes

Fri, 14/10/2016 - 14:30

Continuing on from the previous article, here is a (probably incomplete) list of the critical changes to Gitano which have been, or will be, worked on during the run toward a 1.0 release. Each of these will have a blog posting to discuss what the changes mean for current and future users. Sometimes I'll aggregate postings, sometimes I won't.

The following are some highlights from the past little while of development which has been undertaken by Richard and myself. Each item is, I feel, important enough to warrant commentary, even for those who already use Gitano.

  • Lace now supports a sub-define syntax: [foo bar] which makes for simpler rulesets.
  • Gitano no longer creates auto_user_XXX and auto_group_XXX Lace predicates
  • Gitano no longer supports "basic" simple matches of the form user foo but instead requires a match kind such as group prefix bar-.
  • Gitano is gaining i18n/l10n support, though it will not be complete for version 1.0 the basics will be in place.
  • Gitano is gaining a much larger integration test suite using yarn.
  • Deprecated commands have now been removed from Gitano. (e.g. no more set-owner)
  • Gitano has gained PGP/GPG signature verification for commits and tags.

Any number of smaller things have been done which fall below some arbitrary barrier for telling you about. If you're aware of any of them and feel they are worthwhile telling the world about, then please prod me and I'll add an article to the series.

Finally it's worth noting that the effort to get all this into Debian Stretch proceeds apace. Of the eight packages needed, at the time of posting: one was already in and has been updated (luxio), three have been accepted into Debian already (supple, clod, lua-scrypt), two are in NEW (gall and lace), and that leaves the newest library (tongue) and then Gitano itself still to go. The Debian FTP team have been awesome in helping me with all this, so thanks go to them.

Categories: LUG Community Blogs

Daniel Silverstone (Kinnison): Gitano - Approaching Release - Work

Tue, 04/10/2016 - 05:41

I have been working quite hard, along with my friend and colleague Richard Maw, on getting Gitano ready for a release suitable for inclusion into Debian Stretch.

You can see how we're doing on the various Trello boards for:

As Richard and I work toward a version of Gitano we're prepared to support long-term in Debian we are making many changes to make our lives easier. For those of you who have been using Gitano over the past few years, you'll need to pay attention to some postings which will be coming soon about how to make the changes you need so as to not explode horribly when you upgrade to the version we're releasing soon. For those of you who are not yet using Gitano but feel like you might want to; I'll also be producing some postings about getting started with the packages. And for those happily running current HEAD of Gitano already, I'll be posting about some of the new features over the next little while in case you're not aware of them.

IMPORTANT: If you're using Gitano already and have any issues or feature requests then please please please let me know ASAP otherwise they're unlikely to be resolved/implemented before 1.0. irl already asked for the facility to verify GPG signed commits and tags, but if you want anything else considering then I need to know v. soon. (Ideally email me, but you may comment on this posting too if you must)

Categories: LUG Community Blogs

Chris Lamb: Free software activities in September 2016

Fri, 30/09/2016 - 22:44

Here is my monthly update covering what I have been doing in the free software world (previous month):

Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most Linux distributions provide binary (or "compiled") packages to end users.

The motivation behind the Reproducible Builds effort is to allow verification that no flaws have been introduced — either maliciously and accidentally — during this compilation process by promising identical binary packages are always generated from a given source.

My work in the Reproducible Builds project was also covered in our weekly reports #71, #72, #71 & #74.

I made the following improvements to our tools:


diffoscope is our "diff on steroids" that will not only recursively unpack archives but will transform binary formats into human-readable forms in order to compare them.

  • Added a global Progress object to track the status of the comparison process allowing for graphical and machine-readable status indicators. I also blogged about this feature in more detail.
  • Moved the global Config object to a more Pythonic "singleton" pattern and ensured that constraints are checked on every change.


disorderfs is our FUSE filesystem that deliberately introduces nondeterminism into the results of system calls such as readdir(3).

  • Display the "disordered" behaviour we intend to show on startup. (#837689)
  • Support relative paths in command-line parameters (previously only absolute paths were permitted).


strip-nondeterminism is our tool to remove specific information from a completed build.

  • Fix an issue where temporary files were being left on the filesystem and add a test to avoid similar issues in future. (#836670)
  • Print an error if the file to normalise does not exist. (#800159)
  • Testsuite improvements:
    • Set the timezone in tests to avoid a FTBFS and add a File::StripNondeterminism::init method to the API to to set tzset everywhere. (#837382)
    • "Smoke test" the strip-nondeterminism(1) and dh_strip_nondeterminism(1) scripts to prevent syntax regressions.
    • Add a testcase for .jar file ordering and normalisation.
    • Check the stripping process before comparing file attributes to make it less confusing on failure.
    • Move to a lookup table for descriptions of stat(1) indices and use that for nicer failure messages.
    • Don't uselessly test whether the inode number has changed.
  • Run perlcritic across the codebase and adopt some of its prescriptions including explicitly using oct(..) for integers with leading zeroes, avoiding mixing high and low-precedence booleans, ensuring subroutines end with a return statement, etc.

I also submitted 4 patches to fix specific reproducibility issues in golang-google-grpc, nostalgy, python-xlib & torque.

Patches contributed Debian LTS

This month I have been paid to work 12.75 hours on Debian Long Term Support (LTS). In that time I did the following:

  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 608-1 for mailman fixing a CSRF vulnerability.
  • Issued DLA 611-1 for jsch correcting a path traversal vulnerability.
  • Issued DLA 620-1 for libphp-adodb patching a SQL injection vulnerability.
  • Issued DLA 631-1 for unadf correcting a buffer underflow issue.
  • Issued DLA 634-1 for dropbear fixing a buffer overflow when parsing ASN.1 keys.
  • Issued DLA 635-1 for dwarfutils working around an out-of-bounds read issue.
  • Issued DLA 638-1 for the SELinux policycoreutils, patching a sandbox escape issue.
  • Enhanced Brian May's find-work --unassigned switch to take an optional "except this user" argument.
  • Marked matrixssl and inspircd as being unsupported in the current LTS version.
  • python-django 1:1.10.1-1 — New upstream release and ensure that django-admin startproject foo creates files with the correct shebang under Python 3.
  • gunicorn:
    • 19.6.0-5 — Don't call chown(2) if it would be a no-op to avoid failure under snap.
    • 19.6.0-6 — Remove now-obsolete conffiles and logrotate scripts; they should have been removed in 19.6.0-3.
  • redis:
    • 3.2.3-2 — Call ulimit -n 65536 by default from SysVinit scripts to normalise the behaviour with systemd. I also bumped the Debian package epoch as the "2:" prefix made it look like we are shipping version 2.x. I additionaly backported this upload to Debian Jessie.
    • 3.2.4-1 — New upstream release, add missing -ldl for dladdr(3) & add missing dependency on lsb-base.
  • python-redis (2.10.5-2) — Bump python-hiredis to Suggests to sync with Ubuntu and move to a machine-readable debian/copyright. I also backported this upload to Debian Jessie.
  • adminer (4.2.5-3) — Move mysql-server dependencies to default-mysql-server. I also backported this upload to Debian Jessie.
  • gpsmanshp (1.2.3-5) on behalf of the QA team:
    • Move to "minimal" debhelper style, making the build reproducible. (#777446 & #792991)
    • Reorder linker command options to build with --as-needed (#729726) and add hardening flags.
    • Move to machine-readable copyright file, add missing #DEBHELPER# tokens to postinst and prerm scripts, tidy descriptions & other debian/control fields and other smaller changes.

I sponsored the upload of 5 packages from other developers:

I also NMU'd:

RC bugs

I filed 37 FTBFS bugs against csoundqt, cups-filters, dymo-cups-drivers, easytag, erlang-p1-oauth2, erlang-p1-sqlite3, erlang-p1-xmlrpc, erlang-redis-client, fso-datad, gnome-python-desktop, gnote, gstreamermm-1.0, gtkglextmm, gupnp-dlna, haskell-hmatrix-gsl, jdeb, kryo-serializers, libcmrt, libfso-glib, libmonitoring-livestatus-perl, librasterlite2, network-manager, print-manager, psychtoolbox-3, python-3to2, python-tidylib, recutils, slang2, snd, sugar, tj3, transmission-remote-gtk, vino, webkit2pdf, xml-core, xml-core & xml-core.

I additionally filed 2 "important" bugs for packages that access the internet during build against gnupg2 & libgdata.

FTP Team

As a Debian FTP assistant I ACCEPTed 147 packages: alljoyn-services-1604, android-platform-external-doclava, android-platform-system-tools-aidl, aufs, bcolz, binwalk, bmusb, bruteforce-salted-openssl, cappuccino, captagent, chrome-gnome-shell, ciphersaber, cmark, colorfultabs, cppformat, dnsrecon, dogtag-pki, dxtool, e2guardian, flask-compress, fonts-mononoki, fwknop-gui, gajim-httpupload, glbinding, glewmx, gnome-2048, golang-github-googleapis-proto-client-go, google-android-installers, gsl, haskell-hmatrix-gsl, haskell-relational-query, haskell-relational-schemas, haskell-secret-sharing, hindsight, i8c, ip4r, java-string-similarity, khal, khronos-opencl-headers, liblivemedia, libshell-config-generate-perl, libshell-guess-perl, libstaroffice, libxml2, libzonemaster-perl, linux, linux-grsec-base, linux-signed, lua-sandbox, lua-torch-trepl, mbrola-br2, mbrola-br4, mbrola-de1, mbrola-de2, mbrola-de3, mbrola-ir1, mbrola-lt1, mbrola-lt2, mbrola-mx1, mimeo, mimerender, mongo-tools, mozilla-gnome-keyring, munin, node-grunt-cli, node-js-yaml, nova, open-build-service, openzwave, orafce, osmalchemy, pgespresso, pgextwlist, pgfincore, pgmemcache, pgpool2, pgsql-asn1oid, postbooks-schema, postgis, postgresql-debversion, postgresql-multicorn, postgresql-mysql-fdw, postgresql-unit, powerline-taskwarrior, prefix, pycares, pydl, pynliner, pytango, pytest-cookies, python-adal, python-applicationinsights, python-async-timeout, python-azure, python-azure-storage, python-blosc, python-can, python-canmatrix, python-chartkick, python-confluent-kafka, python-jellyfish, python-k8sclient, python-msrestazure, python-nss, python-pytest-benchmark, python-tenacity, python-tmdbsimple, python-typing, python-unidiff, python-xstatic-angular-schema-form, python-xstatic-tv4, quilt, r-bioc-phyloseq, r-cran-filehash, r-cran-png, r-cran-testit, r-cran-tikzdevice, rainbow-mode, repmgr, restart-emacs, restbed, ruby-azure-sdk, ruby-babel-source, ruby-babel-transpiler, ruby-diaspora-prosody-config, ruby-haikunator, ruby-license-finder, ruby-ms-rest, ruby-ms-rest-azure, ruby-rails-assets-autosize, ruby-rails-assets-blueimp-gallery, ruby-rails-assets-bootstrap, ruby-rails-assets-bootstrap-markdown, ruby-rails-assets-emojione, ruby-sprockets-es6, ruby-timeliness, rustc, skytools3, slony1-2, snmp-mibs-downloader, syslog-ng, test-kitchen, uctodata, usbguard, vagrant-azure, vagrant-mutate & vim.

Categories: LUG Community Blogs

Chris Lamb: Diffoscope progress bar

Wed, 28/09/2016 - 12:45

Diffoscope is a diff utility which recursively unpacks archives, ISOs, etc., transforming a wide variety of files into human-readable forms before comparison instead of simply showing the raw difference in hexadecimal.

I recently added a progress bar when diffoscope is run on a terminal:

Note that as diffoscope can, at any point, encounter an archive or format that requires unpacking, the progress will always be approximate and may even appear to go "backwards".

The implementation, available in version 61, is simple (see #1, #2, #3 & #4) but takes into account of a number of subtleties by using context managers to correctly track the state throughout.

Categories: LUG Community Blogs

Chris Lamb: How to write your first Lintian check

Mon, 05/09/2016 - 16:33

Lintian's humble description of "Debian package checker" belies its importance within the Debian GNU/Linux project. An extensive static analysis tool, it's not only used by the vast majority of developers, falling foul of some of its checks even cause uploads to be automatically rejected by the archive maintenance software.

As you may have read in my recent monthly report, I've recently been hacking on Lintian itself. In particular:

  • #798983: Check for libjs-* binary package name outside of the web section
  • #814326: Warn if filenames contain wildcard characters
  • #829744: Add new-package-should-not-package-python2-module tag
  • #831864: Warn about Python packages that ship information
  • #832096 Check for common typos in debian/rules target names
  • #832099: Check for unnecessary SOURCE_DATE_EPOCH assignments
  • #832771: Warn about systemd .service files with a missing Install key

However, this rest of this post will go through the steps needed to start contributing yourself.

To demonstrate this I will be walking through submitting a patch for bug #831864 which warns about Python packages that ship .coverage files generated by

Getting started

First, let's obtain the Lintian sources and create a branch for our work:

$ git clone […] $ cd lintian $ git checkout -b warn-about-dotcoverage-files Switched to a new branch 'warn-about-dotcoverage-files'

The most interesting files are under checks/*:

$ ls -l checks/ | head -n 9 total 1356 -rw-r--r-- 1 lamby lamby 6393 Jul 29 14:19 apache2.desc -rw-r--r-- 1 lamby lamby 8619 Jul 29 14:19 -rw-r--r-- 1 lamby lamby 1956 Jul 29 14:19 application-not-library.desc -rw-r--r-- 1 lamby lamby 3285 Jul 29 14:19 -rw-r--r-- 1 lamby lamby 544 Jul 29 14:19 automake.desc -rw-r--r-- 1 lamby lamby 1354 Jul 29 14:19 -rw-r--r-- 1 lamby lamby 19506 Jul 29 14:19 binaries.desc -rw-r--r-- 1 lamby lamby 25204 Jul 29 14:19 -rw-r--r-- 1 lamby lamby 15641 Aug 24 21:42 changelog-file.desc -rw-r--r-- 1 lamby lamby 19606 Jul 29 14:19

Note that the files are in pairs; a foo.desc file that contains description of the tags and a sibling Perl module that actually performs the checks.

Let's add our new tag before we go any further. After poking around, it looks like files.{pm,desc} would be most appropriate, so we'll add our new tag definition to files.desc:

Tag: package-contains-python-coverage-file Severity: normal Certainty: certain Info: The package contains a file that looks like output from the Python tool. These are generated by python{,3}-coverage during a test run, noting which parts of the code have been executed. They can then be subsequently analyzed to identify code that could have been executed but was not. . As they are are unlikely to be of utility to end-users, these files should be removed from the package.

The Severity and Certainty fields are documented in the manual. Note the convention of using double spaces after full stops in the Info section.

Extending the testsuite

Lintian has many moving parts based on regular expressions and other subtle logic, so it's especially important to provide tests in order to handle edge cases and to catch any regressions in the future.

We create tests by combining a tiny Debian package that will deliberately violate our check, along with some metadata and the expected output of running Lintian against this package.

The tests themselves are stored under t/tests. There may be an existing test that it would be more appropriate to extend, but I've gone with creating a new directory called files-python-coverage:

$ mkdir -p t/tests/files-python-coverage $ cd t/tests/files-python-coverage

First, we create a simple package, installing dummy file to trigger the check:

$ mkdir -p debian/debian $ touch debian/.coverage $ echo ".coverage /usr/share/files-python-coverage" > debian/debian/install

Note that we do not need a debian/rules file as long as we do not deviate from a "skeleton" debhelper style. We then add the aforementioned metadata to t/tests/files-python-coverage/desc:

Testname: files-python-coverage Sequence: 6000 Version: 1.0 Description: Check for Python .coverage files Test-For: package-contains-python-coverage-file

… and the expected warning to t/tests/files-python-coverage/tags:

$ echo "W: files-python-coverage: package-contains-python-coverage-file" \ "usr/share/files-python-coverage/.coverage" > tags

When we run the testsuite, it should fail because we don't emit the check yet:

$ cd $(git rev-parse --show-toplevel) $ debian/rules runtests onlyrun=tag:package-contains-python-coverage-file […] --- t/tests/files-python-coverage/tags +++ debian/test-out/tests/files-python-coverage/tags.files-python-coverage @@ -1 +0,0 @@ -W: files-python-coverage: package-contains-python-coverage-file usr/share/files-python-coverage/.coverage fail tests::files-python-coverage: output differs! Failed tests (1) tests::files-python-coverage debian/rules:48: recipe for target 'runtests' failed make: *** [runtests] Error 1 $ echo $? 1

Specifying onlyrun= means we only run the tests that are designed to trigger this tag rather than the whole testsuite. This is controlled by the Test-For key in our desc file, not by scanning the tags files.

This recipe for creating a testcase could be used when submitting a regular bug against Lintian — providing a failing testcase not only clarifies misunderstandings resulting from the use of natural language, it also makes it easier, quicker and safer to correct the offending code itself.

Emitting the tag

Now, let's actually implement the check:

tag 'package-installs-python-egg', $file; } + # ---------------- .coverage ( output) + if ($file->basename eq ".coverage") { + tag 'package-contains-python-coverage-file', $file; + } # ---------------- /usr/lib/site-python

Our testsuite now passes:

$ debian/rules runtests onlyrun=tag:package-contains-python-coverage-file private/ .... running tests .... mkdir -p "debian/test-out" t/runtests -k -j 9 t "debian/test-out" tag:package-contains-python-coverage-file ENV[PATH]=[..] pass tests::files-python-coverage if [ "tag:package-contains-python-coverage-file" = "" ]; then touch runtests; fi $ echo $? 0 Submitting the patch

Lastly, we create a patch for submission to the bug tracking system:

$ git commit -a -m "c/files: Warn about Python packages which ship" \ " information. (Closes: #831864)" $ git format-patch HEAD~ 0001-c-files-Warn-about-Python-packages-which-ship-covera.patch

… and we finally attach it to the existing bug:

To: Cc: Bcc: tags 831864 + patch thanks Patch attached. /lamby

I hope this post will encourage at some extra contributions towards this important tool.

(Be aware that I'm not a Lintian maintainer, so not only should you not treat anything here as gospel and expect this post may be edited over time if clarifications arise.)

Categories: LUG Community Blogs

Chris Lamb: Free software activities in August 2016

Wed, 31/08/2016 - 22:48

Here is my monthly update covering what I have been doing in the free software world (previously):

  • Worked on nsntrace, a userspace tool to perform network traces on processes using kernel namespaces:
    • Overhauled error handling to ensure the return code of the wrapped process is returned to the surrounding environment. (#10).
    • Permit the -u argument to also accept uids as well as usernames. (#16).
    • Always kill the (hard-looping) udp_send utility, even on test failures. (#13).
    • Updated to look for iptables in /sbin & /usr/sbin (#11) and to raise an error if pcap.h is missing (#15).
    • Drop bashisms in #!/bin/sh script (#14) and ignore the generated manpage in the Git repository (#12).
  • Independently discovered an regression in the Django web development framework where field__isnull=False filters were not working with some foreign keys, resulting in extending the testsuite and release documentation. (#7104).
  • Proposed a change to django-enumfield (a custom field for type-safe constants) to ensure passing a string type to Enum.get returned None on error to match the documentation. (#36).
  • Fixed an issue in the Mopidy music player's podcast extension where the testsuite was failing tests in extreme timezones. (#40).
  • Proposed changes to make various upstream's reproducible:
    • botan, a crypto/TLS library for C++11. (#587).
    • cookiecutter, a project template generator, removing nondeterministic keyword arguments from appearing in the documentation. (#800).
    • pyicu, a Python wraper for the IBM Unicode library. (#27).
  • Integrated a number of issues raised by @piotr1212 to python-fadvise, my Python interface to posix_fadvise(2), where the API was not being applied to open file descriptors (#1) and moving the .so to a module directory (#2).
  • Various improvements to, a hosted version of the diffoscope in-depth and content-aware diff utility, including introducing an HTTP API (#21), updating the SSL certificate and correcting a logic issue where errors in diffoscope itself were not being detected correctly (b0ff49). Continued thanks to Bytemark for sponsoring the hardware.
  • Fixed a bug in django-slack, my library to easily post messages to the Slack group-messaging utility, correcting an EncodeError exception under Python 3 (#53) and updated the minimum required version of Django to 1.7 (#54).
  • Various updates to tickle-me-email, my Getting Things Done-inspired email toolbox, to also match / in IMAP's LIST separators (#6) and to encode the folder list as UTF-7 (#7). Thanks to @resiak.
  • Clarified the documentation for — my hosted script to easily test and build Debian packages on the Travis CI continuous integration platform — regarding how to integrate with Github (#20).

Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most Linux distributions provide binary (or "compiled") packages to end users.

The motivation behind the Reproducible Builds effort is to allow verification that no flaws have been introduced — either maliciously and accidentally — during this compilation process by promising identical binary packages are always generated from a given source.

Toolchain issues

I submitted the following patches to fix reproducibility-related toolchain issues:

My work in the Reproducible Builds project was also covered in our weekly reports. (#67, #68, #69, #70).


diffoscope is our "diff on steroids" that will not only recursively unpack archives but will transform binary formats into human-readable forms in order to compare them:

  • Added a command-line interface to the web service.
  • Added a JSON comparator.
  • In the HTML output, highlight lines when hovering to make it easier to visually track.
  • Ensure that we pass str types to our Difference class, otherwise we can't be sure we can render them later.
  • Testsuite improvements:
    • Generate test coverage reports.
    • Add tests for Haskell and GitIndex comparators.
    • Completely refactored all of the comparator tests, extracting out commonly-used routines.
    • Confirm rendering of text and HTML presenters when checking non-existing files.
    • Dropped a squashfs test as it was simply too unreliable and/or has too many requirements to satisfy.
  • A large number of miscellaneous cleanups, including:
    • Reworking the comparator setup/preference internals by dynamically importing classes via a single list.
    • Split exceptions out into dedicated diffoscope.exc module.
    • Tidying the PROVIDERS dict in diffoscope/
    • Use html.escape over xml.sax.saxutils.escape, cgi.escape, etc.
    • Removing hard-coding of manual page targets names in debian/rules.
    • Specify all string format arguments as logging function parameters, not using interpolation.
    • Tidying imports, correcting indentation levels and drop unnecessary whitespace.


disorderfs is our FUSE filesystem that deliberately introduces nondeterminism in system calls such as readdir(3).

  • Added a testsuite to prevent regressions. (f124965)
  • Added a --sort-dirents=yes|no option for forcing deterministic ordering. (2aae325)

  • Improved strip-nondeterminism, our tool to remove specific nondeterministic information after a build:
    • Match more styles of Java .properties files.
    • Remove hyphen from "non-determinism" and "non-deterministic" throughout package for consistency.
  • Improvements to our testing infrastucture:
    • Improve the top-level navigation so that we can always get back to "home" of a package.
    • Give expandable elements cursor: pointer CSS styling to highlight they are clickable.
    • Drop various trailing underlined whitespaces after links.
    • Explicitly log that build was successful or not.
    • Various code-quality improvements, including prefering str.format over concatentation.
  • Miscellaneous updates to our filter-packages internal tool:
    • Add --random=N and --url options.
    • Add support for --show=comments.
    • Correct ordering so that --show-version runs after --filter-ftbfs.
    • Rename --show-ftbfs to --filter-ftbfs and --show-version to --show=version.
  • Created a proof-of-concept reproducible-utils package to contain commonly-used snippets aimed at developers wishing to make their packages reproducible.

I also submitted 92 patches to fix specific reproducibility issues in advi, amora-server, apt-cacher-ng, ara, argyll, audiotools, bam, bedtools, binutils-m68hc1x, botan1.10, broccoli, congress, cookiecutter, dacs, dapl, dateutils, ddd, dicom3tools, dispcalgui, dnssec-trigger, echoping, eekboek, emacspeak, eyed3, fdroidserver, flashrom, fntsample, forkstat, gkrellm, gkrellm, gnunet-gtk, handbrake, hardinfo, ircd-irc2, ircd-ircu, jack-audio-connection-kit, jpy, kxmlgui, libbson, libdc0, libdevel-cover-perl, libfm, libpam-ldap, libquvi, librep, lilyterm, mozvoikko, mp4h, mp4v2, myghty, n2n, nagios-nrpe, nikwi, nmh, nsnake, openhackware, pd-pdstring, phpab, phpdox, phpldapadmin, pixelmed-codec, pleiades, pybit, pygtksourceview, pyicu, python-attrs, python-gflags, quvi, radare2, rc, rest2web, roaraudio, rt-extension-customfieldsonupdate, ruby-compass, ruby-pg, sheepdog, tf5, ttf-tiresias, ttf-tiresias, tuxpaint, tuxpaint-config, twitter-bootstrap3, udpcast, uhub, valknut, varnish, vips, vit, wims, winswitch, wmweather+ & xshisen.

Debian GNU/Linux Patches contributed

I also submitted 22 patches to fix typos in debian/rules files against ctsim, f2c, fonts-elusive-icons, ifrit, ldapscripts, libss7, libvmime, link-grammar, menulibre, mit-scheme, mugshot, nlopt, nunit, proftpd-mod-autohost, proftpd-mod-clamav, rabbyt, radvd, ruby-image-science, snmpsim, speech-tools, varscan & whatmaps.

Debian LTS

This month I have been paid to work 15 hours on Debian Long Term Support (LTS). In that time I did the following:

  • "Frontdesk" duties, triaging CVEs, etc.
  • Authored the patch & issued DLA 596-1 for extplorer, a web-based file manager, fixing an archive traversal exploit.
  • Issued DLA 598-1 for suckless-tools, fixing a segmentation fault in the slock screen locking tool.
  • Issued DLA 599-1 for cracklib2, a pro-active password checker library, fixing a stack-based buffer overflow when parsing large GECOS fields.
  • Improved the find-work internal tool adding optional colour highlighting and migrating it to Python 3.
  • Wrote an lts-missing-uploads tool to find mistakes where there was no correponding package in the archive after an announcement.
  • Added optional colour highlighting to the lts-cve-triage tool.
  • redis 2:3.2.3-1 — New upstream release, move to the DEP-5 debian/copyright format, ensure that we are running as root in LSB initscripts and add a README.Source regarding our local copies of redis.conf and sentinel.conf.
  • python-django:
    • 1:1.10-1 — New upstream release.
    • 1:1.10-2 — Fix test failures due to mishandled upstream translation updates.

  • gunicorn:
    • 19.6.0-2 — Reload logrotate in the postrotate action to avoid processes writing to the old files and move to DEP-5 debian/copyright format.
    • 19.6.0-3 — Drop our /usr/sbin/gunicorn{,3}-debian and related Debian-specific machinery to be more like upstream.
    • 19.6.0-4 — Drop "template" systemd .service files and point towards examples and documentation instead.

  • adminer:
    • 4.2.5-1 — Take over package maintenance, completely overhauling the packaging with a new upstream version, move to virtual-mysql-server to support MariaDB, updating package names of dependencies and fix the outdated Apache configuration.
    • 4.2.5-2 — Correct the php5 package names.

Bugs filed (without patches) RC bugs

I filed 3 RC bugs with patches:

I additionally filed 8 RC bugs for packages that access the internet during build against autopkgtest, golang-github-xenolf-lego, pam-python, pexpect, python-certbot, python-glanceclient, python-pykka & python-tornado.

I also filed 74 FTBFS bugs against airlift-airline, airlift-slice, alter-sequence-alignment, apktool, atril, auto-apt-proxy, bookkeeper, bristol, btfs, caja-extensions, ccbuild, cinder, clustalo, colorhug-client, cpp-netlib, dimbl, edk2, elasticsearch, ganv, git-remote-hg, golang-codegangsta-cli, golang-goyaml, gr-radar, imagevis3d, jacktrip, jalv, kdepim, kiriki, konversation, libabw, libcereal, libdancer-plugin-database-perl, libdist-zilla-plugins-cjm-perl, libfreemarker-java, libgraph-writer-dsm-perl, libmail-gnupg-perl, libminc, libsmi, linthesia, lv2-c++-tools, lvtk, mate-power-manager, mcmcpack, mopidy-podcast, nageru, nfstrace, nova, nurpawiki, open-gram, php-crypt-gpg, picmi, projectl, pygpgme, python-apt, python-django-bootstrap-form, python-django-navtag, python-oslo.config, qmmp, qsapecng, r-cran-sem, rocs, ruby-mini-magick, seahorse-nautilus, shiro, snap, tcpcopy, tiledarray, triggerhappy, ucto, urdfdom, vmmlib, yara-python, yi & z3.

FTP Team

As a Debian FTP assistant I ACCEPTed 90 packages: android-platform-external-jsilver, android-platform-frameworks-data-binding, camlpdf, consolation, dfwinreg, diffoscope, django-restricted-resource, django-testproject, django-testscenarios, gitlab-ci-multi-runner, gnome-shell-extension-taskbar, golang-github-flynn-archive-go-shlex, golang-github-jamesclonk-vultr, golang-github-weppos-dnsimple-go, golang-golang-x-time, google-android-ndk-installer, haskell-expiring-cache-map, haskell-hclip, haskell-hdbc-session, haskell-microlens-ghc, haskell-names-th, haskell-persistable-record, haskell-should-not-typecheck, haskell-soap, haskell-soap-tls, haskell-th-reify-compat, haskell-with-location, haskell-wreq, kbtin, libclipboard-perl, libgtk3-simplelist-perl, libjs-jquery-selectize.js, liblemon, libplack-middleware-header-perl, libreoffice, libreswan, libtest-deep-json-perl, libtest-timer-perl, linux, linux-signed, live-tasks, llvm-toolchain-3.8, llvm-toolchain-snapshot, lua-luv, lua-torch-image, lua-torch-nn, magic-wormhole, mini-buildd, ncbi-vdb, node-ast-util, node-es6-module-transpiler, node-es6-promise, node-inline-source-map, node-number-is-nan, node-object-assign, nvidia-graphics-drivers, openhft-chronicle-bytes, openhft-chronicle-core, openhft-chronicle-network, openhft-chronicle-threads, openhft-chronicle-wire, pycodestyle, python-aptly, python-atomicwrites, python-click-log, python-django-casclient, python-git-os-job, python-hypothesis, python-nosehtmloutput, python-overpy, python-parsel, python-prov, python-py, python-schema, python-tackerclient, python-tornado, pyvo, r-cran-cairo, r-cran-mi, r-cran-rcppgsl, r-cran-sem, ruby-curses, ruby-fog-rackspace, ruby-mixlib-archive, ruby-tzinfo-data, salt-formula-swift, scapy3k, self-destructing-cookies, trollius-redis & websploit.

Categories: LUG Community Blogs

Chris Lamb: CLI client

Sun, 14/08/2016 - 19:43

One criminally-unknown new UNIX tool is diffoscope, a diff "on steroids" that will not only recursively unpack archives but will transform binary formats into human-readable forms in order to compare them instead of simply showing the raw difference in hexadecimal.

In an attempt to remedy its underuse, in December 2015 I created the service so that I—and hopefully others—could use diffoscope without necessarily installing the multitude of third-party tools that using it can require. It also enables trivial sharing of the HTML reports in bugs or on IRC.

To make this even easier, I've now introduced a command-line client to the web service:

$ apt-get install trydiffoscope [..] Setting up trydiffoscope (57) ... $ trydiffoscope /etc/hosts.allow /etc/hosts.deny --- a/hosts.allow +++ b/hosts.deny │ @@ -1,10 +1,17 @@ │ -# /etc/hosts.allow: list of hosts that are allowed to access the system. │ -# See the manual pages hosts_access(5) and hosts_options(5). │ +# /etc/hosts.deny: list of hosts that are _not_ allowed to access the system. │ +# See the manual pages hosts_access(5) and hosts_options(5).

You can also install it from PyPI with:

$ pip install trydiffoscope

Mirroring the original diffoscope command, you can save the output locally in an even more-readable HTML report format by appending "--html output.html".

In addition, if you specify the --webbrowser (or -w) argument:

$ trydiffoscope -w /etc/hosts.allow /etc/hosts.deny

... this will automatically open your default browser to view the results.

Categories: LUG Community Blogs

Chris Lamb: Free software activities in July 2016

Mon, 01/08/2016 - 05:20

Here is my monthly update covering a large part of what I have been doing in the free software world (previously):

  • Ensured that the Webconverger web kiosk operating system builds reproducibly. I may rework some of the patches to libisoburn and libisofs before sending them upstream. This work was sponsored by Webconverger.
  • Proposed a pull request for Regex Replace (a Chrome extension to automatically replace text on webpages) to ensure that the rules were correctly HTML encoded on the options page. (#3)
  • Proposed a change to ronn, a documentation generator that "is the opposite of roff", to make the output reproducible. (#98)
  • Fixed an issue in django-enumfield, a custom Django web development field for type-safe named constants, to make the Enum.get interface more consistent. (#36)
  • Proposed a change to txt2tags to make the output use SOURCE_DATE_EPOCH and non-timezone timestamps. (#204).
  • Created a proof-of-concept wrapper for pymysql to reduce the diff between Ubuntu and Debian's packaging of python-django. (tree)
  • Improved the NEW queue HTML report to display absolute timestamps when placing the cursor over relative times as well as to tidy the underlying HTML generation.
  • Tidied and pushed for the adoption of a patch against dak to also send mails to the signer of an uploaded package on security-master. (#796784)

This month I have been paid to work 14 hours on Debian Long Term Support (LTS). In that time I did the following:

  • "Frontdesk" duties, triaging CVEs, etc.
  • Improved the bin/ script to ignore packages that have been marked as unsupported.
  • Improved the bin/contact-maintainers script to print a nicer error message if you mistype the package name.
  • Issued the following advisories:
    • DLA 541-1 for libvirt making the password policy consistent across the QEMU and VNC backends with respect to empty passwords.
    • DLA 574-1 for graphicsmagick fixing two denial-of-service vulnerabilities.
    • DLA 548-1 and DLA 550-1 for drupal7 fixing an open HTTP redirect vulnerability and a privilege escalation issue respectfully.
    • DLA 557-1 for dietlibc removing the current directory from the current path.
    • DLA 577-1 for redis preventing the redis-cli tool creating world-readable history files.
  • redis:
    • 3.2.1-2 — Avoiding race conditions in upstream test suite.
    • 3.2.1-3 — Correcting world_readable ~/.rediscli_history files.
    • 3.2.1-4 — Preventing a race condition in the previous upload's patch.
    • 3.2.2-1 — New upstream release.
    • 3.2.1-4~bpo8+1 — Backport to jessie-backports.
  • strip-nondeterminism:
    • 0.020-1 — Improved the PNG handler to not blindly trust chunk sizes, rewriting most of the existing code.
    • 0.021-1 — Correcting a regression in the PNG handler where it would leave temporary files in the generated binaries.
    • 0.022-1 — Correcting a further regression in the PNG handler with respect to IEND chunk detection.
  • python-redis (2.10.5-1~bpo8+1) — Backport to jessie-backports.
  • reprotest (0.2) — Sponsored upload.
Patches contributed

I submitted patches to fix faulty initscripts in lm-sensors, rsync, sane-backends & vsftpd.

In addition, I submitted 7 patches to fix typos in debian/rules against cme:, gnugk: `incorrect reference to dh_install_init, php-sql-formatter, python-django-crispy-forms, libhook-lexwrap-perl, mknbi & ruby-unf-ext.

I also submitted 6 patches to fix reproducible toolchain issues (ie. ensuring the output is reproducible rather than the package itself) against libextutils-parsexs-perl: `Please make the output reproducible, perl, naturaldocs, python-docutils, ruby-ronn & txt2tags.

Lastly, I submitted 65 patches to fix specific reproducibility issues in amanda, boolector, borgbackup, cc1111, cfingerd, check-all-the-things, cobbler, ctop, cvs2svn, eb, eurephia, ezstream, feh, fonts-noto, fspy, ftplib, fvwm, gearmand, gngb, golang-github-miekg-pkcs11, gpick, gretl, hibernate, hmmer, hocr, idjc, ifmail, ironic, irsim, lacheck, libmemcached-libmemcached-perl, libmongoc, libwebsockets, minidlna, mknbi, nbc, neat, nfstrace, nmh, ntopng, pagekite, pavuk, proftpd-dfsg, pxlib, pysal, python-kinterbasdb, python-mkdocs, sa-exim, speech-tools, stressapptest, tcpflow, tcpreen, ui-auto, uisp, uswsusp, vtun, vtwm, why3, wit, wordgrinder, xloadimage, xmlcopyeditor, xorp, xserver-xorg-video-openchrome & yersinia.

Bugs filed without patches
RC bugs

I also filed 68 RC bugs for packages that access the internet during build against betamax, curl, django-localflavor, django-polymorphic, dnspython, docker-registry, elasticsearch-curator, elib.intl, elib.intl, elib.intl, fabulous, flask-restful, flask-restful, flask-restful, foolscap, gnucash-docs, golang-github-azure-go-autorest, golang-github-fluent-fluent-logger-golang, golang-github-franela-goreq, golang-github-mesos-mesos-go, golang-github-shopify-sarama, golang-github-unknwon-com, golang-github-xeipuuv-gojsonschema, htsjdk, lemonldap-ng, libanyevent-http-perl, libcommons-codec-java, libfurl-perl, libgravatar-url-perl, libgravatar-url-perl, libgravatar-url-perl, libgravatar-url-perl, libgravatar-url-perl, libhttp-async-perl, libhttp-oai-perl, libhttp-proxy-perl, libpoe-component-client-http-perl, libuv, libuv1, licenseutils, licenseutils, licenseutils, musicbrainzngs, node-oauth, node-redis, nodejs, pycurl, pytest, python-aiohttp, python-asyncssh, python-future, python-guacamole, python-latexcodec, python-pysnmp4, python-qtawesome, python-simpy, python-social-auth, python-structlog, python-sunlight, python-webob, python-werkzeug, python-ws4py, testpath, traitlets, urlgrabber, varnish-modules, webtest & zurl.

Finally, I filed 100 FTBFS bugs against abind, backup-manager, boot, bzr-git, cfengine3, chron, cloud-sptheme, cookiecutter, date, django-uwsgi, djangorestframework, docker-swarm, ekg2, evil-el, fasianoptions, fassets, fastinfoset, fest-assert, fimport, ftrading, gdnsd, ghc-testsuite, golang-github-magiconair-properties, golang-github-mattn-go-shellwords, golang-github-mitchellh-go-homedir, gplots, gregmisc, highlight.js, influxdb, jersey1, jflex, jhdf, kimwitu, libapache-htpasswd-perl, libconfig-model-itself-perl, libhtml-tidy-perl, liblinux-prctl-perl, libmoox-options-perl, libmousex-getopt-perl, libparanamer-java, librevenge, libvirt-python, license-reconcile, louie, mako, mate-indicator-applet, maven-compiler-plugin, mgt, mgt, mgt, misc3d, mnormt, nbd, ngetty, node-xmpp, nomad, perforate, pyoperators, pyqi, python-activipy, python-bioblend, python-cement, python-gevent, python-pydot-ng, python-requests-toolbelt, python-ruffus, python-scrapy, r-cran-digest, r-cran-getopt, r-cran-lpsolve, r-cran-rms, r-cran-timedate, resteasy, ruby-berkshelf-api-client, ruby-fog-libvirt, ruby-grape-msgpack, ruby-jquery-rails, ruby-kramdown-rfc2629, ruby-moneta, ruby-parser, ruby-puppet-forge, ruby-rbvmomi, ruby-redis-actionpack, ruby-unindent, ruby-web-console, scalapack-doc, scannotation, snow, sorl-thumbnail, svgwrite, systemd-docker, tiles-request, torcs, utf8proc, vagrant-libvirt, voms-api-java, wcwidth, xdffileio, xmlgraphics-commons & yorick.

FTP Team

As a Debian FTP assistant I ACCEPTed 114 packages: apertium-isl-eng, apertium-mk-bg, apertium-urd-hin, apprecommender, auto-apt-proxy, beast-mcmc, caffe, caffe-contrib, debian-edu, dh-make-perl, django-notification, dpkg-cross, elisp-slime-nav, evil-el, fig2dev, file, flightgear-phi, friendly-recovery, fwupd, gcc-5-cross, gdbm, gnustep-gui, golang-github-cznic-lldb, golang-github-dghubble-sling, golang-github-docker-leadership, golang-github-rogpeppe-fastuuid, golang-github-skarademir-naturalsort, golang-glide, gtk+2.0, gtranscribe, kdepim4, kitchen, lepton, libcgi-github-webhook-perl, libcypher-parser, libimporter-perl, liblist-someutils-perl, liblouis, liblouisutdml, libneo4j-client, libosinfo, libsys-cpuaffinity-perl, libtest2-suite-perl, linux, linux-grsec, lua-basexx, lua-compat53, lua-fifo, lua-http, lua-lpeg-patterns, lua-mmdb, lua-openssl, mash, mysql-5.7, node-quickselect, nsntrace, nvidia-graphics-drivers, nvidia-graphics-drivers-legacy-304xx, nvidia-graphics-drivers-legacy-340xx, openorienteering-mapper, oslo-sphinx, p4est, patator, petsc, php-mailparse, php-yaml, pykdtree, pypass, python-bioblend, python-cotyledon, python-jack-client, python-mido, python-openid-cla, python-os-api-ref, python-pydotplus, python-qtconsole, python-repoze.sphinx.autointerface, python-vispy, python-zenoss, r-cran-bbmle, r-cran-corpcor, r-cran-ellipse, r-cran-minpack.lm, r-cran-rglwidget, r-cran-rngtools, r-cran-scatterd3, r-cran-shinybs, r-cran-tibble, reproject, retext, ring, ruby-github-api, ruby-rails-assets-jquery-ui, ruby-swd, ruby-url-safe-base64, ruby-vmstat, ruby-webfinger, rustc, shadowsocks-libev, slepc, staticsite, steam, straight.plugin, svgwrite, tasksh, u-msgpack-python, ufo2otf, user-mode-linux, utf8proc, vizigrep, volk, wchartype, websockify & wireguard.

Categories: LUG Community Blogs

Chris Lamb: Python quirk: Signatures are evaluated at import time

Thu, 21/07/2016 - 12:07

Every Python programmer knows to avoid mutable default arguments:

def fn(mutable=[]): mutable.append('elem') print mutable fn() fn() $ python ['elem'] ['elem', 'elem']

However, many are not clear that this is due to arguments being evaluated at import time, rather than the first time the function is evaluated.

This results in related quirks such as:

def never_called(error=1/0): pass $ python Traceback (most recent call last): File "", line 1, in <module> ZeroDivisionError: integer division or modulo by zero

... and an—implementation-specific—quirk caused by naive constant folding:

def never_called(): 99999999 ** 9999999 $ python [hangs]

I suspect that this can be used as denial-of-service vector.

Categories: LUG Community Blogs

Chris Lamb: Python quirk: os.stat's return type

Tue, 19/07/2016 - 11:20
import os import stat st = os.stat('/etc/fstab') # __getitem__ x = st[stat.ST_MTIME] print((x, type(x))) # __getattr__ x = st.st_mtime print((x, type(x))) (1441565864, <class 'int'>) (1441565864.3485234, <class 'float'>)
Categories: LUG Community Blogs

Mick Morgan: show me yours

Wed, 13/07/2016 - 17:30

As Theresa May moves from the Home Office to Number 10, it is perhaps timely to reflect on public attitudes to surveillance as evidenced in Liberty’s campaign film “Show me yours” in April of this year. In the film (shown below), comedian Olivia Lee pursues members of the public with the intention of taking details from their mobile phones of all their recent communications or browsing activity. The reactions of the people approached speak for themselves. Unfortunately, Liberty research suggests that 75% of adults in the UK had never heard of the impending legislation laid out in the Investigatory Powers Bill.

Categories: LUG Community Blogs