The post Nom nom. #chocolate #hotelchocolat #spoilt #restraint #temptation appeared first on life at warp.
I'm slowly planning the redesign of the cluster which powers the Debian Administration website.
Currently the design is simple, and looks like this:
In brief there is a load-balancer that handles SSL-termination and then proxies to one of four Apache servers. These talk back and forth to a MySQL database. Nothing too shocking, or unusual.
(In truth there are two database servers, and rather than a single installation of HAProxy it runs upon each of the webservers - One is the master which is handled via ucarp. Logically though traffic routes through HAProxy to a number of Apache instances. I can lose half of the servers and things still keep running.)
When I setup the site it all ran on one host, it was simpler, it was less highly available. It also struggled to cope with the load.
Half the reason for writing/hosting the site in the first place was to document learning experiences though, so when it came to time to make it scale I figured why not learn something and do it neatly? Having it run on cheap and reliable virtual hosts was a good excuse to bump the server-count and the design has been stable for the past few years.
Recently though I've begun planning how it will be deployed in the future and I have a new design:
Rather than having the Apache instances talk to the database I'll indirect through an API-server. The API server will handle requests like these:
I expect to have four API handler endpoints: /articles, /comments, /users & /weblogs. Again we'll use a floating IP and a HAProxy instance to route to multiple API-servers. Each of which will use local caching to cache articles, etc.
This should turn the middle layer, running on Apache, into simpler things, and increase throughput. I suspect, but haven't confirmed, that making a single HTTP-request to fetch a (formatted) article body will be cheaper than making N-database queries.
Anyway that's what I'm slowly pondering and working on at the moment. I wrote a proof of concept API-server based CMS two years ago, and my recollection of that time is that it was fast to develop, and easy to scale.
We have regular sessions on the second Saturday of each month. Bring a 'box', bring a notebook, bring anything that might run Linux, or just bring yourself and enjoy socialising/learning/teaching or simply chilling out!
This month's meeting is at The Feathers Pub, Merstham
42 High St, Merstham, Redhill, Surrey, RH1 3EA
01737 645643 · http://www.thefeathersmerstham.co.uk
NOTE the pub opens at 12 Noon.
If, like me, you’ve just done a Debian netboot install over PXE and discovered that the partitioner suddenly seems to have no option for Ext4 filesystem (leaving only btrfs and XFS), despite the fact that it worked fine a couple of weeks ago, do not be alarmed. You aren’t losing your mind. It seems to be a bug.
As the comment says, downloading netboot.tar.gz version 20150422+deb8u3 fixes it. You can find your version in the debian-installer/amd64/boot-screens/f1.txt file. I was previously using 20150422+deb8u1 and the commenter was using 20150422+deb8u2.
Looking at the dates on the files I’m guessing this broke on 23rd January 2016. There was a Debian point release around then, so possibly you are supposed to download a new netboot.tar.gz with each one – not sure. Although if this is the case it would still be nice to know you’re doing something wrong as opposed to having the installer appear to proceed normally except for denying the existence of any filesystems except XFS and btrfs.
Oh and don’t forget to restart your TFTP daemon. tftpd-hpa at least seems to cache things (or maybe hold the tftp directory open, as I had just moved the old directory out of the way), so I was left even more confused when it still seemed to be serving 20150422+deb8u1.
I deal with compromises often enough of PHP-based websites that I wish to improve hardening.
One obvious way to improve things is to not serve PHP files which are writeable by the webserver-user. This would ensure that things like wp-content/uploads didn't get served as PHP if a compromise wrote valid PHP there.
In the past using php5-suhosin would have allowd this via the suhosin.executor.include.allow_writable_files flag.
Since suhosin is no longer supported under Debian Jessie I wonder if there is a simple way to achieve this?
I've written a toy-module which allows me to call stat on every request, and return a 403 on access to writeable files/directories. But it seems like I shouldn't need to write my own code for this functionality.
Any pointers welcome; happy to post my code if that is useful but suspect not - it just shouldn't exist.
As some of you may know, I have been working on a small hardware project called the Beer'o'Meter whose purpose is to allow us to extend Ye Olde Vic's beer board to indicate the approximate fullness of each cask. For some time now, we've been operating an electronic beer board at the Vic which you may see tweeted out from time to time. The pumpotron has become very popular with the visitors to the pub, especially that it can be viewed online in a basic textual form.
Of course, as many of you who visit pubs know only too well. That a beer is "on" is no indication of whether or not you need to get there sharpish to have a pint, or if you can take your time and have a curry first. As a result, some of us have noticed a particular beer on, come to the pub after dinner, and then been very sad that if only we'd come 30 minutes previously, we'd have had a chance at the very beer we were excited about.
Combine this kind of sadness with a two week break at Christmas, and I started to develop a Beer'o'Meter to extend the pumpotron with an indication of how much of a given beer had already been served. Recently my boards came back from Elecrow along with various bits and bobs, and I have spent some time today building one up for test purposes.
As always, it's important to start with some prep work to collect all the necessary components. I like to use cake cases as you may have noticed on the posting yesterday about the oscilloscope I built.
Naturally, after prep comes the various stages of assembly. You start with the lowest-height components, so here's the board after I fitted the ceramic capacitors:
And here's after I fitted the lying-down electrolytic decoupling capacitor for the 3.3 volt line:
Next I should have fitted the six transitors from the middle cake case, but I discovered that I'd used the wrong pinout for them. Even after weeks of verification by myself and others, I'd made a mistake. My good friend Vincent Sanders recently posted about how creativity is allowing yourself to make mistakes and here I had made a doozy I hadn't spotted until I tried to assemble the board. Fortunately TO-92 transistors have nice long legs and I have a pair of tweezers and some electrical tape. As such I soon had six transistors doing the river dance:
With that done, I noticed that the transistors now stood taller than the pins (previously I had been intending to fit the transistors before the pins) so I had to shuffle things around and fit all my 0.1" pins and sockets next:
Then I could fit my dancing transistors:
We're almost finished now, just one more capacitor to provide some input decoupling on the 9v power supply:
Of course, it wouldn't be complete without the ESP8266Huzzah I acquired from AdaFruit though I have to say that I'm unlikely to use these again, but rather I might design in the surface-mount version of the module instead.
And since this is the very first Beer'o'Meter to be made, I had to go and put a 1 on the serial-number space on the back of the board. I then tried to sign my name in the box, made a hash of it, so scribbled in the gap
Finally I got to fit all six of my flow meters ready for some testing. I may post again about testing the unit, but for now, here's a big spider of a flow meter for beer:
This has been quite a learning experience for me, and I hope in the future to be able to share more of my hardware projects, perhaps from an earlier stage.
I have plans for a DAC board, and perhaps some other things.
I recently ordered some PCBs from Elecrow for the Vic's beer-measurement system I've been designing with Rob. While on the site, I noticed that they have a single-channel digital oscilloscope kit based on an STM32. This is a JYE Tech DSO138 which arrives as a PCB whose surface-mount stuff has been fitted, along with a whole bunch of pin-through components for you to solder up the scope yourself. There's a non-trivial number of kinds of components, so first you should prep by splitting them all up and double-checking them all.
Once you've done that, the instructions start you off fitting a whole bunch of resistors...
Then some diodes, RF chokes, and the 8MHz crystal for the STM32.
The single most-difficult bit for me to solder was the USB socket. Fine pitch leads, coupled with high-thermal-density socket.
There is a veritable mountain of ceramic capacitors to fit...
And then buttons, inductors, trimming capacitors and much more...
THe switches were the next hardest things to solder, after the USB socket...
Finally you have to solder a test loop and close some jumpers before you power-test the board.
The last bit of soldering is to solder pins to the LCD panel board...
Before you finally have a working oscilloscope
I followed the included instructions to trim the scope using the test point and the trimming capacitors, before having a break to write this up for you all. I'd say that it was a fun day because I enjoyed getting a lot of soldering practice (before I have to solder up the beer'o'meter for the pub) and at the end of it I got a working oscilloscope. For 40 USD, I'd recommend this to anyone who fancies a go.
Here is my monthly update covering a large part of what I have been doing in the free software world (previously):
This month I have been paid to work 18 hours on Debian Long Term Support (LTS). In that time I did the following:
I also filed 100 FTBFS bugs against apache-log4j2, awscli, binutils, brian, ccbuild, coala, commons-beanutils, commons-vfs, composer, cyrus-sasl2, debiandoc-sgml-doc-pt-br, dfvfs, dillo, django-compat, dulwich, git-annex, grpc, hdf-eos5, hovercraft, ideviceinstaller, ircp-tray, isomd5sum, javamail, jhdf, jsonpickle, kivy, klog, libcloud, libcommons-jexl2-java, libdata-objectdriver-perl, libdbd-sqlite3-perl, libpam-krb5, libproc-waitstat-perl, libslf4j-java, libvmime, linuxdcpp, lsh-utils, mailutils, mdp, menulibre, mercurial, mimeo, molds, mugshot, nose, obex-data-server, obexfs, obexftp, orafce, p4vasp, pa-test, pgespresso, pgpool2, pgsql-asn1oid, php-doctrine-cache-bundle, php-net-ldap2, plv8, pngtools, postgresql-mysql-fdw, pyfftw, pylint-common, pylint-django, pylint-django, python-ase, python-axiom, python-biopython, python-dcos, python-falcon, python-instagram, python-markdown, python-pysam, python-requests-toolbelt, python-ruffus, pytsk, pyviennacl, ros-class-loader, ros-ros-comm, ros-roscpp-core, roxterm, ruby-celluloid-extras, ruby-celluloid-fsm, ruby-celluloid-supervision, ruby-eye, ruby-net-scp, ruby-net-ssh, ruby-sidekiq, ruby-sidekiq-cron, ruby-sinatra-contrib, seaview, smc, spatial4j-0.4, swift-plugin-s3, tilecache, typecatcher, ucommon, undertaker, urdfdom, ussp-push, xserver-xorg-video-intel & yt.FTP Team
As a Debian FTP assistant I ACCEPTed 201 packages: abi-tracker, android-platform-build, android-platform-frameworks-native, android-platform-libcore, android-platform-system-core, animate.css, apitrace, argon2, autosize.js, bagel, betamax, bittorrent, bls-standalone, btfs, caja-dropbox, cegui-mk2, complexity, corebird, courier-authlib, cpopen, ctop, dh-haskell, django-python3-ldap, e2fsprogs1.41, emacs-async, epl, fast5, fastkml, flask-restful, flask-silk, gcc-6, gitlab, golang-github-kolo-xmlrpc, golang-github-kr-fs, golang-github-pkg-sftp, golang-github-prometheus-common, google-auth-library-php, h5py, haskell-aeson-compat, haskell-userid, heroes, hugo, ioprocess, iptables, ivy-debian-helper, ivyplusplus, jquery-timer.js, klaus, kpatch, lazarus, libatteanx-store-sparql-perl, libbrowserlauncher-java, libcgi-test-perl, libdata-sah-normalize-perl, libfsntfs, libjs-fuzzaldrin-plus, libjung-free-java, libmongoc, libmygpo-qt, libnet-nessus-rest-perl, liborcus, libperinci-sub-util-propertymodule-perl, libpodofo, librep, libsodium, libx11-xcb-perl, linux, linux-grsec-base, list.js, lombok, lua-mediator, luajit, maven-script-interpreter, midicsv, mimeo, miniasm, mlpack, mom, mosquitto-auth-plugin, moxie.js, msgpuck, nanopolish, neovim, netcdf, network-manager-applet, network-manager-ssh, node-esprima-fb, node-mocks-http, node-schlock, nomacs, ns3, openalpr, openimageio, openmpi, openms, orafce, pbsim, pd-iemutils, pd-nusmuk, pd-puremapping, pd-purest-json, pg-partman, pg-rage-terminator, pgfincore, pgmemcache, pgsql-asn1oid, php-defaults, php-jwt, php-mf2, php-redis, pkg-info-el, plr, pnmixer, postgresql-multicorn, postgresql-mysql-fdw, powa-archivist, previsat, pylint-flask, pyotherside, python-caldav, python-cookies, python-dcos, python-flaky, python-flickrapi, python-frozendict, python-genty, python-git, python-greenlet, python-instagram, python-ironic-inspector-client, python-manilaclient, python-neutronclient, python-openstackclient, python-openstackdocstheme, python-prometheus-client, python-pymzml, python-pysolr, python-reno, python-requests-toolbelt, python-scales, python-socketio-client, qdox2, qgis, r-cran-biasedurn, rebar.js, repmgr, rfcdiff, rhythmbox-plugin-alternative-toolbar, ripe-atlas-cousteau, ripe-atlas-sagan, ripe-atlas-tools, ros-image-common, ruby-acts-as-list, ruby-allocations, ruby-appraiser, ruby-appraiser-reek, ruby-appraiser-rubocop, ruby-babosa, ruby-combustion, ruby-did-you-mean, ruby-fixwhich, ruby-fog-xenserver, ruby-hamster, ruby-jeweler, ruby-mime-types-data, ruby-monkey-lib, ruby-net-telnet, ruby-omniauth-azure-oauth2, ruby-omniauth-cas3, ruby-puppet-forge, ruby-racc, ruby-reek, ruby-rubinius-debugger, ruby-rubysl, ruby-rubysl-test-unit, ruby-sidekiq-cron, ruby-threach, ruby-wavefile, ruby-websocket-driver, ruby-xmlhash, rustc, s-nail, scrm, select2.js, senlin, skytools3, slurm-llnl, sphinx-argparse, sptk, sunpy, swauth, swift, tdiary, three.js, tiny-initramfs, tlsh, ublock-origin, vagrant-cachier, xapian-core, xmltooling, & yp-tools.
I additionally REJECTed 29 packages.
I first wrote about the difficulties of submitting a UK self-assessment tax return back in January 2009, with a follow up in January of 2010. In the subsequent years I have submitted my tax return under Windows with nary a thought about attempting it under Linux.
This year was different, I went to download TaxCalc as usual, only this time there was a Linux option. I barely noticed at first, downloading the Windows version before I thought "Hang on, did that say Linux?". It did indeed.
Despite the fact that I do have a Windows PC now, as well as a Linux PC and a Linux laptop, it probably makes more sense to perform my tax return on Windows now, but having posted on this subject before I really ought to give it a go!
I won't detail all the installation instructions, which are clearly described on the TaxCalc website.Installation
Unfortunately the download is for a compiled .run file and not a distribution package, so you will have to decide whether you are happy to run the file and potentially dump unknown rubbish all over your system which might then be difficult to remove.
I decided to give it a go, downloading the .run file and marking it as executable. The instructions provided did not suggest running with sudo or as root, so I decided to follow the letter of the instructions and simply executed the .run file.
This entered a typical Wizard style installer, which resulted in the application being installed within my home directory, including a 127MB lib directory containing a large number of Linux libraries. This is very much the "Windows" way of doing things, bundling everything you need within the installation, rather than simply specifying dependencies. But the fact is that it works and gives TaxCalc control over the installation, in a repeatable and supportable way.Import from previous year
The first step in using TaxCalc is always to import the previous year's files. This process completed for all files without error.Step by Step Process
Probably the main reason for using TaxCalc is the simple step process for completing your tax return, and I am pleased to report that this aspect was identical to Windows.On-line Submission
The one aspect that I was concerned about was whether the on-line submission to HMRC would work okay, again I am pleased to say that it worked flawlessly, both for the test submission and the live submission.Conclusions
I must confess that I assumed that this day would only arrive when TaxCalc moved to a web-based service, something that surely is inevitable. I am delighted to at last be able to perform my tax at home, rather than on a Windows PC at work. Good news for those wanting to eradicate Windows from their lives.
Thank you TaxCalc.
Last month Troy Hunt posted an interesting comment on his blog about the problems around the etiquette of allowing guests onto your home wifi network. In his post, Hunt notes that guests can be deeply offended at being refused access. This is understandable. If they are guests in your home then they are probably close friends or family. Refusing access can make it seem that you don’t trust them. However, as Hunt goes on to point out, it is not the guests per se you need to worry about. Anyone on your network can cause problems – usually completely unintentionally. In my case I have the particular problem that my kids assume that they can use the nertwork when they are here. Worse, they assume that they may access the network through their (google infested) smart phones. Now try as I might, there is no way I can monitor or control the way my kids (or their partners) set up their phones. Nor should I want to.
Hunt asks how others handle this problem. Like him I don’t much trust the separation offered by “guest” networks on wifi routers. In my case I decided long ago to split my network in two. I have an outer network which connects directly to my ISP and a second, inner network, which connects through another router to my outer network. Both networks use NAT and each uses an address range drawn from RFC1918. Furthermore, the routers are from different manufacturers so, hopefully, any vulnerability in one /may/ not be present in the other. My inner network has all my domestic devices, including my NAS, music and video streaming systems, DNS server etc. attached. These devices are mostly hard wired through a switch to the inner router. I only use wifi where it is not possible to hard wire, or where it would make no sense to do so. For example, my Sonos speakers and the app controlling them on my android tablet must use wifi. However, there is no reason why my kids, who insist on using Facebook, need to have access to my internal systems. So I run a separate wifi network on the outer router and they only have access to that. The only systems on the external screened network is one of my VPN endpoints (useful for when I am out and about and want to appear to be accessing the wider world from my home), and my old slug based webcam. My policy stance on the inner network is to consider the screened outer network as almost as hostile as the wider internet. This has the further advantage that bloody google doesn’t get notification of my internal wifi settings through my kids leaving “backup and restore” active on their android phones.
The post My first #linux mag. May 2000. #career #tech #computer appeared first on life at warp.
So after living here in Finland for 6 months I've now bought a flat.
We have a few days to sort out mortgage paperwork, and assuming there are no problems we'll be moving into the new place on/around the 1st of March.
Finally I'll be living in Finland, with a sauna of my very own.
A couple of years ago I was dissatisfied with mutt, mostly because the mutt-sidebar patch was dropped from the Debian package. That lead to me thinking "How hard can it be to write a modal, console-based mail-client?"
It turns out writing a client is pretty simple if you limit yourself solely to Maildirs, and as I typically read my mail over SSH on the mailhost itself that suited me pretty well.
Recently I restarted the mail-client. Putting it together from scratch to simplify the implementation, and unify a lot of the adhoc scripting which is provided by Lua. People seem to like the client, but the single largest complaint was "Can't use it - no IMAP."
This week I've mostly been adding IMAP support, and today I'll commit the last few bits that mean it is roughly-functional:
The outstanding niggles will be relating to getting/setting the new/read/seen/unseen flags, and similar. But I'm pleased that the job wasn't insurmountable.
I've used libcurl to provide the IMAP functionality because most of the IMAP libraries I looked at were big, scary, and complex. Using curl to access IMAP is pretty neat, simple, and straightforward. The downside is you're making a lot of "http" requests. So I might need to revisit things.
Happily my imap wrapper doesn't need much functionality. So if I can find a better library swapping it out will be simple.
In conclusion: Lumail almost has IMAP support, and that might mean it'll be more useful to others.
The following contributors got their Debian Developer accounts in the last two months:
The following contributors were added as Debian Maintainers in the last two months:
I am a paying member of both Amnesty International and the Open Rights Group. Both those organisations, along with many other Civil Rights organisations, technology companies and concerned individuals are signatories to an open letter to Governments across the world demanding that we retain the right to strong encryption in order to protect our privacy. That letter says:
I’ve signed. You should too.
#lastfm tells me I’m a good fit from DnBHeaven. Luckily, I love DnB, so this
The post I’m a good fit from DnBHeaven. I knew that already. appeared first on life at warp.
I don’t wish to comment on that evidence here, Adrian Kennard has already provided much useful comment on the failings of the Draft Bill. My purpose in this post to highlight the absurdity of the Parliamentary Committee’s request that the ISPA evidence be withdrawn from it’s Website. The Register article ends with this update:
ISPA contacted The Register after the publication of this story to inform us: “ISPA was requested to remove the written evidence it submitted to the Joint Committee on the Investigatory Powers Bill from the ISPA website by the Joint Committee. Their guidance states that submissions become the property of the Committee and should not be published elsewhere until the Committee has done so itself.”
As of now (14.30 on 7 January) that evidence is still on the ISPA Website. Even if removed, it will still, of course, be available from a huge range of sources such as search engine caches (apologies for the google reference, but it is the obvious one). Or you could get it here.
The point is, once such a document has been published electronically on the net, no-one, but no-one, can put the genie back in the bottle and unpublish it.
The officials supporting the Joint Parliamentary Committee should know that. And if they don’t then I would submit that they are not technically competent enough to be supporting the Committee.