All I want for 2015 is a Free/Open Source Software social network which is:
Is this too much to ask for? Does it exist already?
Every few days, someone publishes a new guide, tutorial, library or framework about web scraping, the practice of extracting information from websites where an API is either not provided or is otherwise incomplete.
However, I find these resources fundamentally deceptive — the arduous parts of "real world" scraping simply aren't in the parsing and extraction of data from the target page, the typical focus of these articles.
The difficulties are invariably in "post-processing"; working around incomplete data on the page, handling errors gracefully and retrying in some (but not all) situations, keeping on top of layout/URL/data changes to the target site, not hitting your target site too often, logging into the target site if necessary and rotating credentials and IP addresses, respecting robots.txt, target site being utterly braindead, keeping users meaningfully informed of scraping progress if they are waiting of it, target site adding and removing data resulting in a null-leaning database schema, sane parallelisation in the presence of prioritisation of important requests, difficulties in monitoring a scraping system due to its implicitly non-deterministic nature, and general problems associated with long-running background processes in web stacks.
In other words, extracting the right text on the page is the easiest and trivial part by far, with little practical difference between an admittedly cute jQuery-esque parsing library or even just using a blunt regular expression.
It would be quixotic to simply retort that sites should provide "proper" APIs but I would love to see more attempts at solutions that go beyond the superficial.
I've finally found a real, practical use for my Raspberry Pi: it's an always-on machine that I can ssh to and use to wake up my home media server :)
It doubles as yet another syncthing client to add to my set.
And of course, it's running Arch ;)
Once upon a time I worked from home for seven years, for a company called Bytemark. Then, due to a variety of reasons, I left. I struck out for adventures and pastures new, enjoyed the novelty of wearing clothes every day, and left the house a bit more often.
Things happened. A year passed.
Now I'm working for Bytemark again, although there are changes and the most obvious one is that I'm working in a shared-space/co-working setup, renting a room in a building near my house instead of being in the house.
Shame I have to get dressed, but otherwise it seems to be OK.
We have regular sessions on the second Saturday of each month. To kick off the New Year we're meeting at the Tattenham Corner, Epsom. A commanding view of Epson Downs, good food (a Beefeater pub), beer and a welcome whether you're a regular Surrey LUG enthusiast or a first-timer.
A while ago, I started using keychain to manage my ssh and gpg agents. I did this with the following in my .bashrc# Start ssh-agent eval $(keychain --quiet --eval id_rsa)
Unfortunately, tools like keychain don't know about that and still expect it to be set, leading to some annoying breakage.
My fix is a quick and dirty one; I appended the following to .bashrcexport GPG_AGENT_INFO=~/.gnupg/S.gpg-agent:$(pidof gpg-agent):1
Here's this year's TODO diff as compared with last year.New Year's Resolutions
Read even more (fiction and non-fiction)
Write at least one short story
Write some more games
Go horse riding
Learn some more turkish
Play a lot more guitar
I did this but want to do more!
Lose at least a stone (in weight, from myself)
I almost did this and then put it all back on again
Try to be less of a pedant (except when it's funny)
Try to be more funny ;)
I'm sure I've achieved these ;)
Receive a lot less email
In particularly, unsubscrive from things I don't read
Particularly about technical subjects
Write more software
Release more software
Be a better husband and father
I think I'm doing alright but I'm sure I can do better
Improve or replace the engine I use for my blog
We used to think that happiness is based on succeeding at our goals, but it turns out not so much.
Most marathon runners, for example — not professionals but the amateur runners — their high for completing the marathon usually disappears even before their nipples stop bleeding.
My point is that the high lasts for a very short amount of time. If you track where people's happiness and satisfaction is, it is in "getting" or in making progress towards our goals. It's a more satisfying, life-affirming, motivating and happy thing than actually reaching them.
So it's a great thing to keep in mind... Health, for example. When you define health as something you want to do, living your life and looking back on the week and saying "That was a healthy week for me: I worked out this number of times, I did this amount of push-ups, I ate reasonably most of the time..." that's very satisfying and that's where you'll feel happy about yourself. And if you're too focused on a scale for some reason, then that's an external thing that you'll hit... and then what?
So it is about creating goals that are longer lasting and really focusing on the journey because that's really where we get our happiness and our satisfaction. Once it's achieved... now what?
Was lent a 15-course baroque lute.
A lot of triathlon training but also got back into cooking.
Returned to the Cambridge Duathlon.
Raced 50 and 100 mile cycling time trials & visited the Stratford Olympic pool (pictured).
Paced my sister at the Downtow-Upflow Half-marathon. Also released the first version of the Strava Enhancement Suite.
Visited Cornwall for my cousin's wedding (pictured). Another month for sport including my first ultramarathon and my first sub-20 minute 5k.
Entered a London—Oxford—London cycling brevet, my longest single-ride to date (269 km). Also visited the Tour of Britain and the Sri Chomnoy 24-hour endurance race.
London—Paris—London cycling tour (588 km).
Performed Handel's Messiah in Kettering.
Rather late but I guess that just confirms it’s really me, right? The signed text and IDs should be at http://mjr.towers.org.uk/transition-statement.txt
Thank you if you help me out here I’ll resign keys in a while.
Arbitrary tweets made by TheGingerDog up to 16 June 2014
Attempt to pass homeopathy off as credible by combining it with empirically valid medicine.
We woke to continual thunder.
I think it is time to leave the country.
It seems Mozilla is targeting emerging markets and developing nations with $25 cell phones. This is tremendous news, and an admirable focus for Mozilla, but it is not without risk.
Bringing simple, accessible technology to these markets can have a profound impact. As an example, in 2001, 134 million Nigerians shared 500,000 land-lines (as covered by Jack Ewing in Businessweek back in 2007). That year the government started encouraging wireless market competition and by 2007 Nigeria had 30 million cellular subscribers.
This generated market competition and better products, but more importantly, we have seen time and time again that access to technology such as cell phones improves education, provides opportunities for people to start small businesses, and in many cases is a contributing factor for bringing people out of poverty.
So, cell phones are having a profound impact in these nations, but the question is, will it work with FirefoxOS?
I am not sure.
In Mozilla’s defence, they have done an admirable job with FirefoxOS. They have built a powerful platform, based on open web technology, and they lined up a raft of carriers to launch with. They have a strong brand, an active and passionate community, and like so many other success stories, they already have a popular existing product (their browser) to get them into meetings and headlines.
Success though is judged by many different factors, and having a raft of carriers and products on the market is not enough. If they ship in volume but get high return rates, it could kill them, as is common for many new product launches.
What I don’t know is whether this volume/return-rate balance plays such a critical role in developing markets. I would imagine that return rates could be higher (such as someone who has never used a cell phone before taking it back because it is just too alien to them). On the other hand, I wonder if those consumers there are willing to put up with more quirks just to get access to the cell network and potentially the Internet.
What seems clear to me is that success here has little to do with the elegance or design of FirefoxOS (or any other product for that matter). It is instead about delivering incredibly dependable hardware. In developing nations people have less access to energy (for charging devices) and have to work harder to obtain it, and have lower access to support resources for how to use new technology. As such, it really needs to just work. This factor, I imagine, is going to be more outside of Mozilla’s hands.
So, in a nutshell, if the $25 phones fail to meet expectations, it may not be Mozilla’s fault. Likewise, if they are successful, it may not be to their credit.
I am a firm believer in building strong and empowered communities. We are in an age of a community management renaissance in which we are defining repeatable best practice that can be applied many different types of communities, whether internal to companies, external to volunteers, or a mix of both.
I have been working to further this growth in community management via my books, The Art of Community and Dealing With Disrespect, the Community Leadership Summit, the Community Leadership Forum, and delivering training to our next generation of community managers and leaders.
Last year I ran my first community management training course, and it was very positively received. I am delighted to announce that I will be running an updated training course at three events over the coming months.OSCON
On Sunday 20th July 2014 I will be presenting the course at the OSCON conference in Portland, Oregon. This is a tutorial, so you will need to purchase a tutorial ticket to attend. Attendance is limited, so be sure to get to the class early on the day to reserve a seat!
Firstly, on Fri 22nd August 2014 I will be presenting the course at LinuxCon North America in Chicago, Illinois and then on Thurs Oct 16th 2014 I will deliver the training at LinuxCon Europe in Düsseldorf, Germany.
Tickets are $300 for the day’s training. This is a steal; I usually charge $2500+/day when delivering the training as part of a consultancy arrangement. Thanks to the Linux Foundation for making this available at an affordable rate.
Space is limited, so go and register ASAP:
So what is in the training course?
My goal with each training day is to discuss how to build and grow a community, including building collaborative workflows, defining a governance structure, planning, marketing, and evaluating effectiveness. The day is packed with Q&A, discussion, and I encourage my students to raise questions, challenge me, and explore ways of optimizing their communities. This is not a sit-down-and-listen-to-a-teacher-drone on kind of session; it is interactive and designed to spark discussion.
The day is mapped out like this:
I will warn you; it is an exhausting day, but ultimately rewarding. It covers a lot of ground in a short period of time, and then you can follow with further discussion of these and other topics on our Community Leadership discussion forum.
I hope to see you there!
Arbitrary tweets made by TheGingerDog up to 04 June 2014
Arbitrary tweets made by TheGingerDog up to 01 June 2014
I just want to say how touched I have been by the response. The comments, social media posts, emails, and calls from you have been so kind and supportive. You are all good people, and I am going to miss every single one of you.
The reason why I have devoted my life to understanding communities is that I believe communities bring out the best in people, and all of you are a perfect example of that. I cannot express just how much I appreciate it.
Over the course of the next few weeks my replacement will be sourced and announced. and in the interim my team (Daniel Holbach, Michael Hall, David Planella, Nicholas Skaggs, Alan Pope) will take over my duties. Everything has been transitioned over, and remember, the weekly Q&As will continue at 6pm UTC every Tuesday on Ubuntu On Air with my team filling in for me. As ever, any and all Ubuntu questions are welcome!
Of course, I will still be around. I am going to continue to be a member of the Ubuntu community and an avid Ubuntu user, tester, and supporter. I will continue to be on IRC, you can email me at email@example.com, I will continue to do Bad Voltage, and I have a busy schedule at the Community Leadership Summit, OSCON, and more. I am also going to continue to have my own Q&A session every week where you can ask questions about my perspectives on Ubuntu, Canonical, community management, XPRIZE, and more; I will announce this soon.
Ubuntu has a tremendous future ahead of it, built on the hard work and passion of a global community. We are only just getting started with a new era of Ubuntu convergence and cloud orchestration and while I will miss being there in an official capacity, I am just thankful that I can continue to be along for the ride in the very community I played a part in building.
I now have a few weeks off and then my new adventure begins. Stay tuned.