This week from Tues 27th – Thurs 29th August 2013 from 2pm – 8pm UTC we will be running our online Ubuntu Developer Summit. This is an event we run every three months to discuss, debate, and plan the next three months of Ubuntu as well as wider goals.
All sessions will run using a combination of Google+ streaming video hangouts and IRC, and you can see the full schedule on summit.ubuntu.com. Consequently, for those who cannot attend or might miss certain sessions, all sessions will be available pre-recorded from the session pages when the session is complete.
We will kick off at 2pm UTC with my normal intro and a keynote from Mark Shuttleworth, founder of Ubuntu.
Yesterday I took a few days off to fly out to Indianapolis to provide the keynote for the annual Start With Art event run by the Arts Council Of Indianapolis. Many thanks to Dave Lawrence for the kind invitation.
The event I spoke at was a luncheon attended by 1000 people which included awards given out to local artists and community leaders, a speech from the First Lady Of Indianapolis, and some musical performances. My keynote wrapped up the two hour event.
My presentation focused on the intersection between art and community, and the many similar driving forces behind both. Art is meant to be created, distributed, shared, and enjoyed, and communities are a wonderful way to infuse artists with opportunity. Likewise creativity is the backbone of great community.
My presentation touched on a number of experiences and take-aways from my experience as both an artist (citing examples of Severed Fifth and LugRadio) and a community manager (covering Ubuntu) and a set of general lessons and conclusions that I have learned over the years. Although I had never been to a Start With Art event before, and was a little nervous as I didn’t really know the audience, the presentation was well received.
I love speaking, and I love meeting new people at the events I speak at, but I have to admit, this event felt different to most.
I must confess that I didn’t have a particularly large scope of knowledge about what the Arts Council Of Indianapolis actually do, but the opening remarks included a range of announcements of new areas of focus and work that the organization are working on as well as updates about existing programs. The council highlights artists, provides funding campaigns, released a local crowd-funding portal to connect donors to artists, built a central arts website for ticket and performance information and more. The strong overriding message I got from all of this was that they are doing everything they can to make Indy a national example of a thriving arts eco-system.
The level of passion that I experienced today from the organizers, attendees, and sponsors was inspiring. They are clearly charting a course for Indianapolis to be to arts that Nashville is to music and Silicon Valley is to technology. The core takeaway from my presentation was that great communities succeed when united around a mission, and the organizers from the Arts Council Of Indianapolis and their community leaders are a thundering example of this sense of purpose. They are not just talking about how to improve the arts in Indianapolis, they are making it happen, and the event today was a strong testament to their efforts.
A truly inspiring trip, and many thanks to everyone at the Arts Council Of Indianapolis for taking such good care of me while I was in town. Indianapolis is an awesome city, and it is wonderful to know that the arts community is in such good hands.
This article is part of a series of blog posts covering the many different areas of work going on in Ubuntu right now. See the introduction post here that links to all the articles.
In my last article I talked about the new app upload process, but today I am going to talk about how developers write apps in the first place.
For a long time our app developer story in Ubuntu has been quite fragmented. This has been due to a number of reasons:
We tried to rectify some of these issues by recommending people write apps with Python and GTK, and we wrote a tool called Quickly to optimize this process. Quickly would generate a project and help with tasks such as editing, creating your UI, and generating a package, but quickly was a somewhat primitive and incomplete solution to the problem.
The work on Quickly also showcased some limitations in our tooling. At the time we recommended people write apps using GEdit, Glade, and GTK. Unfortunately, this collection of tools just didn’t compare favorably to the developer experience on Apple and Google’s platforms, despite the best efforts of the respective upstreams. We needed to provide an end-to-end SDK for developers that would take a developer from a new project through to submitting the app into the Ubuntu Software Center.Choosing a Technology
We set out to resolve these issues and build a consistent Ubuntu SDK.
The first decision we made was around which frameworks we wanted to support when developers write their apps. These frameworks needed to be highly efficient and able to converge across multiple devices. We finalized this list as:
Some time ago we decided to focus on Qt as a platform for not only building our SDK but building our convergence story too. Qt has many benefits:
We also started looking into the best way in which we could support HTML5 developers. While the IDE decision had been made (Qt Creator) we also decided to invest in building Apache Cordova support into our SDK to make writing HTML5 as flexible as possible. This way you can either write a stock HTML5 app or use the cordova functionality…all accessible within the same IDE.The Ubuntu SDK
We formed the SDK team and started work. This work was broken into two areas.
Firstly, we started work on the app developer platform. This is largely identifying the needs of app developers for writing apps for Ubuntu devices, and ensuring we have support for those needs (which largely requires integrating that support and creating APIs). This has included:
This work is currently on-going and in various stages of completeness, but all of these platform APIs will be ready by the end of August and many apps are already consuming them. Remember, these services will be made available across all form factors.
The second piece was the SDK itself. This is tuning the Qt Creator IDE for our needs and ensuring it can be used to create QML, HTML5, and OpenGL apps. This work has touched on a number of different areas and has resulted in the following features:
We rolled all of these features into the first Beta of the SDK which was released about a month ago and you can get started with it on developer.ubuntu.com.developer.ubuntu.com
Speaking of developer.ubuntu.com, we have invested significantly in making the site a central resource for all of your development needs.
Currently the site provides tutorials for building apps, API documentation, and a cookbook that brings together the top rated questions from Ask Ubuntu. The site provides a good spring-board for getting started.
We are however in the process of making a number of improvements to developer.ubuntu.com. This will include:
We are expecting to have many of these improvements in place in the coming weeks.Are We There yet?
As we stand today we now have a powerful Ubuntu SDK with support for writing convergent apps in Qt/QML, HTML5, OpenGL, and writing Scopes that fit into the dash. You can go to developer.ubuntu.com to find out more, install the SDK, and fine tutorials for getting started.
We are only just gettin started though. The 1.0 of the SDK will be released in October and expect to find more refinements, better integration, and more features as we understand the needs of our developers better and expand the platform.
As many of you will have seen, unfortunately the Ubuntu Edge campaign did not reach our goal of $32million. The final total reached was $12,812,776. I am hugely proud and thankful to everyone who pledged, supported the campaign, wrote about it, and helped to spread the word.
Some have described us not meeting the goal as a “failure”. I don’t see it that way. Let’s be honest: $32million was always an incredibly ambitious target. We would have liked to have done it for less money, but building a F1 superphone doesn’t come cheap (and remember that the $32million didn’t include any costs for software engineering and project management…Canonical were providing that for free). It was an ambitious target, but disrupting an industry is ambitious in itself, and we gave the crowd-funding campaign our best shot. The story does not end here though.
I am not surprised that we didn’t hit this ambitious $32million target, but I am surprised at what we did achieve. We broke all the crowd-funding records, garnered media attention across CNBC, Engadget, The Independent, TechCrunch, the BBC, T3, Stuff, The Verge, The Guardian, Wired, pandodaily, Fast Company, Forbes, The Telegraph and more. Every single person who put their support into the Ubuntu Edge campaign should be proud of their achievements and we are all thankful for your tremendous and inspiring support.
One thing to be critically clear about is that the Ubuntu convergence story does not end here. We are as voraciously excited and committed to bringing this Free Software convergence story to the world as ever before; our work with OEMs, Carriers, and ISVs continues apace. We have fantastic work going on across all fronts, and we are on track to have a 1.0 release of the Ubuntu Phone platform in October.
What this experience demonstrated to me more than anything was the passion and commitment of the Ubuntu family. We are a global and diverse family all united by a dream of what the future can look like, a future in which powerful, elegant technology is freely available to all, available in the devices people care about and use to learn, create, and live better lives. Our Ubuntu family is what makes us strong, and while we didn’t hit the $32million we saw yet another example of our family coming together as one and the wider industry getting a peek into our world and the technology we have to offer.
Onwards and upwards!
“Who was looking after the $7,000 option? It’s only $7,000 for pity’s sake! Surely out of a global business world you can fined more than several?”
[Several actually means nothing, it's like saying many years or a number of years it means nothing. People generally use it to con you into thinking it's more than it actually is.]
“20,000 backers. 20,000! There’s over seven billion people in the world? What was you doing? Targeting Brighton?”
“Who was looking after the Persone Piccolo? $20 and $50! How hard can it be to get $20 out of our community? What the hell was the Loco’s doing?”_
[With my FOSS hat on I'm not happy about this one but as a Manager I think it's a valid statement.]
I can think of more but I’m not doing Canonical’s job for them without some sort of consultancy fee.
I have no issues with paying for anything FLOSS related or GPL’d my issues with this #ubuntuedge campaign are;
1. It’s a mobile phone, I have no interest in mobile phones, the vast majority of the FOSS community mainly develop desktops and or servers so will potentially only have a cursory interest in a mobile phone. I honestly cannot fathom the excitement at this move away from what we have all been doing for the last 10 years to a totally different environment.
2. I’m totally against the form and method this campaign has taken. The business world already views FOSS as twelve year old bedroom hackers, and this just makes us look like a charity case. A company like Canonical should have funded this themselves, they should have arranged for 10,000 units to be built and sold them on a pre-order basis. Clearly Mark Shuttleworth actually has no faith in this concept at all as he was not prepared to gamble his money but more than happy to gamble with the Ubuntu communities money. Personally, IMO he should hang his head in shame for cynically playing on the fact that the loyal Ubuntu user base was obviously going to dip into their savings at a time when the world is in a recession and everyone is feeling the credit crunch.
3. It is painfully obvious that the vast majority of people including Ubuntu users actually have no idea about this campaign other than“They want $32 and you can have a phone.”I’ve seen over twenty posts both on G+ and Twitter from people who are convinced there is actually a phone that they can buy now! Why do they think that? Because of all the images of people holding a fake#ubuntuedge phone!
4. There is far too much vapour-ware from Canonical or bullshit if you prefer. It’s getting to the stage whereby I wish the whole pack of cards would come crashing down because it seems to me the only thing keeping Canonical going these days is hot air and bluster! As the old saying goes “when the money runs out you’d all better watch out!”
5. I have done sales for over 30 years now. I know sales I know marketing. Yes they’ve had some column inches over this but a mention on the BBC and a report in the Daily Telegraph does not pay the staffs wages! SALES pays the wages, MONEY pays the bills, hard CASH pays for development. I suspect there will be certain people will still be stupidly saying “Ah but look at the publicity we’re getting” while filling out their Job Seekers form!
This article is part of a series of blog posts covering the many different areas of work going on in Ubuntu right now. See the introduction post here that links to all the articles.
In this article I am going to discuss some improvements we making to significantly simply and speed up how app devs get their apps into Ubuntu.
You can think of an Ubuntu system as two layers of software:
When we started working on Ubuntu back in 2004 the system and the applications where closely intermingled. We basically synced the Debian archive with Ubuntu, applied a bunch of patches, and whatever was in the Debian and Ubuntu archives was available as applications.
There were some inherent problems with this approach:
With these issues we tried to remedy the problem by creating the Application Review Board; a community group who would review packages with a simplified criteria. Unfortunately, despite the best efforts of the ARB, they were never really set up for success due to the technical limitations in the platform.
The primary technical limitation the ARB faced is that the Ubuntu system has traditionally lacked sand-boxing to protect it from naughty applications doing something…well, naughty. This meant that every app they reviewed would need a full code review for every version submitted. So, an app developer could release 1.0 and then release 1.1 a few weeks later and this would require two full code reviews. Not only this, but Debian packages have something called maintainer scripts that can theoretically do anything as root, so the packaging needed checking too.
For a group of volunteers with a growing list of submitted apps the ARB quickly fell behind. This resulted in frustration all around – for the ARB, for app devs, and for those of us working to refine and improve Ubuntu.The New App Upload Process
So, the previous system had the following primary limitations:
We started building a new specification for the new process some time ago. My team wrote the initial spec with input from a range of teams, and this has formed the foundation for the work that is reaching maturity now. Let’s cover how it works.
There are three primary pieces to this work, click packages, sand-boxing, and uploading.Click Packages
Firstly, remember how earlier I mentioned two layers in an Ubuntu system:
The Debian packaging format provides a remarkable range of features that are really designed for the former, for building Operating Systems. For the latter, app devs who simply want to package their code into something that can be loaded by users, we have created a new, simplified format for packages. It is called the click package format.
Click packages are simple to create: when you have your project open in the Ubuntu SDK you can press a button to generate a click package. Done! Click packages also don’t include maintainer scripts: they are simply not needed for the vast majority of apps, so, that already removes a security risk and something that needs reviewing.
A key benefit of click packages also is that they don’t include full dependency resolution. Many of you will be familiar with running apt-get update on your system. Well, this syncs the list of packages from the archive and figures out all the dependencies and how they map out (e.g. knowing that GIMP requires GTK, which in turn requires X). This takes quite some time and doesn’t scale to thousands of packages that getting updated all the time.
With a click package the software simply depends on the Ubuntu SDK. This means we don’t need to worry about all that complex dependency resolution: we know the dependency, the Ubuntu SDK. Additional dependencies can simply be bundled into the package. Also, instead of maintaining a list of packages on the system…they are on a web service. You need a connection to download the package anyway, so why not ask a service which packages are available?Sand-boxing
With a simple and efficient package format all set we next have the issue of sand-boxing.
This is more work than you might imagine. The team identified a big list of sand-boxing requirements that were needed for us to be sure an app could be run without a code review. This is not only protecting the system from inappropriate calls, but also handling issues with other components (e.g. sniffing keyboard events in X apps).
Well, the awesome Ubuntu Security Team has been working on full sand-boxing throughout the system and most of this work has been completed. This has also influenced some other design decisions: as an example, Mir (our new display server) is designed to not expose the keyboard event sniffing issue X has faced.
The basic result of this work is that the security team have an app called evil app that does a series of naughty things, and the sand-boxing protects the system from such naughtyness.
With this sand-boxing in place it essentially mitigates the need for a full code review, which was the primary bottleneck previously. This combined with click packages not having maintainer scripts and complex dependency chains makes reviews much easier and more efficient.Uploading
The process for uploading an app to Ubuntu is going to remain largely the same as it works generally pretty well. As before you can select a package to upload, set information about the app, add screenshots and more and then it will hit a review queue before it appears to Ubuntu users.
The good news is that because the review only really require a check of permissions, meta-data, and a few other simple bits (thanks to click packages and sand-boxing), the review can be done in under 15 minutes as opposed to multi-day code reviews. This means apps should go in more quickly.
Once your app is in the archive you will be able to search for it across all Ubuntu devices. Currently all apps will be displayed, but we are adding features to only show apps that will work on that device (e.g. only show apps with a phone UI on the phone).Are We There Yet?
So where do we stand with all this goodness? Fortunately, we are getting close.
The click package format is largely complete and we have tools for creating click packages in the SDK and installing them on a system.
Much of the sand-boxing work has been largely completed, although one caveat is that because we are working on Mir as our new display server, we are not investing in fixing keyboard sniffing in X (which would be a huge amount of work). This means that we won’t release this system on the desktop until Mir is on by default (around 14.10). Our existing system will be in place until then.
The app upload piece is online and largely complete (although going through a series of reviews), and the scope for browsing new click packages is on the mobile images although isn’t quite yet hooked up.
You can see an early video demo of this working below:
Can’t see it? See it here!
Our goal is to have this full system in place for mobile in 13.10, and for desktop in 14.10. This will all make getting apps into Ubuntu quicker, easier, and more rewarding than ever before.
There is some really awesome work going on right now in Ubuntu, and much of it is fixing and resolving issues and bottlenecks that have been an issue in Ubuntu for many years. Not only are we building an awesome convergence platform and cloud orchestration story, but we are re-building much of the core foundations to make these efforts more successful.
One of the challenges we have with all this great work is that even though everything is out in the open, some folks don’t have a crisp summary of the different pieces. So, I am kicking off a blog series that will summarize many of these different efforts as they stand.
My goal is to provide a overall summary of the work (not a huge wall of text) and when we expect it to be completed. I will regularly update this first post with a link to all the articles as I write them, so folks can point people to this post which will link to them all.