Ubuntu quality: or, “but what about my bug?”

They make these decisions by estimating the importance of each bug, identifying whether it is a $url[https://wiki.ubuntu.com/QATeam/RegressionTracking,regression], assessing the risk of potential fixes, and by applying their best judgement. Developers can then focus their efforts where they are most needed. On the whole, I think that we do remarkably well at this. In September, for example, the total number of open bugs in Ubuntu increased by only 70. This doesn’t sound like much of an achievement, if not for the fact that in the same time period, 7872 new bug reports were filed. The remaining 7802, or over 99%, were resolved (some duplicates, some invalid, some fixed, etc.). The news isn’t all good, of course. There are currently over 46000 open Ubuntu bug reports in Launchpad. Even at this impressive rate of throughput, and even if we were to freeze all development and stop accepting new bug reports entirely, I estimate it would take over half a year just to sift through the backlog of reports we have already received. There is a lot of noise in that data. When 8.10 is released, as with each previous release, some users will be disappointed that it has a bug which affects them. This is regrettable, and I feel badly for affected users each time that I read about this, but it is unlikely to ever change. There will never be a release of Ubuntu which is entirely free of bugs, and every non-trivial bug is important to someone. So, what do we do? What should be our key goals where quality is concerned? We don’t currently have a clear statement of this, but here’s a strawman: Prioritize bug reports effectively. It’s usually difficult to say whether a bug report is valid or serious until a human has reviewed it, so this means having enough people to review and acknowledge incoming bug reports, and helping them to work as efficiently as possible. The $url[https://wiki.ubuntu.com/BugSquad,Ubuntu bug squad] is a focal point for this type of work, though a great deal of this is done by developers in their everyday work as well. Projects like $url[https://wiki.ubuntu.com/5-A-Day,5-a-day] are a good way to get started. Measure our performance objectively. By tracking metrics for each part of the quality assurance process, we can understand where we need to improve. The $url[https://wiki.ubuntu.com/QATeam,QA team] has been developing tools, such as the $url[http://status.qa.ubuntu.com/,package status pages], to collect hard data on bugs. Improve incrementally. By minimizing regressions from one release to the next, and making some progress on old bugs, we can hope to make each Ubuntu release better, in terms of overall quality, than the one before it (or at least no worse). The $url[http://people.ubuntu.com/~sbeattie/regression_tracker.html,regression tracker] (which will hopefully move to the $url[http://qa.ubuntu.com/,QA site] soon) will help to coordinate this effort. Ensure that the most serious bugs are identified and fixed early. It’s important to the success of the project that we continue to produce regular releases, and showstopper bugs risk delaying our release dates and adversely affecting the next release cycle. The release management $url[http://people.ubuntu.com/~ogasawara/weatherreport.html,weather report] is one tool which helps monitor this process, though a great deal of coordination is required in order to provide it with useful data. Communicate about known bugs. It is inevitable that there will be known bugs remaining in each release, and we should do our best to advise our users about them, including any known workarounds. The $url[http://www.ubuntu.com/testing/intrepid/beta#Known%20Issues,Ubuntu 8.10 beta release] notes are a good example of this. I think that to do well in all of these areas would be a good goal for quality in Ubuntu. $url[http://mdzlog.wordpress.com/2008/10/29/ubuntu-quality/,link]