Planet Twisted

Stephen Thorne: Trace that route

2008-07-20T01:31:21+00:00

$ traceroute6 mirror.ipv6.internode.on.net
traceroute6 to mirror.ipv6.internode.on.net (2001:44b8:8020:7a80::20) from 2001:44b8:61::e3, 30 hops max, 12 byte packets
 1  2001:44b8:61::e2  87.049 ms  89.126 ms  125.079 ms
 2  vl67.cor1.adl6.internode.on.net  98.27 ms  77.689 ms  77.303 ms
 3  mirror.internode.on.net  109.942 ms  177.193 ms  148.519 ms

Huzzah. Internode is now running a tunnel broker. :)

Christopher Armstrong: Joss Whedon Writes Stuff

2008-07-19T03:54:36+00:00

Check out Dr. Horrible's Sing-Along Blog. It's a video musical. It's by Joss Whedon, famous Hollywood writer/director/terrorist. It's free to watch on that web site until Sunday, July 20th, 2008. After that you'll be able to purchase it for a nominal fee.

Here are some things which he said about this project that I think are cool.

The idea was to make it on the fly, on the cheap – but to make it. To turn out a really thrilling, professionalish piece of entertainment specifically for the internet. To show how much could be done with very little. To show the world there is another way. To give the public (and in particular you guys) something for all your support and patience. And to make a lot of silly jokes. Actually, that sentence probably should have come first.

...

It’s time for the dissemination of the artistic process. Create more for less. You are the ones that can make that happen.

Glyph Lefkowitz: Don't Call It Blogging

2008-07-17T13:11:54+00:00

Despite my own impeccable credentials as an elite cyber-hacker, I am friends with a number of people who are bewildered by the profusion of different technologies that the internet now affords us to interact. I recently had a conversation where someone was just confused by the whole "blogging" thing. Why do people blagoblog on the intertron? What is the point? I'm a prolific "blogger" myself, I guess, but I found myself sympathizing as I tried to explain.

I'm a huge fan of the activity of blogging, but I have never liked the word, "blogging". I never really understood why until I was attempting to explain what it's all about.

For thousands of years — well, okay I don't have any citations of exactly how long, due to the evolution of English as a language, but, for a really long time — we've had one word for the activity of "blogging". We called it writing. That's all you're doing when you're blogging.

If we were to describe the activity of a Sumerian scribe pressing symbols into soft clay, we'd say they were writing on that clay. An ancient Egyptian putting words onto a sheet of papyrus: they are writing. Similarly, we don't typically have separate words for "scrolling", "codexing", "booking", "newspapering", "magazining", and so on. Each new technology for moving writing around didn't need a new verb. So why has "blogging" gotten one?

I think there is a good reason this term exists, but that reason doesn't justify the term, it provides a warning, and a reason to try to actively resist the term and just say "writing". The web is a more radical and democratizing shift in publishing technology than any of the ones which preceded it, so publishing on the web (especially automated publishing, as on a blog) affords a feedback cycle where the author and the audience are effectively peers. In fact, the nature of the terms "author" and "audience" has changed; formerly a description of social classes, the people who produce and the people who consume, they have been re-framed as roles within an individual conversation. You might be the audience when you're reading someone else's blog, but ten minutes later you can easily reverse that relationship with that author as you're writing your own. This extremely rapid cycle has given a wholly new quality to the style of many blogs, unseen in any prior form of written media.

So, why resist the term "blogging"? It confuses the possibilities that the medium presents with conventions that it enforces. Writing is a powerfully diverse art. A lot of it's good, a lot of it's bad. "Blogging", however, is more specific, and unfortunately implies a sort of perpetual half-finished conversation. It calls to mind a semi-private, informal, ephemeral, link-heavy style of extremely short-form writing. This form has its masters: Tycho of Penny Arcade infamy leaps to mind immediately. It also has a sea of mediocrity. Statistically speaking, you can probably click the 'next blog' link at the top of this page for an immediate example. I don't have a problem with any of this. Even the "mediocrity" is just evidence of the degree to which this is empowering people: much of what I'd consider "mediocre" just isn't relevant to me, and isn't written for me.

But blogs can be, and are, so much more than that. They are a disruptive technology in the world of publishing, where any style of writing can easily be published, circulated, and promoted. One can write an entire novel, serialized chapter-by-chapter as blog posts. Many people have, in fact, done this already. You don't even need to emulate older forms of writing to step outside the style implied by "blogging". The tools that the web affords — instant publishing, hyperlinks — are ideal for collaborative scientific research. Hyperlinks take the work out of footnoting.

Prominent web writers who I respect also seem to avoid the use of the term "blog". Joel Spolsky refers to other people's blogs, but the term "blog" does not appear anywhere describing his site, despite the fact that there is quite a bit of self-descriptive text that refers to "this site". Paul Graham goes a step further, foregoing many traditional blog trappings and has a link that says, simply, "Essays". I wonder if it's for this reason.

So, if you need to explain to someone who doesn't quite get what all the whole "blogging" thing is about, don't talk about social dynamics and the singularity and the mass popularization of media. That's all great stuff, but it's a confusing distraction. It's just like writing a book — or, more likely, a magazine. Except you don't have to talk to a publisher. And you don't have to have an editor. And it's free. And the publishing part doesn't actually take any time. And it's accessible from anywhere in the world. And you can read it on your cell phone. When you stack up all the advantages, the lack of some bound paper doesn't seem like a big deal.

If you find this explanation useful, feel free to point your relatives at this post. Tell them that you saw it on my blog, but don't tell them I blogged about it. Tell them I wrote about it.

Duncan McGreggor: OSCON 2008

2008-07-16T20:56:41+00:00

Hey all, thanks to a friend's amazingly generous offer, I'll be attending OSCON this year :-) I only have to pay for my airfare and food! I've contacted several people already who I know are going to be there (including Van Lindberg of Haynes and Boone and Bradley Kuhn of the SFC and the SFLC), and look forward to meeting up with others. Leave a comment or email me if you're going to be there!

Itamar Shtull-Trauring: The Good War, or, The Most Famous Classic Blunder

2008-07-14T12:13:25+00:00

I'm reading some essays Chomsky wrote in the 1960s, about Vietnam... It's worth remembering that Vietnam was a liberal project, supported by two Democratic presidents. Here's an update on the war our current batch of Democratic leaders can all believe in:

KABUL, July 9 (Reuters) - Foreign troops led by NATO and the U.S. military must exercise caution to avoid further civilian casualties while hunting militants in Afghanistan, or people will rise against them, lawmakers have warned.

...
"They should not act like the former Soviet Union did in Afghanistan," the [Afghan upper house of parliament] said in a statement, the Anis daily newspaper reported.

It was referring to the 10 years Soviet occupation of Afghanistan in the 1980s when Moscow's forces relied heavily on air raids in the war against guerrillas known as mujahideen, or Muslim holy fighters, causing widespread destruction and civilian deaths.

Update:

From today's Washington Post:

Congress has quietly used fiscal 2008 legislation on military construction to signal that it plans on a long-term military presence in Afghanistan.

In the recently approved supplemental funding bill for the Iraq and Afghanistan wars, legislators approved construction of a $62 million ammunition storage facility at Afghanistan's Bagram Air Base, where 12 planned "igloos" will support Army and Air Force needs.

...

In another sign that U.S. troops will be there a long time, the Army requested, and Congress provided, $41 million for a 30-megawatt power plant at Bagram. It is capable of generating enough electricity for a town of more than 20,000 homes.

Allen Short: PyMeta 0.3.0 Released

2008-07-12T22:07:19+00:00

Originally when I was implementing PyMeta I was sure that the input stream implementation that the Javascript version used was inefficient. Rather than having a mutable object with methods for advancing and rewinding the stream, it has immutable objects with "head" and "tail" methods, which return the value at the current position and a new object representing the next position in the stream. All that allocation couldn't be healthy.

Turns out I was wrong. I misunderstood the requirements for OMeta's input stream. Various operations require the ability to save a particular point in the stream and go back to it. To further complicate matters, arguments to rules are passed on the stream by pushing them onto the front, and rules that take arguments read them off of the stream. This is very handy for certain types of pattern matching, but it totally destroys any hope of simply implementing the input as a list and an index into it, because there has to be a way to uniquely identify values pushed onto the stream. If a value gets pushed onto the stream, is read from it, then another one is pushed on, both of them have the same predecessor, but they don't have the same stream position. It becomes more like a tree than a sequence. JS-OMeta handled this by just creating a new stream object for each argument. I didn't give up soon enough on my clever idea when initially implementing PyMeta, and it grew more complicated with each feature I implemented, involving a bunch of lists for buffering things and a complicated mark/rewind scheme.

After writing a rather complicated grammar with PyMeta, I began to wonder if I could improve its speed. By this time I knew the JS version's algorithm was less complicated so I decided to try it out. It cut my grammar's unit tests' runtime from around 40 seconds to 4 seconds. Also, it fixed a bug.

Moral of the story: I'm not going to try to implement clever optimizations until I understand the original version any more. :)

I've released a version with this new input implementation. Get it in the usual place.

Twisted Matrix Laboratories: June/July TSF Sponsored Development

2008-07-10T21:02:47+00:00

The second (of what should end up being many) round of TSF sponsored development has just wrapped up. Like last time, this was two weeks of work by yours truly (with an afternoon of Glyph's time spent reviewing code to keep things moving). This time around there were fewer tickets waiting to be reviewed when I got started, however there was still plenty of code to look at. Over the full period, I reviewed:

#1144 Documentation: twisted.internet.reactor does not appear in the API docs
#1253 Create index.xhtml files for non-core doc trees
#1255 Update copyrights in the man pages
#1878 twisted.web.monitor traceback_ AttributeError: class IChangeNotified has no attribute '__class__'
#1890 move examples from core into correct sub-packages
#1900 Error in documentation online
#2169 twisted.plugin documentation errors
#2208 Standardize on the Python shebang line
#2438 Get rid of references to maintainer email addresses from code
#2552 broken links in intro.xhtml
#2607 conch.checks.SSHPublicKeyDatabase calls os.seteuid/os.setegid even if it's not necessary
#2716 Eliminate relative imports from twisted.conch
#2815 Update VFS backends to an async interface
#2821 create twistd plugin for vfs
#2845 twisted.internet.thread._putResultInDeferred should be public
#3182 Trivial typo in twisted.internet.protocol
#3257 Rewrite twisted.web.static.File.directoryListing to not use woven
#3269 curses.setupterm must only be called once per process
#3300 twistd should support setting the syslog facility
#3326 Typo in gtkmanhole.py

Other tickets I worked on included:

#1069 log observers that throw exceptions should not be removed
#1152 xmlrpc.html doesn't describe how to return errors to the client
#1291 Expose "process exited" hook on ProcessProtocol_ turning current processEnded into user-overriddable behavior
#1493 static File web module doesn't support byte ranges
#2303 Deprecate setUpClass and tearDownClass and_ if possible_ fix the subclassing behaviour.
#2327 Intermittent failure in PB tests
#2631 Update coding standard to indicate preference for TestCase methods which being with "assert" and which do not have an underscore in their name
#2874 _sslverify.problemsFromTransport should be deprecated
#3029 documentation for twisted.python.deprecate.deprecated is incomplete (and other sundries)
#3059 twisted.internet.tcp.Client.getPeer incorrectly returns hostnames
#3116 Errors in processEnded can cause processes to be eternally reaped
#3159 t.i.utils process functions should have a default cwd of None_ not '.'
#3218 SSL disconnection sometimes hangs indefinitely with pyOpenSSL 0.7
#3255 Trial fails to display the line where the error occured in case of SyntaxError
#3300 twistd should support setting the syslog facility
#3301 superfluous local in AMP.__init__
#3305 CR IAC ignored in TelnetClient
#3306 twisted.test.test_ssl.StolenTCPTestCase has a number of defects
#3339 mailmail raises an exception instead of giving an error message

In particular, I spent a lot of time on #1291 which will make it possible to control child processes more precisely with Twisted by separating out the notification that a child process has actually exited from the notification that all of its file descriptors have been closed.

Thanks to the SFC (<http://conservancy.softwarefreedom.org/>) and all of the sponsors (<http://twistedmatrix.com/trac/wiki/TSF/FoundingSponsors>) who made this possible, as well as to all the other Twisted developers who helped out by writing or reviewing code.

Glyph Lefkowitz: Conference FAIL

2008-07-09T21:54:08+00:00

Last night at a dinner with Ivan Krstić and Itamar Shtull-Trauring, we were all lamenting that too many (all?) software conferences focus specifically on positive results. This is what you want, of course, if you treat a conference as purely a marketing venue. However, most learning takes place based on something that someone did wrong and then needed to correct, not something that they did right.

All of the great software developers I know have at least one great story of how a project they were working on was a complete disaster. Often these projects are shielded from the public eye, since nobody wants to talk about failure. So, how do we make a public discussion of these ideas socially acceptable?

Thus, an idea was born: FAILcon. The idea is simple: submitted talks and papers must be related to projects which failed in an interesting way. The larger the better, of course — the bigger they are, the harder they fail — but anything that failed in an interesting way would be a valid subject for discussion.

I'm writing about it so that it won't be forgotten, because I think it's a great idea. But I doubt that any of us are going to organize a conference any time soon. So please, steal this idea. Does anyone out there with conference-organizing skills want to get something together based around the common theme of failure?

Moshe Zadka: Upgrading to Ubuntu Hardy on my Lenovo 3000 N100

2008-07-09T16:43:26+00:00

Everything seemed to work, except for my wireless. Yes, big deal…

After googling a little bit, I found it’s a bug in the new kernels. I rebooted and chose the “2.6.15″ kernel, and everything worked fine.

Paul Swartz: Free Hugs Campaign

2008-07-09T15:25:10+00:00

Free Hugs Campaign

Paul Swartz: "Plaintiff has a great deal to say, But it seems he skipped Rule 8(a). His Complaint is too..."

2008-07-09T00:54:38+00:00

“Plaintiff has a great deal to say,
But it seems he skipped Rule 8(a).
His Complaint is too long,
Which renders it wrong,
Please rewrite and refile today.”

- A federal judge responds to a lawsuit that was too long with a limerick (via The Associated Press)

Paul Swartz: Sliced bread 'a world first' | NEWS.com.au

2008-07-08T17:20:56+00:00

Sliced bread 'a world first' | NEWS.com.au: Happy (belated) birthday pre-sliced bread! It turned 80 on July 6th.

Kevin Turner (LJ): little bear's vegetable omelette

2008-07-08T02:28:53+00:00

It's that time of year again. Does anyone want to join me for da Vinci Days in Corvallis this year? My current plans are to be there on Saturday the 19th. (I have a thing in Keizer to be at on Sunday afternoon.) Kinetic sculptures, human kaleidoscope, fun stuff.

Let me know if you're interested in going.

Paul Swartz: 1 Year Anniversary!

2008-07-07T20:04:31+00:00

I didn’t note it when it happened, but 6/28 was the 1 year anniversary of working with the Participatory Culture Foundation. I didn’t really know what to expect when I started with them, but it’s been an awesome ride so far.

Jonathan Lange: I Love Data

2008-07-07T08:22:37+00:00

At work, we recently started to graph the time it takes to access a branch in Launchpad via SSH, both on our staging and production servers.

I love this so much. Having this data is liberating. It's like turning on a light in a dark room: suddenly I can go from uncertain, careful, hesitant steps to bold, confident strides.

In fact, the metaphor stretches further. This new light has revealed objects of interest (that is, spikes in the time taken) that I wasn't aware of before. I need to know how long these spikes last, how they correlate with load on the system and so forth. Soon there'll be more graphs, and I'll be able to correlate them and analyze them and suck on their delicious, numerical marrow (the light metaphor long abandoned).

Regardless of whether it's liberating illumination or nourishing, savoury meat, this new graph makes me wonder why I don't chart other things I care about. Having such graphs would help me see the impact of my actions. I could chart things like my bank balance, my waistline or the number of Latin words I learned this week. Then I could answer questions like "do I spend more on Tuesdays?" or "can I afford this iPod/cake/gerund?". This is important, because when I am tempted with a sleek/delicious/perplexing iPod/cake/gerund, I fall back on my own judgment. I think it's time to confess, dear reader, that my own judgment isn't very good. And yet I continue to trust it.

The reason I don't have such graphs is that they are inconvenient to maintain. A graph of branch access times is easy. All you need to do is describe how to get the measurement, and then do a bit of once-off set up. If you want to graph your body weight, you need to get on the scales at a fixed time and then look at the dial and adjust for parallax and then write a number down somewhere and maybe note down whether this is before or after a meal and then take the number and then add it to a spreadsheet. Most online banking sites I've seen are even less convenient than this.

Getting data on life is hard, but for programming it's easy. What things do you care about on your project? Features, user experience, how fast bugs get fixed? Can you quantify these things? Can you make a pretty picture out of it? If so, do it now! Post your answers here, set up something like Cricket and then profit!

P.S. I can't believe I got to the end of this post without saying how awesome the Canonical sysadmins are. Let me correct that now. The Canonical sysadmins are heck of awesome. They could simply walk into Mortor.

Duncan McGreggor: Native LoadBalancing for Twisted Apps

2008-07-05T21:51:40+00:00

Yesterday, right before midnight, I tagged the 1.1.0 release of txLoadBalancer on Launchpad after completing the last of the planned features. There are some pretty radical changes that have been developed for this release... and the coolest part is this is just the beginning :-) (See the TODO if you don't believe me!)

You can checkout from lp:~oubiwann/txloadbalancer/1.1.0 or download from PyPI. If you're a PyPI expert, I've got some questions for you at the end of this post... Been having some sucky experiences with PyPI lately :-(

So here's what's going on with txLoadBalancer:

Improved API

The biggest thing you'll notice if you've switching from PythonDirector is the massive overhaul the API has undergone. Things are cleaner and generally more modern, with a concise and well-defined module layout.

New Load Balancing Algorithm

I've added support for a weighted host scheduler. Given a weight that represents the frequency a host should be used, a host will be randomly selected, based on it's weight. For example, with two hosts, one having a weight of 1 and the other having a weight of 3, host 2 will be chosen about 75% of the time and host 1 will get about 25% of the requests.

Right now, this algorithm has to make several calls to other parts of the code in order to get all the data it needs (it also builds some crazy iterators). As such, it's rather slow and performs poorly when compared to the very light-weight least-connections algorithm. That being said, the next release will include optimizations for the weighted scheduler that make use of a Twisted timer and caching.

Native Twisted Load-Balancing

Here's the sexiest part: you can now load-balance your Twisted application by using the txLB API; you don't even need to run the load-balancer as a separate app! This evolved as a feature after a conversation with an as-yet unnamed cloud hosting provider, a follow-up discussion with the Divmod team, and then some quiet pondering about ways in which Twisted applications could be supported in cloud/grid/massively-multi-core architectures.

The "self load-balancing" API in txLB is not a comlete solution for grid-hosting, but it is a first step in one direction (we've been discussing lots of others, too, including the use of our deployment tool).

Before I show you how to use the self load-balancing API, let's take a quick look at a normal Twisted application service:

from twisted.web import static, server
from twisted.application import service, internet

application = service.Application("Demo Web Server")
web = server.Site('/home/oubiwann/public_html')
service = internet.TCPServer(7001, web)
service.setServiceParent(application)

You start that with the command twistd -noy myweb.tac. For use with the next example, you can also start two more, one on port 7002 and the other on port 7003.

Now here's what you do to make a self load-balanced app:

from twisted.application import service

from txlb import manager
from txlb.model import HostMapper
from txlb.schedulers import leastc
from txlb.application.service import LoadBalancedService

proxyServices = [
  HostMapper(proxy='127.0.0.1:8080', lbType=leastc, host='host1',
      address='127.0.0.1:7001'),
  HostMapper(proxy='127.0.0.1:8080', lbType=leastc, host='host2',
      address='127.0.0.1:7002'),
  HostMapper(proxy='127.0.0.1:8080', lbType=leastc, host='host3',
      address='127.0.0.1:7003'),
]

application = service.Application('Demo LB Service')
pm = manager.proxyManagerFactory(proxyServices)
lbs = LoadBalancedService(pm)
lbs.setServiceParent(application)

As you would expect, you need to indicate the proxy host:port, the algorithm to use, and the hosts that are to be balanced. The host setup assumes that you have three services running on localhost ports 7001, 7002, and 7003. All that's needed now is to just run that code with the usual twistd -noy myapp.tac. Also, for demonstration purposes, this is a somewhat simplified example of what is possible.

This may seem like a lot of extra work when compared to the simple web host above, but think about it: we're load-balancing here :-) This saves you from having to manage yet another application. With a few extra lines of code, you can keep it all in one place and have it manage itself.

Note that this API is in development and continuing to improve. The example above is from code running in trunk. For the more verbose configuration that is in the 1.1.0 release, be sure to see ./bin/txlbWeb.tac from the source tarball. To play with the latest and greatest, you'll want to checkout the code here: lp:txloadbalancer.

Other Goodies

Here is some other good stuff in the release:

You can now ssh into a txLB instance and mainipulate the load-balancer in real time from an interactive Python interpreter.
You can change the proxy to listen on a different port while the application is running (no restart requred!).
Changes made to the configuration while running are no longer volatile; they are saved to disk (and your old config gets backed up).
Work from Apple, Inc. was included in this release, too (they use the old PythonDirector in their Calendaring server). This includes a bug fix and management socket feature.
There is a significant jump in performance between this release and the previous one. I believe this to be due to the separation of concerns in the API, but haven't yet confirmed that.

Coming Work

There are a lot of exciting features coming for txLB. Just to name a few:

improved weighted algorithm
resources-based algorithm (a scheduler that determins the weight of a proxied host by memory, CPU, etc., utilization)
smarter proxied host failover and recovery
a heartbeat manager
txLB-powered application cloning (when started, an app will determine if it needs to run the clone as the managing load-balancer or simply as a proxied host)
auto-discovery of balanced hosts
proxy fail-over (a balanced host taking over as manager in the event that the manager goes down)
ApacheMQ/Stomp integration
LDAP/RADIUS authentication

Additionally, I'll be putting together some basic performance metrics contrasting Apache and load-balanced Twisted apps. I will also be comparing previous versions of txLB/PythonDirector with the latest release(s).

Problems with PyPI

I will close this post on a sad note: PyPI used to be an amazing experience for me (a couple years ago, when it was still being called "cheeseshop"). Everything worked as it was supposed to. This hasn't been the case when I've used it recently (over the past few months).

For all that I say about PyPI, I allow for the fact that I may just be missing something, and it may be entirely my fault. That being said, I spent about 3 hours online last night combing though the SIG mail list, the bug list on sourceforge, and blog posts about setuptools and PyPI, and could find no answers to my questions. Well, with the possible exception of a bug report, but it doesn't look like it was confirmed by a PyPI team member, so I'm not sure if it's valid or not.

Here are my issues:

When I upload my project using python setup.py [sdist|bdist_egg] upload, no metadata defined in my setup() function is presented on my package's PyPI page. When I click the metadata link, it's only got three sparse lines.
When I manually upload from the package's PKG-INFO itself, all the metadata is presented on the page as it should be, with the exception of the long description. It is in plain text instead of ReST (I am checking that it is valid ReST using distutils settings of reporter.halt_level = 5, reporter.report_level = 1, settings.pep_references = False, and settings.trim_footnote_reference_space = None; these are the same settings that Zope Corp uses to verify the ReST that it uploads to PyPI).
When I manually edit the long description in the form, I get the same thing: plain text, no ReST.
When I upload a package that is displayed properly on PyPI (such as zc.twist; uploaded as one of my projects by chaning the name), I get the same problem (this is why I think it might be something that I'm doing wrong...): no metadata, and when I upload the PKG-INFO manually, no ReST.

Why, oh why, cruel fates, does this not work any more? I used to be able to upload to PyPI without any of these issues...

Paul Swartz: SimplyNoise.com - The best free white noise generator on the Internet.

2008-07-05T20:46:40+00:00

SimplyNoise.com - The best free white noise generator on the Internet.: Pretty simple Flash application which generates white noise right from your browser. Personally I like it at about 60%.

Jonathan Lange: Tired Hippo

2008-07-05T12:01:28+00:00

Chronomancy... is generally a fictional and sensational school of magic. Although the school is based on quantum physics and certain scientific theories, there is no concrete evidence of the perfected use of time manipulation.

Thank you Wikipedia.

Jonathan Lange: Five Years

2008-07-05T11:21:31+00:00

That's not quite how long it took Glyph to write a post explaining what he meant when he told me that xUnit should use the visitor pattern instead of the composite pattern.

I'm going to write a more detailed response later, God willing. I just wanted to flag the post and say thanks to Glyph for finally posting it — the hassling must have worked.

Jonathan Lange: The Economist Index

2008-07-05T11:21:30+00:00

The Economist occasionally publishes a "Big Mac Index" — how much the world's favorite "burger" costs in each country, translated into a common currency.

Why they don't publish an index of how much an issue of the Economist costs? All of my issues have the price of the newspaper in lots of Asia-Pacific currencies. A small sample:

Australia, 9.39855 USD

Cambodia, 6.00 USD

China, 10.43655 USD

Hong Kong, 7.6917 USD

India, 5.0522 USD

Japan, 10.70535 USD

New Zealand, 8.6669 USD

Can anyone tell me why the price varies so much?

Jonathan Lange: Three-ways on Pidgin?

2008-07-05T11:21:29+00:00

Dear Lazyweb,

How do I initiate a three-way chat with Pidgin 2.2.1?

jml

Jonathan Lange: Merging New trunk Features to a Development Branch (redux)

2008-07-05T11:08:02+00:00

Oubiwann has recently posted about the joys of using Combinator to do branch-based development using Subversion. I thought it'd be fun to do the same post, except this time with Bazaar.

Background

You're working on a project called "Project", you have a copy of the mainline branch (i.e. 'trunk') in your src directory.


~$ cd ~/src/Project
~/src/Project$ ls
trunk

You want to implement a new feature, so you branch trunk to work on it:


~/src/Project$ bzr branch trunk viking-feature-836

Bazaar is a version control system, not a PYTHONPATH-managing system, so it doesn't maintain a global list of projects and the branches that are currently active for each project.

Perhaps your company focuses on historical invasions of Britain. You decide to start work on another feature:
lass objects.


~/src/Project$ bzr branch trunk norman-feature-1066

You multi-task for a bit, until you finish 'viking-feature'. You decide to merge 'viking-feature-836' into trunk.


~/src/Project$ cd trunk
~/src/Project/trunk$ bzr merge ../viking-feature-836
~/src/Project/trunk$ bzr ci

At this point, you begin to suspect that Bazaar treats branches as first-class objects. However, at this point, a developer on the obverse side of your continent calls you,

"Where's your viking feature? I need it to invade Britain!"

"I've just put it into trunk. Have you got the latest copy?"

"Yeah, I do, I just pulled from trunk."

"It's in trunk, you fool! ... Oh, wait, gimme a sec."


~/src/Project/trunk$ bzr push
Pushing to bzr+ssh://bzr.example.com/Project/trunk...
~/src/Project/trunk$

"Try now."

Merging

OK, enough background, let's merge.

Say you need some of the changes in trunk in order to finish work on your norman feature. No problems.


~/src/Project$ bzr merge trunk norman-feature-1066
~/src/Project$ bzr ci -m "Merge from trunk."

It's hard not to feel smug at this point.

Wait a second, you also want to look at the experimental branch that a friend is working on:


~/src/Project/norman-feature-1066$ bzr merge bzr+ssh://yourfriend.example.com/branches/sealion-1946
~/src/Project/norman-feature-1066$ bzr diff | less # better double check this one
~/src/Project/norman-feature-1066$ bzr revert # nope, doesn't seem like a good idea

Summary

Bazaar treats branches as first-class objects and treats trunk just like any other branch. Although Combinator is great for branch-based development in Subversion, it is more complex and less flexible than doing branch-based development in Bazaar.

With Combinator, you lose history when you merge in changes from trunk, with Bazaar you don't.

With Combinator, you can only merge in changes from trunk, with Bazaar you can merge from any branch.

With Combinator, merging from trunk leaves a bunch of changed files in the trunk checkout on your system (this has tripped me up more than once). With Bazaar, this doesn't happen.

Jonathan Lange: It's a Gibbon

2008-07-05T11:08:01+00:00

Over the weekend I upgraded my laptop to Ubuntu 7.10 aka "The Gutsy Gibbon".

Obvious improvements include having Pidgin installed and a much superior version of Deskbar. There's also a swanky new panel to actually control display settings (dual monitor and all that jazz). Unfortunately, I can't quite get it to work just yet. My 22" 1680x1058 LCD will just have to sit there, gathering dust :(

In another weirdness, all of my fonts are suddenly much, much bigger. I had to resize everything down to 8 or 9 to make them sane.

My pretty Emacs has also stopped working, but that's to be expected. It's a third-party package that hasn't yet been built for Gutsy.

Other than that, seems to be a fairly routine upgrade. Is there anything exciting that I've missed?

Jonathan Lange: Around like Grover

2008-07-05T11:07:59+00:00

What's up hep-cats?

I've been travelling around the world, weaving in and out of timezones like Jason Bourne navigates facial expressions. Last week's adventure took me to tropical Dunedin — the southiest outpost of Canonical Ltd. (I'm trying to get a mention on Language Log.)

Those few Hobartians who keep reading will notice that Dunedin, well, umm... here's a photo:

Pay careful attention to the hills in the background.

Jonathan Lange: Cracked Mac

2008-07-05T11:07:58+00:00

OK, I've started with this whole photography thing, I might as well run with it.

My Mac cracked:

Jonathan Lange: The most retentive thing I've done

2008-07-05T11:07:57+00:00

Jonathan Lange: Fake Plastic Tests

2008-07-05T11:07:56+00:00

Ever wondered what's the difference between a FakeReactor, a StubReactor and a MockReactor? Find out in this brief article on test doubles.

Jonathan Lange: Obligatory

2008-07-05T11:07:55+00:00

Not using Facebook anymore.

Have fun kids.

Jonathan Lange: WoW account temporarily suspended

2008-07-05T11:07:53+00:00

Howdy Partners,

My World of Warcraft account is temporarily suspended for silly administrative reasons. I'm not in any rush to clear those up, so don't expect to see me on Khaz'goroth or Frostmourne any time soon.

Regards,

jml

Jonathan Lange: What I do all day

2008-07-05T11:07:52+00:00

Kick-arse summary here: The great source code supermarket.

Jonathan Lange: User Experience — When Reality Attacks

2008-07-05T11:07:50+00:00

I've just got back from a hectic week in London, where members of the Bazaar community got together and thrashed out a bunch of important topics.

We talked about "user experience" and how we all want Bazaar to be a joy to use. More than one person said that we have been focusing too much on features and performance instead of user experience. The term was never really pinned down, but it's fair to say that there are things other than convenience and speed that affect how users feel while using Bazaar and that we need to work on those things, once we figure out what they are.

I think I might know the name of one of them: errors. Next post: "Notes on error".

Jonathan Lange: bzr-removable

2008-07-05T10:40:06+00:00

The plugin I talked about in a previous post is now at https://launchpad.net/bzr-removable. Please file bugs.

Thanks to Michael Hudson and Daniel Watkins for submitting patches.

You can get the plugin by typing 'bzr branch lp:bzr-removable' on the command line.

Jonathan Lange: Neat Bazaar feature

2008-07-05T10:40:04+00:00

Ever find yourself working away on a branch, enjoying yourself and getting just a little carried away? Maybe you're working on a feature and you notice and fix a bug that's not strictly related to that feature.

If you catch yourself in time, there's a nice little feature in Bazaar that can help with this: bzr merge --uncommitted. It will merge in the changes that you've made to your working tree but haven't committed yet.

e.g.


$ cd some-feature-branch

... hack hack hack ... oops!

$ cd ..

$ bzr branch trunk bug-fix-2357

$ cd bug-fix-2357

$ bzr merge --uncommitted ../some-feature-branch

$ bzr ci -m "Fix up bug 2357. Found this while working on some-feature."

$ bzr send

Don't know what bzr send does? Trust me, you want to find out.

Jonathan Lange: Somebody Do This Please

2008-07-05T10:39:55+00:00

<jml> What's the difference between Gobby and a wiki?
<mwh> NAT

Jonathan Lange: OK I lied

2008-07-05T10:09:16+00:00

The next post is actually about a Bazaar plugin that I've now got ready to share. To get it, 'bzr branch lp:~jml/+junk/merged-branches'.

Once you've got it, run 'bzr merged-branches' in the trunk of your project. It will then show you all branches in sibling directories that are safe to delete. That is,

They have no uncommitted changes.

They have no "unknown" files. (Files outside of version control that haven't been explicitly ignored.)

They have no shelved changes. The plugin will only check for this if it can find bzrtools.

The branch is now at the point where it works for me, but it still belongs in '+junk' — here's why:

It assumes that you have a trunk branch in the same directory as all of your other branches.

It assumes that branches and working trees are the same thing.

It's called 'merged-branches' when it really means 'safe-to-delete'.

It doesn't make it easy to see why a branch is not safe to delete.

It doesn't let you customize the conditions of the search. Maybe you want to see all branches with uncommitted changes.

Still, if you are like me and make a lot of branches, it's quite useful. I'll tolerate bugs, accept patches and welcome encouragement.

Jonathan Lange: What I meant

2008-07-05T10:09:13+00:00

z3p recently blogged about a comment I made in a code review. In the review, I linked to:

That's a negative and grumpy way of phrasing an idea that I've come to value a lot: good code expresses its intent clearly.

When looking at a patch, the reviewer needs to understand two things: the intent of the code and the intent of each change to the code. To be clear on the former, you need:

intent-revealing names.

good abstractions / interfaces.

good, small tests.

simple implementations where possible.[1]

docstrings where appropriate

comments where appropriate.

That's not exhaustive, but it's in a rough order.

To be clear on the intent of your change to code, you need:

Small patches.

A good bug / spec with a good, short summary.

A review request letter, summarizing your implementation strategy, any compromises you made, gaps in testing, future work etc.

That's not exhaustive either. In #2037, I didn't understand the motivation for lots of the code, nor for some of the changes to the code.

I'm indebted to Andrew Bennetts for teaching me that the first duty of a reviewer is to ensure that the code is clear and to Tom Berger for reminding me that compromises are worth noting.

----

[1] Actually, this reminds me of something I heard a preacher say, "before I give a sermon, I go through it, find everything clever, and take it out" (I paraphrase, not having a reference on hand).

In as much as sermons and code should both be ego-free communications of ideas, I think this is sound advice for hackers.

Jonathan Lange: Real Life

2008-07-05T10:09:10+00:00

Inspired by Mikey, I've set up my own real-life blog.

I've done this one using Blogger's "upload to remote server" feature. It looks nice, has the features I need and means less WordPress & PHP. I'm hoping to migrate this sucker to the same technology.

Jonathan Lange: Another Bazaar story

2008-07-05T10:09:04+00:00

I'm currently hacking away on Launchpad's support for "stacked branches", something that will really make Launchpad's codehosting a joy to use.

At the moment, I'm writing some tests that require a user to login. This was becoming a bit cumbersome, until I remembered something: Tim has recently landed some code to make this easy. But how do I get these changes without messing up all of the work I'm doing now? bzr shelve to the rescue!

'shelve' interactively goes through each change you've made to your current working tree and allows you to decide whether to keep a change or shelve it. It comes with a twin command 'unshelve', which lets you interactively restore your changes.

In this case, I don't even care about the interactivity, so here's what I did:


# Shelve my changes
bzr shelve --all
# Fetch the latest trunk
cd ../trunk
bzr pull
# Merge it into my branch
cd ../stacking
bzr merge ../trunk
bzr ci -m "Merge in changes from trunk to get login testing improvements."
# Restore my changes
bzr unshelve --all

The 'shelve' command comes with the bzrtools plugin, and I am basically in love with it.

As a parting shot, I should mention that things like bzrtools aren't accidents. They are natural and inevitable when you have good APIs in a high-level language and a very nice plugin system.

And now I'm off to keep working on this branch.

Jonathan Lange: Aren't code reviews great?

2008-07-05T09:24:08+00:00

Ben Sussman-Collins writes about programmer insecurity and how a lot of programmers dread having others see their code.

Some of the best programmers I've worked with are in love with code reviews, and all great writers have editors. (The "linked list" is in no way exhaustive.) Personally, I see reviews as being a great opportunity to learn.

Sussman-Collins also writes about how distributed version control can exacerbate the "isolated genius" problem by shielding ones code from the public eye. This is definitely a big potential drawback of DVCS, but one that's largely mitigated by a culture of short-lived branches and an easy-to-use public registry of all branches for a project.

Jonathan Lange: Really Satisfying

2008-07-05T09:24:07+00:00

I have just returned from the shops with a Snickers bar. The packet says that one out of every six Snickers bars will instantly win me a free Snickers bar.

So, statistically speaking, how many Snickers bars do I have? Show working.