Planet Twisted

Paul Swartz: M1FY Goes West

Mon, 08 Sep 2008 02:41:43 +0000

M1FY Goes West: Communal tumblelog about our cross-country road trip

Paul Swartz: Into the Wild West

Mon, 08 Sep 2008 02:40:25 +0000

Into the Wild West: Meg’s tumblelog about her move to Utah

Paul Swartz: Alas, I’m coming to the end of my staycation. Luckily the rain NPR was predicting for this...

Sun, 07 Sep 2008 18:56:00 +0000

Alas, I’m coming to the end of my staycation. Luckily the rain NPR was predicting for this morning/afternoon doesn’t appear to be coming. That means I get to spend my day outside (although on the computer nonetheless). Today has mostly been a reading day, as was Friday. I’m a little over halfway through Picture of Dorian Gray and I read Into the Wild again on Friday. I did a couple of errands like vacuuming out the cars and buying a couple things that I needed, but mostly I’ve stayed around the house.

Thinking that today was going to be raining, I went out for a hike (more of a walk, really) on the North Country Trail. It stretches from New York to North Dakota, but I just did a couple miles out and back.

Tomorrow we’ll be getting the last of our things together for the road trip, then Tuesday morning we’ll be off for the Chi-town.

Paul Swartz: The Muskegon River, from Tess’ front yard

Sun, 07 Sep 2008 18:25:00 +0000

The Muskegon River, from Tess’ front yard

Paul Swartz: "All art is quite useless."

Sun, 07 Sep 2008 01:57:00 +0000

“All art is quite useless.”

- Oscar Wilde, The Picture Of Dorian Gray

Paul Swartz: "The Stone Age didn’t end because we ran out of stones."

Sat, 06 Sep 2008 22:54:57 +0000

“The Stone Age didn’t end because we ran out of stones.”

- Sheikh Yamani, former OPEC oil minister

Stephen Thorne: Booting and Stuff

Stephen Thorne (noreply@blogger.com) — Sat, 06 Sep 2008 04:59:36 +0000

So it turns out that my epic journey into "why doesn't this board boot" this week had a really anti-climatic ending.

Just so everyone knows, the Intel DG945GCLF board, which is a mini-itx board with 1.6ghz atom 230 chip on it, will not boot from a hdd unless one of the partitions on that drive is marked bootable.

This might seem obvious, but it doesn't make much sense :( the BIOS is supposed to just load the first 512 bytes of the drive and run it*.

So, things I learned this week:

grub-install is a shell script.
What grub stage1, e2fs_stage1_5 and stage2 are.
The layout of the partition table.
Various tricks with 'dd' and 'xxd' to read partition tables.
Editing hex values in partition tables and the grub stage1 on the disk is kinda fun the first time.
Knowing what bytes are what on a raw disk is knowledge that can't be erased with beer.
Beer is good for day 3 of dealing with non-booting systems.

So after something like 15 kernel compiles, various operating systems and rescue disks, it turns out that I needed to make a 2 character change to the sfdisk script that created our partition layout.

* yes, i know this is a lie.

Paul Swartz: Luna napping on my leg while I read Time Magazine

Fri, 05 Sep 2008 20:01:00 +0000

Luna napping on my leg while I read Time Magazine

Paul Swartz: Maryland -> Pennsylvania -> Ohio -> Michigan

Fri, 05 Sep 2008 15:28:16 +0000

After getting Arlen and Meg up at 6:30am, we left for Michigan around 7:15am. Other than a lot of construction in PA and some traffic and rain in MI, the 11-hour drive was uneventful.

The three ladies, as well as Tess’ parents have all left for the Wheatland Music Festival. They’ll be there all weekend, while I’m “cat-sitting” for the cats that don’t really need any. Mostly I’m reading and enjoying doing nothing for a bit; neither the bluegrass-y music nor the huge crowd (camping with 13K other people) seemed attractive. I do have some errands to run, but those can wait until this later.

Kevin Turner (LJ): spicy tlc

Fri, 05 Sep 2008 01:14:25 +0000

I can't get in to the Python bug tracker right now (its promised registration email has not arrived), so I post this here in the meantime. (Update: Finally got the email. Bug filed with patch as #3801.)

The code: import cgi; cgi.parse_qsl('a=1&b=2&b=3')

The result in Python 2.5: [('a', '1'), ('b', '2'), ('b', '3')]

The result in Python 2.6b3+ (trunk:66224): {'a': ['1'], 'b': ['2', '3']}

Probably a cut-and-paste bug, cgi.parse_qsl is in fact implemented by calling urlparse.parse_qs.

Guess I should have my code running on the community buildbots after all.

Allen Short: Ecru 0.1.3

Allen Short (noreply@blogger.com) — Thu, 04 Sep 2008 23:05:21 +0000

Time for another Ecru release. This release should be a lot more stable: ctypes is no longer used for the Python interface. The major behaviour change I've added is support for Selfless objects, allowing for value equality: for example, Map objects with the same contents now compare equal. The rest of the work has been in code reorganization and preparation for concurrency support; vats are implemented now, and the REPL executes code by putting it into the vat's event queue. Eventual sends work now (i.e., "foo <- doStuff(x, y)"). Goals for the next couple releases are actual multi-vat execution, support for the 'when' expression, and portability to other OSes.

Paul Swartz: Ian Bicking: a blog :: On the RNC, Monica Bicking, Eryn Trimmer, and Protest

Wed, 03 Sep 2008 18:31:20 +0000

Ian Bicking: a blog :: On the RNC, Monica Bicking, Eryn Trimmer, and Protest: Wonderful post from Ian Bicking (whose sister (and her boyfriend) were arrested at the RNC protests) about why they’re protesting.

Cory Dodt: Google Chrome - no extensions!

Cory (noreply@blogger.com) — Tue, 02 Sep 2008 22:10:42 +0000

So, ugh. Chrome does not support extensions. That means no Adblock Plus, no NoScript, no FireGPG, and all those other tweaks we all rely on to get through our day - for each of us, a different set of tweaks, adding up to thousands of features Google won't be developing on its own and therefore nobody will have.

They have to rectify this situation, or quickly see Chrome fork (or worse for them - an anti-Chrome developer groundswell). I can only assume that this is a temporary situation. To tell them what you think, and in the hopes that this situation is only temporary (it's a 0.2 browser right now, after all) go to the support link.

Paul Swartz: John Ward of Hull’s The Northern Whale Fishery: The...

Tue, 02 Sep 2008 16:57:00 +0000

John Ward of Hull’s The Northern Whale Fishery: The “Swan” and “Isabella”

Cory Dodt: The most exciting part of Google Chrome

Cory (noreply@blogger.com) — Tue, 02 Sep 2008 16:55:12 +0000

You've probably already heard about tomorrow's release of Google's browser project, Google Chrome. (This link is broken, but broken quite differently from other Google 404's. It's a safe bet they are already proxying this page to some other server and will put something there When It's Ready.)

There are many awesome in this, but I'd like to draw you web developers' attention to something that promises to be as exciting as the browser itself:

Comic pages: page 9, page 10, page 11

These pages describe the testing apparatus used to develop Chrome. It describes a continuous integration server that processes every build against a perfect rendering of tens of thousands of sites. The perfect rendering is described as being "a schematic of what the browser thinks it's displaying". I read this as being an internal data structure representing the rendered view.

There are two reasons I love this. First, it means that Google will be Ready on Day One to render the web. It will be "another platform to support", but not to nearly the degree that (for example) Safari or IE7 is another platform to support. This kind of testing apparatus can only lead to a more compliant, more reasonable platform. (At the least, it should have only the same bugs Firefox does. I wonder what they used to make their benchmark renderings?) Edit: I know Chrome is based on WebKit, I should clarify that, if they used Firefox for reference renderings, then their rendering bugs will match Firefox's. Apparently WebKit already does this, but it wouldn't make much sense to use WebKit as the reference rendering for a test of WebKit! So they must be using some other browser; Firefox would make sense.

Second, it means we may finally get the holy grail of web testing: a headless DOM! We currently have a test apparatus that consists of a big, fast Mac machine with a 1920px display and 4 VMware machines, running Selenium on multiple browsers. Automating this is nightmarish, and completely unsuitable for agile development methodologies. We only have one of these, so access to it has to be gated, and the barrier to entry to test your code is huge. (You have to commit it, to start with.) The web badly needs a way to test applications without a browser showing up on your desktop. I just want to see green/red for the question "did my login page render the same way it did last time?"

The phrase "schematic" tantalizingly hints at a test suite with the ability to tell you about test failures in a descriptive way. I imagine seeing something like "div#nameEntry expected position:(258,317) got position(258,0)". But now we've veered away from speculation into wishful thinking.

This makes a big assumption that Google will release the code of the test apparatus. But I'm betting they will, because it makes extremely good business sense. If web devs start to rely on their test engine, their applications automatically support Chrome.

Paul Swartz: Van Gogh’s Self-Portrait

Tue, 02 Sep 2008 16:40:00 +0000

Van Gogh’s Self-Portrait

Paul Swartz: Or just pray it doesn’t happen to you (via indexed)

Tue, 02 Sep 2008 01:22:23 +0000

Or just pray it doesn’t happen to you (via indexed)

Paul Swartz: For the first time in almost 2 years, I have a new pair of...

Tue, 02 Sep 2008 00:02:00 +0000

For the first time in almost 2 years, I have a new pair of shoes: Merrell Cruise Control trail running shoes.

Paul Swartz: "If you want to go forward, put it on D — if you want to go backwards, put it in R."

Mon, 01 Sep 2008 16:30:42 +0000

“If you want to go forward, put it on D — if you want to go backwards, put it in R.”

- At Speedway, Clinton Says to Put it in D | The Trail | washingtonpost.com

Stephen Thorne: SSH Fail

Stephen Thorne (noreply@blogger.com) — Sun, 31 Aug 2008 23:10:26 +0000

I'm very very angry just at the moment, I'm angry about a total mismatch of expectations in the name of 'usability' that has invalidated what I thought was a totally reasonable security mechanism that I more or less took for granted.

I have an SSH key. ~/.ssh/id_dsa. I have a passphrase on that key, so that if someone compromises my machine, all they have is a key, and they have to brute force my (rather long) passphrase. I have ssh-agent to remember my passphrase in memory so that I don't ever let that passphrase hit disk, but have the passphrase cached.

Running ssh-add -D should wipe that passphrase from memory so I have to type it again. I was toying with doing this nightly so that it would wipe the passphrase every night so when I log in the next morning my passphrase needs to be re-entered.

This is where the nightmare begins. On the weekend my machine crashed so I had to start a fresh this morning. I sit down, log in, fire up a terminal, and ssh into another host.

Bam, I'm straight in. No passphrase, no prompt, nothing. Just straight in. This shouldn't be possible. Either the passphrase has been removed from my key so that it can be used without a passphrase, or something is saving it to disk without my knowledge.

Fedora 9 in its default configuration will save your passphrase to disk if you're logged in under gnome. I don't know how to turn it off. I feel angry, violated, annoyed and really really frustrated. It was a simple thing and it's been fucked. I can't turn it off, I can't stop gnome from remembering my passphrase, I feel like pulling the drive, hitting it with a hammer, and going back to OSX instead.

At least OSX can get simple things like ssh-agent right.

*sigh*

Cory Dodt: Gnome/Firefox tip: Pandora as a Separate Application

Cory (noreply@blogger.com) — Sun, 31 Aug 2008 19:26:05 +0000

The problem: Pandora is more like a desktop application than a website. It should stay open when you close your browser. If you do web development, or your browser tends to crash a lot, or you just like to close Firefox sometimes, you probably don't want to lose your tunes. Figure out a way to keep Pandora open when the rest of your web browsing session is gone.

This is a walkthrough for Gnome/Compiz users for isolating Pandora from the rest of Firefox cleanly. Much of this will be applicable to you if you don't use Compiz, and even if you don't use Gnome.

This assumes Firefox 3.0.1/Linux, I can't vouch for command line options on any other version.

When I refer to your Default Firefox Icon, I am talking about whatever means you usually use to launch Firefox, whether it's a toolbar icon, a menu icon, Alt+F2 "firefox", etc.

Create a Clean Pandora Profile

Start by shutting down all instances of firefox completely. Then bring up a Terminal window and type:

firefox -no-remote -ProfileManager

You'll get the small Profile Manager window. Create a new profile, named 'pandora'. You may, if you wish, clean it up. I did all of the following:

moved all the controls to the menu bar line at the very top (right-click a blank space and "customize...").
hid all the toolbars.
edit > preferences and changed the homepage to http://pandora.com/

Now quit from that firefox window. Once again, run:

firefox -no-remote -ProfileManager

Firefox makes your latest profile the default in all situations. To fix that (you don't want Pandora as your default browser): Select 'default' (or your original profile) from the window that just appeared, and let firefox open. Now your Default Firefox Icon won't open Pandora.

Make a Launcher

Right click on your Gnome menu panel, and "Add to Panel..." Select Custom Application Launcher. Enter the following:

Type: Application
Name: Pandora
Command: firefox -no-remote -P pandora
Comment: Opens in a separate Firefox process

(Only the Type and Command must match what I show above.)

Leave that window open for now. Re-launch Firefox using your Default Firefox Icon.

Let's get a pretty icon for the launcher you're about to make. In Firefox, visit:
http://pandora.com/favicon.ico and save the file as ~/.mozilla/firefox/*.pandora/pandora.ico

Now, go back to your New Launcher window (you didn't close it, right? :) and click on the springy-looking icon to change the icon. Choose the icon file you just saved as your icon. Now finish/OK until you are out of the New Launcher window and you have a new launcher.

Bonus Compiz Step: Place Windows Support

The Compiz "Place Windows" plugin lets you put windows on a particular workspace automatically. I want Pandora windows to automatically be placed on desktop 8, and other Firefox windows to automatically be placed on desktop 3. This was the trickiest part of the entire process.

This isn't an explanation of how to use Place Windows (find that elsewhere). This is an explanation of why using Place Windows for the tricked-out Pandora profile is difficult, and how you can fix it.

Problem: Compiz requires you to identify the windows you want to place by window Class, Name, Title, Type, or a few other things. For all firefox windows, Class, Name, Type and so on are the same; so if you want some firefox windows to go one place and some to go another, well, you can't do that with Place Windows unless you can differentiate them by title. But firefox always starts with the same title: "Mozilla Firefox". Compiz only looks at the initial title when it tries to place a window.

Solution: A plugin called MozFox. This is a version of FireSomething, that plugin that lets you randomly name your browser "Fireslug", etc. Bring up your Pandora profile using your new launcher icon, and visit the link above. You will have to create an account and log in there because this is currently an "experimental" plugin.

Install the plugin, and let the Pandora profile restart. When it comes back up, Tools > Addons > MozFox > Preferences. You will have to manually delete everything in the three lists. (You can select multiple to delete at once, then right-click and "delete". This is a pain in the butt.)

Add to each list: "Pandora", "Fire", "fox".

Now in the Place Windows plugin, set it up to match "title=Pandora.*". (You must have Regex Matching on.) Close out of that window, and restart Pandora. When it comes back up, Compiz will automatically position it where you told it to go!

Paul Swartz: Edge: ENGINEERS' DREAMS By George Dyson

Sun, 31 Aug 2008 01:28:17 +0000

Edge: ENGINEERS' DREAMS By George Dyson

Paul Swartz: Tor.com / Science fiction and fantasy / Stories / The Things that Make Me Weak and Strange Get Engineered Away by Cory Doctorow

Sun, 31 Aug 2008 01:28:17 +0000

Tor.com / Science fiction and fantasy / Stories / The Things that Make Me Weak and Strange Get Engineered Away by Cory Doctorow

Paul Swartz: Exclusive: Philip Pullman's essential reading list - Times Online

Sun, 31 Aug 2008 01:28:17 +0000

Exclusive: Philip Pullman's essential reading list - Times Online: I’m bringing copies of The Anatomy of Melancholy and The Brothers Grimm Fairy Tale on the road trip.

Paul Swartz: Bundle wrapping (via OneBag)

Sun, 31 Aug 2008 01:11:41 +0000

Bundle wrapping (via OneBag)

Paul Swartz: Sarah Palin is Laura Roslin (via scarndp)

Sun, 31 Aug 2008 00:54:05 +0000

Sarah Palin is Laura Roslin (via scarndp)

Paul Swartz: "Memory is like an orgasm. It’s a lot better if you don’t have to fake it."

Thu, 28 Aug 2008 22:56:34 +0000

“Memory is like an orgasm. It’s a lot better if you don’t have to fake it.”

- Seymore Cray (on virtual memory) (via 13 Geek-y Programmer Quotables)

Paul Swartz: paroneayea: I'll start on it [a new feature request - ed] after I finish the dishes, which I've been saying I'm going to do every night for going on a week now :D

Thu, 28 Aug 2008 00:05:57 +0000

paroneayea: I'll start on it [a new feature request - ed] after I finish the dishes, which I've been saying I'm going to do every night for going on a week now :D
paroneayea: protip, this is not a good way to please your life partner
z3p: paroneayea: that request was filed 2007-11-09; you're still beating me to it :)
paroneayea: z3p: you haven't done the dishes since 2007?? :)
paroneayea: j/k
z3p: paroneayea: yeah, I ask my gf to use bugzilla for requests like that; much easier for me to manage than just having her tell me
paroneayea: hehe
z3p: much easier to 'reassign' than in real life
paroneayea: wontfix

Duncan McGreggor: netaddr Python Library

Duncan McGreggor (noreply@blogger.com) — Wed, 27 Aug 2008 15:36:00 +0000

I recently got several feature requests for my NetCIDR Python library, and in the course of a conversation with one user in particular, I was made aware of the netaddr project. I took some time to explore the code details and was quite impressed: drkjam did a great job. The manner in which he implemented the many features (especially the IP math) was the kind of thing I wanted to do for NetCIDR ... at some point. After about an hour of digging around, testing out the API, and pondering, I decided to retire NetCIDR and encourage my users to migrate to netaddr.

There are a couple more esoteric features in NetCIDR that netaddr currently doesn't have, but we've started talking about adding support for those in netaddr, at which point there will be no need to use NetCIDR.

To facilitate this, I've added a wiki page on the netaddr Google Code project for helping users make the transition from NetCIDR to the netaddr API.

Itamar Shtull-Trauring: Past lessons

Wed, 27 Aug 2008 12:50:03 +0000

The more I read about China, the more I am reminded of Tocqueville's analysis of the French Revolution:

Revolutions are not always brought about by a gradual decline from bad to worse. Nations that have endured patiently and almost unconsciously the most overwhelming oppression, often burst into rebellion against the yoke the moment it begins to grow lighter. The regime which is destroyed by a revolution is almost always an improvement on its immediate predecessor, and experience teaches that the most critical moment for bad governments is the one which witnesses their first steps toward reform. A sovereign who seeks to relieve his subjects after a long period of oppression is lost, unless he be a man of great genius. Evils which are patiently endured when they seem inevitable, become intolerable when once the idea of escape from them is suggested. The very redress of grievances throws new light on those which are left untouched, and adds fresh poignancy to their smart : if the pain be less, the patient's sensibility is greater.'

Then again, China is still pretty damn authoritarian, so maybe it'll all turn out fine.

Christopher Armstrong: This is a Rant

Christopher Armstrong (noreply@blogger.com) — Wed, 27 Aug 2008 00:13:58 +0000

Edit: see end of post

Linux.com :: Protecting your MySQL database from SQL injection attacks with GreenSQL

This is an article (written by a very clueless person) about GreenSQL which is a tool (written by a very clueless person) that acts as a proxy between an application and a MySQL database which attempts to detect malicious, likely-injected SQL statements.

Do not interpolate strings into your SQL statements.

Then, there are all the hilariously dreadful comments.

"better yet, encode the bloody data before you shove it in there" --Anonymous

Do not interpolate strings into your SQL statements.

"Do you honestly think that anybody who doesn't know how to use simple, foolproof SQL-quoting functions is really going to be able to figure out how to correctly set up a package like this?" --Anonymous

Do not interpolate strings into your SQL statements.

"Why don't you try to actually learn to secure your code instead of being a lazy (or completely unskilled) administrator? Surely mysql_real_escape_string() isn't too hard to incorporate?" --Anonymous

Do not interpolate strings into your SQL statements.

Look, it's simple. Most database interfaces have a function called 'execute' or similar which takes two arguments: a string of SQL with markers like '?' in it, and then a tuple of arguments to be used as the values of those markers.

execute('SELECT * FROM users WHERE name = ? AND email = ?', ("radix", "radix@twistedmatrix.com"))

Do that. Don't do any of the following:

execute('SELECT * FROM users WHERE name = %s and email = %s' % ("radix", "radix@twistedmatrix.com"))

name = "radix";
email = "radix@twistedmatrix.com";
execute('SELECT * FROM users WHERE name = $name and email = $email');

String escaping is an absolutely retarded alternative to this. Why would you bother escaping or "encoding" your strings when you can simply use the database API as it was intended, without interpolating strings?

Edit: This concept of passing parameters has nothing at all to do with the "prepared statements" feature of popular databases. This is a much simpler feature. This is not a new feature. This feature is commonly called "bind parameters", and it has been around for decades.

Why do so few people know about this?

Duncan McGreggor: SOA in Practice: A Handbook for Early-Stage ULS Systems (Part 1)

Duncan McGreggor (noreply@blogger.com) — Tue, 26 Aug 2008 17:42:58 +0000

The ULS Series

Required Reading: Ultra Large Systems
Ultra Large Systems: An Example
SOA in Practice: A Handbook for Early-Stage ULS Systems (Part 1)

A Book Review

First off, this is an O'Reilly publication. What's more, if O'Reilly had something like a "criterion collection," this work would be in it. This title is what it says it is, "SOA in Pactice: The Art of Distributed System Design." Authored by Nicolai M. Josuttis, this is one of the best written technical overview works I have ever read, both for writing style and content. For anyone interested in ULS and/or SOA, I have one thing to say: buy this book immediately, with expedited shipping.

I'm not going to write a formal review with pros, cons, deep analysis about message, etc. However, what I will do is spend some time discussing the crossover from SOA to ULS, covering details with quotes from "SOA in Practice." I will not cover the book in detail and reveal all of its precious nuggets, but I will give a taste of what it has to offer and how it applies to ultra large-scale systems.

Divergence

Since most of what I want to discuss is about what we can gain by taking lessons learned from SOA and applying them to efforts in exploring or prototyping ULS systems, I want to initially outline the stark differences between those systems and SOA.

The most obvious difference is scale. To put things in perspective, imagine implementing a large SOA for a large organization. Imagine the requirements, the project planning, the logistics, the code, the bugs, the setbacks, the short-term failures, and finally, the successful delivery. Now multiply that: two related but semi-autonomous SOA projects. And again, with four. How about a third time for eight?

Any reader with experience in working with large projects is probably having heart palpitations right now (and for that, I apologize). You have first hand experience of the difficulties and the pain: with a linear increase in the size of a project, there is an exponential increase in the difficulty of managing that project (people, code, timelines, etc.), asymptotically approaching 100% unmanageability, regardless of the amount of resources you throw at the project.

The point just past the asymptote is where ULS systems and SOA meet. In other words, a ULS system as a whole -- by definition -- cannot be built. Such a system can accrete over time, but is simply too large to be designed, built and managed. Rather, it is emergent. Efforts being made in ULS systems research right now are focused on how we can best facilitate that emergence.

Convergence

One of the profound problem solving skills that maths like analytic geometry teach us is understanding potentially intractable problems by examining discrete and meaningful chunks. It's easy to chop something up; it's quite a different matter to chop such that the pieces are useful and provide further insight.

If working with ULS systems is like integrating over the volume of a complex solid in 11-space, then SOAs provide us with the tools of breaking part of that work up into a manageable chunk, one that we can wrap our heads around. Many of the same problems that technicians are anticipated to encounter when working with ULS systems exist at a smaller scale and are well understood within the context of SOA.

And this is where our friend Nicolai's book comes in.

ULS Systems Review

Before we continue, let's take a quick look back at some of the ULS basics laid out by the report of the Software Engineering Institute (SEI) of Carnegie Mellon:

ULS systems are systems of systems at internet scale.
ULS systems will be interdependent webs of software-intensive systems, people, policies, cultures, and economics.

In order to become a functional reality, these systems will require extensive research in the following areas:

Human Interaction
Computational Emergence
Design
Computational Engineering
Adaptive System Infrastructure
Adaptable and Predictable System Quality
Policy, Acquisition, and Management

This means exploring for use in ultra-large scale systems such things as potential mechanisms for user interfaces, genetic algorithms/programming, new patterns in systems design, behavioral simulations of systems components in varying compositions, decentralized infrastructure, ultra-high availability, and integration with countless third-party support systems. Just to name a very bare minimum.

Intersection

Of those research areas, lessons learned from SOA can be applied to ULS systems research most predominantly in the following areas:

Human Interaction
Design
Adaptive System Infrastructure
Adaptable and Predictable System Quality

In Part 2, it is with an eye towards these that I will comment on Nicolai Josuttis' excellent work.

Kevin Turner (LJ): zucchini stir-fry

Sun, 24 Aug 2008 05:55:58 +0000

Today I

Pulled weeds with AJ and Kenny at Meek ProTech
Met Principal Speed at Woodlawn
Ate salad and played frisbee with our new neighbors (who, it turns out, are old neighbors of Salut)
Made dinner with empty_fork in akatchoom's kitchen.

Duncan McGreggor: MySQL, Storm, and Relationships

Duncan McGreggor (noreply@blogger.com) — Sun, 24 Aug 2008 02:23:34 +0000

I rarely work seriously with databases, but I've been building an API for a contract with PBS.org, and though we have DBAs tasked for the project, everyone's pretty busy. So I dusted off my decade-old DB (formerly known as) skills, and did the work myself.

I've worked with the Storm ORM a fair amount since it was released, but only on small projects. Any time I've needed to use relationships with Storm, I've been using SQLite and so it was all faked. Due to the impact of the PBS gig (which is almost done now!), I really needed to sit down and map everything out. The first thing I needed to do was get a quick refresher on MySQL's dialect with regard to foreign keys. The next thing I needed to clarify was exactly how to ensure that what I've been doing with Storm relationships in SQLite was valid for MySQL and suitable for production use at PBS. It was :-)

Given how infrequently I use this stuff, I thought that my notes would be good to document, for future quick-reference. Given that there are likely users out there who would also benefit from this, a blog post seemed a nice way to do this :-)

The SQL below is modified from an example in the MySQL documentation, slightly tweaked to be a smidge more interesting. The two CREATE TABLE statements define the schemas for a one-to-many table relationship:


CREATE TABLE parent (
    id INT NOT NULL AUTO_INCREMENT,
    name VARCHAR(50),
    PRIMARY KEY (id)
    ) ENGINE=INNODB;

CREATE TABLE child (
    id INT NOT NULL AUTO_INCREMENT,
    parent_id INT,
    name VARCHAR(50),
    PRIMARY KEY (id),
    INDEX par_ind (parent_id),
    FOREIGN KEY (parent_id) REFERENCES parent(id)
    ) ENGINE=INNODB;

Next, to be able to play with this in Storm, we need to define some classes and set up some references:


class Parent(Storm):
    __storm_table__ = 'parent'
    id = Int(primary=True)
    name = Unicode()

class Child(Storm):
    __storm_table__ = 'child'
    id = Int(primary=True)
    name = Unicode()
    parent_id = Int()
    parent = Reference(parent_id, Parent.id)

Parent.children = ReferenceSet(Parent.id, Child.parent_id)

The parent attribute on the Child class is a Storm reference to whatever parent object is associated with the child object that gets created; the parent_id attribute is what is actually mapped to the MySQL field parent_id (which, in turn, MySQL references to the parent table). I hope I just didn't make that more of a confusing mess than it needed to be :-)

The children attribute that gets added to the Parent class is a reference to all Child instances that are associated with a particular Parent instance. I've got some usage below, if that's not clear.

Let's create a parent:


>>> from storm.locals import *
>>> p1 = Parent()
>>> p1.name = u'parent 1'
>>> store.add(p1)
>>> store.flush()
>>> print p1.id

Note that if you add an __init__ method to your Storm classes, you can save a step or two of typing in these usage examples (see the Storm tutorial for more information).

Next, we'll create and associate a child:


>>> child1 = Child()
>>> child1.name = u'child 1'
>>> store.add(child1)
>>> child1.parent = p1
>>> store.flush()
>>> print child1.id

There's more than one way to do this, though, given the way in which Storm has encoded relationships. Above, we created the child and then set the child's parent attribute. Below, we create the child and then use the chilren's add method to associate it with a parent:


>>> child2 = Child()
>>> child2.name = u'child 2'
>>> store.add(child2)
>>> p1.children.add(child2)
>>> store.flush()
>>> print child2.id

We're doing all that flushing so that the created objects refresh with their new ids.

Lastly, let's take a look at what's we've just added to the database:


>>> store.commit()
>>> p1.children.count()
2
>>> for ch in p1.children:
...     print "id: %s, name: %s" % (ch.id, ch.name)
...
id: 1, name: child 1
id: 2, name: child 2

And that should just about do it :-)

Duncan McGreggor: Summer Days

Duncan McGreggor (noreply@blogger.com) — Sun, 24 Aug 2008 01:42:17 +0000

This August was just incredible, like a Summer out of childhood. First, there was the week-long retreat up in the Colorado mountains. What can I say about that? It was beyond words... from the wildlife to bathing in an icy mountain cascade; from waking at 5:30am to running through a rainstorm; from the peaceful silence to the hilarity of jokes shared with friends.

Next up was a visit from a Tibetan friend of mine and his wife: they drove out to Nebraska to do some relaxing and horseback riding. When we were out riding, Anyen Rinpoche said that the grasslands and the sandhills reminded him of Amdo, Tibet... he felt like he was back home. He regaled us with stories of horse racing, riding competitions, and childhood capers... and then he and I raced around on horseback like a couple of teenage yak herders. We also managed to make it to a private swimming hole on the Niobrara: as soon as I saw it, I took off running, ripping off my shirt while I ran and laughed, and jumped into the river, cooling the very hot Nebraska sun; I felt like a kid again :-)

This week, my girlfriend's dad was in town. He, Marjorie, her daughter, her youngest son and I went kayaking for two hours down the Niobrara. It was my first time kayaking (though I grew up canoeing in Maine), and I was thrilled. That's the best upper-body workout I've had in years... and I want to do it again!

All in all, I couldn't have asked for a better break :-)

Paul Swartz: Official 2008 Obama/McCain Presidential Debate Schedule | You Decide 2008

Fri, 22 Aug 2008 20:11:56 +0000

Official 2008 Obama/McCain Presidential Debate Schedule | You Decide 2008: I’ll be road-tripping, but hopefully I can catch them on the radio or at a bar somewhere.

Paul Swartz: Web Development (via GraphJam) I’d also add...

Fri, 22 Aug 2008 17:09:44 +0000

Web Development (via GraphJam)

I’d also add ‘Debugging in Safari’ as a large slice of the pie.

Moshe Zadka: Steganographic Markov Chains

moshez — Fri, 22 Aug 2008 13:13:44 +0000

I am working now on a design for steganographic markov chains, intended for plausible and quiet encryption.

First of all, here is a rough design

Implement a JSON-backed database for markovian statistics about words (partially implemented, backed currently by Pickle)
Implement an encoder (currently I am planning nibble/word). That works already.
Implement a decoder (that works already)
Implement break-document-into-words for quality input into the db (implemented naive algorithm)
Add “smarts” about sentences/paragraphs to the encoder
Add compression/encryption layers, in order to supply better hiding (I’ll probably use 256-AES)
Design a key exchange scheme — if I know someone’s PGP public key, could I still steg. the encrypted key in the beginning without skewing statistics?

I plan to be able to use it for e-mail (markov chaining would hopefully, at worst, look like spam), blogs (wrap it with a nice interface for uploading to wordpress/lj, and think about a firefox extension for auto decryption (if you put a marker in the compressed plain text, you could opportunistically decrypt and check for the marker) and IM (wrap it with libpurple, and a public interface for key negotiation — ideally implement it as a plugin to pidgin which can be made to work without hassle).

Moshe Zadka: The Hacker Crackdown

moshez — Thu, 21 Aug 2008 16:54:45 +0000

I am reading “The Hacker Crackdown”, by Bruce Sterling. It’s published, and republished, everywhere — electronically. It is a book detailing the Phreaking/Hacking scene of the 1990s. It is well-written, and what’s more — with amazing technical depth, for all that it is written by a layman. I want to quote one part of it, near the end, to show what I mean –

Kapor is determined to tackle the technicalities of the Internet in the service of the public interest. “The problem with being a node on the Net today is that you’ve got to have a captive technical specialist. We have Chris Davis around, for the care and feeding of the balky beast! We couldn’t do it ourselves!”

He pauses. “So one direction in which technology has to evolve is much more standardized units, that a non- technical person can feel comfortable with. It’s the same shift as from minicomputers to PCs. I can see a future in which any person can have a Node on the Net. Any person can be a publisher. It’s better than the media we now have. It’s possible. We’re working actively.”

Kapor is in his element now, fluent, thoroughly in command in his material. “You go tell a hardware Internet hacker that everyone should have a node on the Net,” he says, “and the first thing they’re going to say is, ‘IP doesn’t scale!’” (”IP” is the interface protocol for the Internet. As it currently exists, the IP software is simply not capable of indefinite expansion; it will run out of usable addresses, it will saturate.) “The answer,” Kapor says, “is: evolve the protocol! Get the smart people together and figure out what to do. Do we add ID? Do we add new protocol? Don’t just say, we can’t do it.”

Oh, yes. I am fully aware of the irony of publishing it in my blog, managed entirely through the web

Moshe Zadka: Meme: Personality test

moshez — Mon, 18 Aug 2008 10:10:03 +0000

I rarely do memes. This one appealed to me. And I liked what I got…

Your result for The Perception Personality Image Test…

HFPS - The Humanitarian

You perceive the world with particular attention to humanity. You focus on what’s in front of you (the foreground) and how that fits into the larger picture. You are also particularly drawn towards the shapes around you. Because of the value you place on humanity, you tend to seek out other people and get energized by being around others. You like to deal directly with whatever comes your way without dealing with speculating possibilities or outcomes you can’t control. You are in tune with all that is around you and understand your life as part of a larger whole. You prefer a structured environment within which to live and you like things to be predictable.

The Perception Personality Types:

Take The Perception Personality Image Test at HelloQuizzy