Planet Linux Australia

Brendan Scott: Developers: Legal Tips for Young Players - Record Keeping

Mon, 06 Oct 2008 11:34:56 +0000

Brendan Scott, October 2008

Records Records Records

In the mid 90s I acted for a large government institution in a dispute over the implementation of a failed search engine development being carried out by one of the worlds’ largest developers/systems integrators. As is usual in litigation both sides conducted “discovery” - where you produce to the other side any documents of yours which are relevant to the dispute. Part of the discovery related to the backups of the actual code of the software being developed. In theory each side should have had a monthly back up for 12 months - that’s what they told us. When it came to recovering from the backups we discovered most could not be restored. Out of 24 copies which should have been available, only (I think) 2 could be recovered. Neither of two large organisations (each of which might fairly be regarded as a paragon of record keeping) were able to keep adequate records.

Small and medium enterprises typically have poor record keeping processes. Without records, it can be very difficult to prove any point you’re trying to establish - especially if the other side has kept their own (self serving) records. A good record keeping system will permit you to understand what was happening at a particular time and this can have a lot of important consequences.

Structure Your Records

A record keeping system not only records information, it also structures and stores it in an ordered fashion. This can sometimes mean having to replicate the record in a number of different places if it happens to be relevant to a couple of different things. If you don’t keep your records in a structured way you might, in theory, be able to reconstruct what was happening from the jumbled mass of disparate notes you have kept, but, in all likelihood, you won’t be able to do so as a matter of practice. It is unwise to rely on an after the fact search to find relevant documents. You might choose to structure things implicitly through the file system. Alternatively you can use meta-data in a document management system. However, my experience (now somewhat dated) with document management systems has been that structure is poorly represented (my impression was that they made it easy for me to find other people’s documents, but rarely my own).

Maintain Separation of Projects

A failure to properly structure your files (whether physical or electronic) can have legal implications. For example, if you fail to maintain a clear demarcation between the open and closed source work that you do, the code from one area may contaminate the other, potentially leading to untold heartache. Indeed, this applies pairwise to any two projects you are working on. Particularly important is that you maintain a strict demarcation between any of your personal coding infrastructure (eg common libraries that you may have developed and reuse for multiple clients) and the specific projects you are working on.

For example, if you were to lump your private libraries into a directory shared with a client project that might seem to a judge (who are not renowned for their understanding of coding practices) as if they formed a single work - which may be bad if you’ve assigned IP as part of your engagement.

Make Notes

In some cases records can serve as a substitute for an express contract. Equally, where a contract is ambiguous or apparently overreaching, consistent record keeping can, over time, establish some certainty within that ambiguity or can put bounds on the scope of an overbroad contract. Such records might include, for example, notes of conversations where decisions have been made about what is in or out of scope, or emails which have been sent to this effect. To the extent there is ambiguity don’t be shy in recording your understanding of the events. While courts may not necessarily take your view of the world (as instantiated in your records) as gospel, it’s better than having nothing. Send the notes to the other side prefaced by something along the lines of “This is what I understood we are doing going forward: ….” This gives them an opportunity to correct you if you’re wrong. If they don’t it will look odd for them to argue it was wrong at a later date.

Date it

To the extent practicable notes the date the note is taken should be included as part of the note. This makes it easier to date the note - which can be crucial. Often cases ride on when a person knew or said something. If you have an undated note of what was said, it may not end up helping your case if you are unable to place it in relation to some critical date. For example a note that the other party said “don’t worry, the contract has nothing in it that you should be concerned about” will have a completely different consequence if it can be proven to be said the day before you sign (perhaps it induced you to sign) as opposed to the day after (when it can have had no effect on your decision to sign). In addition, courts will generally pay more attention to a record which is contemporaneous with the details it records rather than one which attempts to reconstruct the situation at a later date.

Never use automatic date fields in documents if they will auto-update at some later time. Date fields may seem like a good idea at the time, but once they update, their value is lost. If you must use date fields, have some process by which they are converted to plain text on save (eg when the note is finished, or the relevant document has been sent).

Benjamin Carlyle: REST Model-View-Controller (cont.)

benjamincarlyle at soundadvice.id.au (Benjamin Carlyle) — Mon, 06 Oct 2008 11:18:00 +0000

My last article introduced the application of model-view-controller to REST client and server design. I thought I would expand on that material a little further, and provide some diagrams as food for thought.

The REST Triangle

The first thing I would like to do is re-introduce the REST triangle, roughly in line with the terminology I have been preferring of late.

When I first introduced the triangle we were all talking about nouns and verbs, and the obviousness of preferring to have more nouns than verbs. Honestly, I think it caused as much confusion as it did clarity. The obviousness is hard to draw from a set of natural languages that have a lot of both nouns and verbs, and the analogy doesn't really help people come to grips with the impact on their architecture.

Instead I have been tending to talk about Resources and Patterns, URLs and Protocols. Resources are network-exposed objects, and tend not to be that easy to grasp until you relate them directly to the concept of URLs. People get URLs. They understand what they look like and roughly what they are for. If you can stick to that end of the story you shouldn't get too much blow-back.

I prefer Patterns to "verbs". I think the verb or method concept is too low level, and doesn't really give an architectural feel for why limiting them is useful. Patterns on the other hand give in my view about the right feel for the real architectural motivation. People understand that you want to catalogue and reuse patterns. Every message exchange is part of some sort of pattern when you operate over an unreliable network. There are always message retries or redirection, or an imperative not to retry, etc. Communication at this level is hard for the uninitiated, and keeping to a fixed set of well-understood set of patterns only makes sense.

Part of the Uniform Interface constraint is that patterns are baked into protocols. You don't talk about a GET pattern, then make up a new method every time you apply it. You make the method name match the pattern. This bypasses much of the service discovery jiggerypokery that you would otherwise have to do. If you already know what pattern you want to apply, and the server supports that pattern, then you know which method to call. After that it is just a matter of making sure you have the right URL in hand to complete the request.

It almost goes without saying, but I'll say it anyway: Any pattern in REST architecture must of course conform to REST principles. That does not mean that the set of patterns is fixed by any means, but we have gotten away with surprisingly few patterns so far.

Squeezed away in the corner there you might call Content Types, but that doesn't really mean much to many people. I tend to call them document types these days. That has a reasonable mapping for most people to what we really mean when we talk about document types: An Excel document isn't much use to Word. A Word document isn't much used to Powerpoint. They are different document types.

The separation of Document Types and the patterns that transfer documents around the network is evolvability gold. You can introduce new kinds of information into the architecture as new document types without breaking your existing infrastructure for moving documents around. However, we also want to catalogue and reuse Document Types. We want to reuse our parsers and generators. We want to reuse our human understanding of what a particular document actually means. Your patterns will likely even let you negotiate between client and server to ensure that everyone can relate the conversation to a document type they were designed to understand.

Coding the Triangle

By no means is there only one way to implement a service or a client. One of the big selling points of REST and SOA generally is that they can operate in a heterogeneous environment where different technology and approaches are used on either side of the client/server divide. With that in mind, let's look at what I would consider a basic "body plan" for a REST client and server. I'll focus on an application of the GET pattern for purposes of example, but the same plan fits any pattern you would care to mention.

Model-view-controller as a pattern means many things to many people in many different contexts. Its exact meaning has been somewhat eroded over time and replaced with specific variants for special situations. The overall goal of the pattern, however, has not changed: Isolate appearance from implementation. Separate application logic from the constraints of how that logic is presented to users and to other applications.

Down the middle of my plan I have placed the corners of my REST Triangle. I have the set of Document Types in the architecture. I have the set of Patterns and Protocols. I have the set of Resources or URLs in the architecture. These are abstract entities included to give the diagram a bit of context. Each resource participates in a defined subset of the architecture's overall set of Patterns and protocols, and provides the public face to application logic behind the scenes between client and server.

Down the bottom right and left corners of the diagram I have the client and server model components. These components rightly do not depend on anything and do not play any public role in the message exchange. They are shielded from view by their respective Controllers.

The client controller includes code to pilot a pattern's state machine from its initial to its terminal state, despite possible adverse network conditions along the way. Lost response? Retry the request. Redirection? OK, follow if it looks reasonable. Fail the pattern if it looks dodgy. Positive response from the server? You beauty. Process the response.

The role of the server controller is quite different. It will include objects in its scaffolding to handle requests to any of its Resources. That might mean one object per resource. More likely it will mean an object tree with parameters along the path that collect a number of different resources together. These parameters will be passed into the appropriate function on a scaffolding object.

Let's say our request was a GET to http://example.com/light-bulb/1234. Chances are the request would arrive at a function internally identified as something like Resources.light_bulb.get(id). The controller component will then consult its model as part of fulfilling the request:

Resources.light_bulb.get(accept, id):
	bulb = Model.get_bulb(id);
	return Transform.to_acceptable(accept, bulb.status);

As well as consulting the model, this server consults its set of transforms. It wants to return information about the light bulb in a form that the client understands. Therefore, it uses the accept header from its request and passes it through to the Server Transforms component. Multiple controller functions may return the same Document Type and make use of the same transformations to prepare the returned document.

Once returned, the client will also apply its own transform to the document. Its objective is to map the document into data structures that be easily applied to its own model.

Conclusion

Client and Server Controllers deal with the set of Resources and the application of Patterns and Protocols. The Models of each will be queried and updated as part of the transaction, and the Server Model contributes to the set of Resources by containing objects that can be looked up through the Server Controller. Transformations at both end relate documents to model-ready data, and model data to documents.

Colin Charles: foss.my open for participation

Mon, 06 Oct 2008 10:55:25 +0000

What started as some chat about having a one-day event, after MyGOSSCon, on the 26^th of September 2008, about open source, has turned into a full-blown conference, to be pulled off in about a month, affectionately known as foss.my. Its being held at APIIT from November 8-9 2008, and is touted to be the most technical conference of its kind in South East Asia.

We want people to participate as speakers, delegates, sponsors, or volunteers. This is a grassroots event, and its purely non-commercial - no vendor talks, or marketing gimmicks are permitted. Largely the motto is very foss.in/linux.conf.au-ish - both community events I truly enjoy going to, and wouldn’t miss for the world.

The Call for Participation is open till midnight October 10 2008, so if I were you, I’d rush and submit some topics. If you’re nearby - Thailand, Singapore, Indonesia, the rest of South East Asia, it would be silly not to show up, with all the budget carriers there are today (maybe next time, we’ll get an official airline ;)).

Conferences must have themes, and foss.my is no exception. We believe the world is just awesome, and there’s so much FOSS goodness that can be shared with others. So if you’re involved in something free and open source related, its only sensible that you submit a talk topic. Share the knowledge, foster more open source development growth and contributions.

There are expected to be talks on topics such as: hacking on phpMyAdmin (a Google Summer of Code Project, run under the MySQL project), MyMeeting (a Malaysian government open source project), Asterisk, Django, CSS, source control, OSS Development on OS X, PHP, open source databases (MySQL, PostgreSQL, etc.), and the list goes on. There will be four great keynotes, and lightning talk sessions (so everyone can share).

Just show us your awesomeness. Read more from Aizat, Khairil, and Izhar.

Jeremy Visser: Broken PyGTK in Gentoo

Mon, 06 Oct 2008 09:41:01 +0000

If you have a broken PyGTK in Gentoo after upgrading to Python 2.5, like what happened to me, try this:

emerge -1 pygobject pycairo pygtk

…which fixed it for me.

Apparently, running python-updater is supposed to take care of all of that for you, but running it did nothing for me.

If you’re encountering similar Python-based breakages in Gentoo after a Python upgrade, check your /usr/lib/python2.4/site-packages directory. If you’ve removed Python 2.4 and installed Python 2.5, any files still lurking in your python2.4 directory are being very bad indeed.

Simply re-emerging the offending packages usually fixes it and places them in python2.5 instead. For example, if pyxslt broke after upgrading, you would find which package /usr/lib/python2.4/site-packages/libxslt.py belongs to by running:

$ equery belongs /usr/lib/python2.4/site-packages/libxslt.py

[ Searching for file(s) /usr/lib/python2.4/site-packages/libxslt.py in *... ]
dev-libs/libxslt-1.1.24 (/usr/lib/python2.4/site-packages/libxslt.py)

That would tell you the package (libxslt) to re-emerge.

Donna Benjamin: Linux users of Victoria - October 7

Mon, 06 Oct 2008 08:48:07 +0000

Linux users of Victoria October meeting is tomorrow night!

It snuck up on me what with Software Freedom day, International travel and the fog of jet lag!

2 excellent talks lined up

Stewart Smith talking about Drizzle - new database derived from the MySQL codebase.

Ben Balbo talking about streaming media in a sneak peak of his OSDC talk to be delivered at this year's OSDC conf in Sydney.

Our venue is the Evan Burge Lecture Theatre at Trinity College, Parkville.

Full details online: http://luv.asn.au/2008/10

Admission is Free, and all are welcome.

Various peoples tend meet to eat before or after the meeting

Before: 6pm - Classic Curry House, Elizabeth St, Melbourne

After: 9.15 pm-ish - Maria's Trattoria, Peel St, North Melbourne

Arjen Lentz: Sebastian visits Sydney! Quality Assurance in PHP Projects

Mon, 06 Oct 2008 07:53:53 +0000

Back on popular demand, Sebastian Bergmann will teach his 3-day workshop Quality Assurance in PHP Projects in Australia again! It's scheduled 8-10 December in Sydney.

Many applications using MySQL are written in PHP... this three-day workshop will introduce/update PHP Developers to writing unit tests for the backend and system tests for the frontend of a web application as well as managing the quality from development to deployment and maintainance using tools such as PHPUnit, Selenium RC, phpUnderControl, PHP_CodeSniffer, and PHP_Depend. Sebastian is the author of PHPUnit, and long-time contributor to PHP itself.

Pricing is AUD 1695 + GST. Since this is a workshop, the number of seats will be limited to 10-12. Speedy registrants will receive a free PHP-related book of choice (from a Sebastian shortlist, he knows which are good).

Around the course dates Sebastian will also be available for consulting and/or in-house training in the region, simply contact Open Query to discuss.

Davyd Madeley: a quick lunchtime update

Mon, 06 Oct 2008 05:16:22 +0000

My pansy-arse body is clearly not cut-out for extensive physical exertion.

For a weekend with nothing really planned, it turned out to be very eventful. On Saturday, Stephanie and I walked down to the Perth esplanade for the (not so well advertised) Perth Loves Earth fair. Which was advertising sustainable living stalls and vegetarian food. There weren't that many people there, I put that down to a combination of poor advertising and the sporadic rain-showers and strong winds off the river. There was however vegan cake (OMG, vegan cake) and vegan burgers to be had (from the Soul Food Cafe stall). Stephanie and I want to plan a trip up to the hills to visit the Soul Food Cafe at some point. Steph has a better writeup.

In the evening Stephanie went to Telethon (I didn't even know people could do that) with Erin (my brother's fianceé), where she met a person who honestly believed the Prime Minister's name was Kevin Judd. I decided that I would go to the UniSFA's Zone3 (laser tag) game. Zone3 is one of those games where you spend a lot of time crouching behind walls and barrels. You notice that people who play it a lot bring knee pads. After five hours of it, you start to feel more than a little sore.

Woke up on Sunday morning feeling a little stiff, but not terrible. Stephen was having a birthday picnic in King's Park, so we decided we'd pack everything we wanted into our backpacks and walk (was probably about 3-4kms with the bus we jumped on part of the way). I figured the walk would help work out the stiffness from the night before.

We sat there, ate food and got slowly sloshed on red wine in the sun (whole set). When people decided they'd had enough of the park we all went back and kept drinking in our courtyard. Thai was ordered. I smashed a glass.

This morning I woke up very sore in all the places I was only a little bit sore on Sunday, especially in my thighs (which makes things like stairs, sitting, standing and walking all fun). My knees and elbows have come up bruised too. Seriously, ow.

Arjen Lentz: Open Query goes Eventum

Sun, 05 Oct 2008 23:47:24 +0000

I didn't really evaluate other issue tracking tools this time. I know Eventum from my time at MySQL when it got acquired from Joao Prado Maia. It's currently maintained by Bryan Alsdorf (lordrashmi on Freenode IRC #eventum) who is very helpful. It works well, it has a pretty decent user interface, I knew it would do the job for OQ, and I got a local company (with Eventum experience) to do the necessary customisation and integration stuff.

So now any incoming general mail or form ends up there, which properly removes me (and my inbox) as a communication bottleneck for Open Query's effective growth. It also allows everybody (including myself) to work more efficiently, since tasks have a clear status and "owner" that is not dependent on email tagging, flagging or location. Threads and issue can't just get lost - not that they were, but it takes time, effort and brainpower to make sure and now all that energy is freed up at least for new issue ;-) and this works both when I'm online as well as over email as I reply to an issue through Eventum and it takes care of the status changes.

Chris Samuel: Back to QTopia 4.3.3-snapshot

Sun, 05 Oct 2008 20:07:09 +0000

OK, I spoke too soon, I’m back running the 4.3.3 snapshot due to people complaining about echo on the phone. Oh well!

Back to QTopia 4.3.3-snapshot

Andrew Cowie: Wine your own business

andrew@operationaldynamics.com (Andrew Cowie) — Sun, 05 Oct 2008 13:05:00 +0000

The things that drive you crazy…

There’s one (and only one) legacy Windows application that we have. We used to run it in VMware, which worked, but that was a pain, looked terrible, and of course was required us to have our copy of Windows 2000 installed there. Decrepit. But far worse, meant we had to use Windows. Yuk. And worst of all to have to keep switching in and out of it. Bah.

Life would be far better if we could run it on Linux under Wine — then it’d just be another app on running on the GNOME desktop and under the control of the window manager. We’d tried seeing if we could get this application to run in Wine a few times over the years; no joy. But somewhere along the line it started noticing list posts from other people reporting positive experiences. So {shrug} I gave it another try this month, and to my intense pleasure it installed cleanly and is running fine. Terrific! The people who hack on Wine are amazing.

So I started working away, only to suddenly realize that the dates were in American mm/dd/yy format. Gah. More searching, but none of the workarounds suggested (mostly to do with manually changing things with Wine’s regedit) seemed to work. Quote a pain.

I had come across a number of references saying that Wine and in turn the program in question would pick up the locale settings provided by your environment. I’d checked that, and tried various combinations. Nothing seemed to work.

I mostly run in en_CA, ie:

$ echo $LANG
en_CA.UTF-8
$

the impact of which you can see by running the locale command:

$ locale
LANG=en_CA.UTF-8
LC_CTYPE="en_CA.UTF-8"
LC_NUMERIC="en_CA.UTF-8"
LC_TIME=en_DK.UTF-8
LC_COLLATE="en_CA.UTF-8"
LC_MONETARY="en_CA.UTF-8"
LC_MESSAGES="en_CA.UTF-8"
LC_PAPER=en_AU.UTF-8
LC_NAME="en_CA.UTF-8"
LC_ADDRESS="en_CA.UTF-8"
LC_TELEPHONE="en_CA.UTF-8"
LC_MEASUREMENT="en_CA.UTF-8"
LC_IDENTIFICATION="en_CA.UTF-8"
LC_ALL=
$

all as one would expect And yet it had no impact on running the program running under Wine [Setting LC_TIME to en_DK is an old trick for people in en locales to get proper 24-hour time formatting; yes, I tried unsetting LC_TIME; I tried dropping the UTF-8 settings; I tried LANG=en_AU. Nothing made any difference].

After a lot more frustration searching around, I finally came across a support article on CodeWeaver’s website that mentioned setting the locale environment variables. Yes yes, I thought, but then I looked more closely. They were setting LC_ALL. Wait a minute. LC_ALL is a special variable that “if set to a non-empty string value, override the values of all the other internationalization variables.” You’re not supposed to set that… certainly one doesn’t set that at login — that’s what LANG is for. I didn’t really expect anything to come of it, but I tried it anyway:

$ env LC_ALL=en_CA.UTF-8 WINEPREFIX="/home/andrew/.wine" wine "C:\Premier12\Myobp.exe"

and ta-da everything worked: I had dd/mm/yy date formatting just like we wanted. Yeay!

I immediately closed everything down and went out for a beer. Sometimes we forget to celebrate those brief moments when things actually work.

But talk about bitterness. I’m not sure if this was a Wine behaviour, a Windows behaviour, or just nonsense code in MYOB. I suspect the latter. But either way, I saw so many posts asking “how can I get the date format to behave under Wine” that I thought I should write about the workaround that did it for us.

AfC

Andrew Cowie: Congratulations Manly!

andrew@operationaldynamics.com (Andrew Cowie) — Sun, 05 Oct 2008 12:58:00 +0000

There really is nothing better than being in your home town at a pub surrounded by several hundred screaming fans when the home side is playing in the Grand Final of its league and then watching them trounce the opposition 40 to 0.

Photo by Brendan Esposito, as presented on the League HQ website

Congratulations to the Manly Sea Eagles for winning the 2008 Rugby League premiership!

AfC

Mark Greenaway

Sun, 05 Oct 2008 12:04:55 +0000

Apparently Linux audio is still an absolute nightmare. Maybe I should just write audio software for the Mac.

Anand Kumria: How to solve a credit crisis ...

wildfire@progsoc.org (Anand Kumria) — Sun, 05 Oct 2008 10:31:00 +0000

Apologies to anyone bored of this topic but it is something I have been thinking about recently.

You have people commenting on why Banks are special and need government funded bailouts. Others arguing that the Free market is not dead.

We have even have a few solutions from some people within the Free Software community.

I think the first thing is to identify who should lose, and who should win.

Investors should lose.
Customers should win.

Much of the annoyance about the rescue from ordinary people is that it seems that investors are not losing (enough). They took some risks, those gambles failed, and now everyone else is paying the price.

If IBM were to go bankrupt, would the government step in? Unlikely. Investors would lose (money), staff -- another word for investors -- would lose (jobs), but customers would win (their computers would keep working). Some customers would win more than others (especially those who had the equipment on lease); if no one is collecting, why pay?.

So let's apply the same set of outcomes to banks.

Borrowers would win, since they are living in a mortgaged house.
Investors lose, since no one will buy their shares (no matter how cheap they are)
Tax payers win, since they are not rescuing a particular industry sector

But hang on, you rightly ask, - a bank has the title deed (a mortgage is a promise to pay amount X over Y year in return for the deed) - why would customers (depositors), realising a bank is no longer viable, since remove their funds

I think that those two issues can be addressed fairly simply. The government would guarantee all depositors money. Customers never lose. They never have to worry about their funds.

For banks, who enter into administration, the standard laws about possession should apply. If a borrower is utilising (living in) the asset to which the bank has title for over Z years (where Z equals 5 or 7), then the possesor now owns it outright. That is plenty of time for a bank to either be bought, have the underlying asset value recalibrated, or to completely go bust.

All without a detrimental effect.

Michael Still: Blathering for Sunday, 05 October 2008

Sun, 05 Oct 2008 09:30:00 +0000

19:30: Mikal shared: The Daily Dish | By Andrew Sullivan

Wonder how bad sub-prime is in America? 700 homes in SoCal are foreclosed per day. This video gives an interesting look at the "trash out" process, which is what happens when you abandon stuff instead of paying your loan.

Tags for this post: blather() Comment RSS with no blather

Matt Palmer: The Healing Power of Hate

Sun, 05 Oct 2008 09:26:32 +0000

They say that there's very little difference between love and hate. A good example of this is the Linux Hater's Blog. It's written by someone who obviously has a lot of experience with Linux and Open Source, and I get the feeling that the hate comes of frustration and pain. Certainly, I've seen plenty of similarly themed articles coming out of Planet Debian at one time or another.

One of the best articles I've found yet is One bug report to rule them all, a humourous but altogether painfully accurate rendition of far too many bug reports I've seen and dealt with of the years.

Simon Horms: Struan and Genny's Wedding

Sun, 05 Oct 2008 02:20:59 +0000

Yesterday Chiz and I had the pleasure to gather together with Struan and Genny's friends and family for their wedding in the Royal Botanical Gardens in Sydney. It was a wet morning but the weather cleared just in time for the event. I took a few snaps of the occasion. more...

I also have some snaps of the Skydiving that took place in Picton two weeks ago as part of Struan's bucks party. more...

Michael Still: paramiko exec_command timeout

Sun, 05 Oct 2008 02:20:00 +0000

I have a paramiko program which sshs to a large number of machines, and sometimes it hits a machine where Channel.exec_command() doesn't return. I know this is a problem with the remote machine, because the same thing happens when I try to ssh to the machine from the command line. However, I don't have any way of determining which machines are broken beforehand.

Paramiko doesn't support a timeout for exec_command(), so I am looking for a generic way of running a function call with a timeout. I can see sample code which does this using threads, but that's pretty ugly. I can't use SIGALARM because I am not running on the main thread.

Can anyone think of a better way of doing this?

Tags for this post: python() paramiko() Comment

Mark Greenaway

Sat, 04 Oct 2008 22:11:02 +0000

Rehearsing for hours is really tiring. I'm exhausted today.

Colin Charles: how much for georgia?

Sat, 04 Oct 2008 19:10:12 +0000

spotted in riga, latvia, at the university, during software freedom day 2008.

Russell Coker: Getting Started with Amazon EC2

Sat, 04 Oct 2008 12:46:21 +0000

The first thing you need to do to get started using the Amazon Elastic Compute Cloud (EC2) [1] is to install the tools to manage the service. The service is run in a client-server manner. You install the client software on your PC to manage the EC2 services that you use.

There are the AMI tools to manage the machine images [2] and the API tools to launch and manage instances [3].

The AMI tools come as both a ZIP file and an RPM package and contain Ruby code, while the API tools are written in Java and only come as a ZIP file.

There are no clear license documents that I have seen for any of the software in question, I recall seeing one mention on one of the many confusing web pages of the code being “proprietary” but nothing else. While it seems most likely (but far from certain) that Amazon owns the copyright to the code in question, there is no information on how the software may be used - apart from an implied license that if you are a paying EC2 customer then you can use the tools (as there is no other way to use EC2). If anyone can find a proper license agreement for this software then please let me know.

To get software working in the most desirable manner it needs to be packaged for the distribution on which it is going to be used, as I prefer to use Debian that means packaging it for Debian. Also when packaging the software you can fix some of the silly things that get included in software that is designed for non-packaged release (such as demanding that environment variables be set to specify where the software is installed). So I have built packages for Debian/Lenny for the benefit of myself and some friends and colleagues who use Debian and EC2.

As I can’t be sure of what Amazon would permit me to do with their code I have to assume that they don’t want me to publish Debian packages for the benefit of all Debian and Ubuntu users who are (or might become) EC2 customers. So instead I have published the .diff.gz files from my Debian/Lenny packages [4] to allow other people to build identical packages after downloading the source from Amazon. At the moment the packages are a little rough, and as I haven’t actually got an EC2 service running with them yet they may have some really bad bugs. But getting the software to basically work took more time than expected. So even if there happen to be some bugs that make it unusable in it’s current state (the code for determining where it looks for PEM files at best needs a feature enhancement and at worst may be broken at the moment) then it would still save people some time to use my packages and fix whatever needs fixing.

[1] http://aws.amazon.com/ec2/

[2] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=368

[3] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351

[4] http://www.coker.com.au/ec2/

Paul TBBle Hampson: Grinding code in Warhammer Online

Sat, 04 Oct 2008 10:59:07 +0000

In which our hero shows that he is in fact willing to pay to program

Mark Greenaway: The joys of electric music making

Sat, 04 Oct 2008 10:24:48 +0000

Occasionally the nerdy skills I've acquired over the years pay off. My tuner pedal stopped working during rehearsal today, so when I got home switched batteries with another pedal that I don't use much. But it still didn't work! "Is Behringer living up to its reputation of producing crappy gear?", I wondered.

Of course, being a nerd, I own a digital multimeter. So I measured the voltage on the battery I'd placed in the tuner - it was down to 5.72 volts, whereas the pedal was expecting 9V. I took the battery out of my distortion pedal and measured that, 7.2V or so. Hopefully, enough to drive the tuner. I quickly switched them over, plugged a cable into the tuner's input jack and sure enough, it lit up like a Christmas ornament.

9 volt batteries are like $4 each for Duracells, but I guess that's going to add up quickly as I have two pedals already and will probably end up with more at some point. Maybe it's time to get a power supply. Although I am pretty sick of buying things right now. Oh, and the tone police say that using a power supply changes the sound. But who knows, maybe it'll be better ...

Chris Samuel: 50%+ of Standards Norway Tech Ctte Resign Over OOXML Approval

Sat, 04 Oct 2008 08:54:50 +0000

Thirteen of the twenty three members of Standards Norway have resigned over its decision to recommend OOXML to ISO when 19 voted no and 2 voted yes for it (one of whom was Microsoft). The Inquirer has a rough Google translation of the letter, which says things like:

Standard Norway chose to defy their own technical committee and vote yes to a specification that is immature, useless, and unworthy of being called an ISO standard.

and the damning:

The administration of Standard Norway trust 37 identical letters from Microsoft partners more than their own technical committee.

Ars Technica describes that last issue as:

Standards Norway has defended its conduct and asserts that its vote in favor of OOXML approval was based on the outcome of a public inquiry in which a majority of the responses it received encouraged support of OOXML. The standards body has also admitted, however, that a significant number of those responses were identical submissions authored by Microsoft.

All the ex-members say they will continue to work towards meaningful standards outside of Standards Norway.

50%+ of Standards Norway Tech Ctte Resign Over OOXML Approval

Michael Still: Blathering for Saturday, 04 October 2008

Sat, 04 Oct 2008 02:30:00 +0000

12:30: Mikal shared: Tracing the origin of HIV-1

I didn't realize HIV is so old. Apparently they've found samples going back to 1960, whereas I have always thought of HIV as a 1980s problem.

Tags for this post: blather() Comment RSS with no blather

Chris Smart: Taking a SIP of VoIP

Sat, 04 Oct 2008 00:42:28 +0000

It’s time for me to jump on the VoIP bandwagon..

This was the post I made to the CLUG list the other day. Why? Mendy is moving back to Australia permanently in December (yay!) and in the mean time I want to be able to call her at reasonable rates. Currently this means calling her Macau mobile at a hefty price through Telstra. Once she is back I also want for her to be able to call her parents in Hong Kong for cheap.

I knew about VoIP and had played a little with Ekiga (the open source VoIP client for GNOME), but that was about it. Looking into it seemed pretty complex, with a myriad of providers, software and hardware. I wasn’t quite sure which provider to go with, what hardware I needed and what was the best way to set it all up. What traps were there? As usual, my fellow CLUG’ers came through with great advice, so thanks!

It appeared that the general consensus were things like:

Make sure your router supports QoS (Quality of Service), so that the quality of your call doesn’t drop when someone is downloading

Get a provider in Australia (preferably your ISP) to reduce latency and help with billing

Get an ATA (Analogue Terminal Adapter), so I could use my current analogue phone

This is how I got it working..

Step 1: High quality VoIP provider

After researching it some more on places like Whirlpool, I discovered that my ISP, Internode (of whom I am a big fan), has one of the best VoIP networks in Australia which they call Nodephone. Their rates appear to be a little more than some other providers, but in just about every post I read on this issue people said it is worth it for the quality of service. Sounded good to me, so I signed up. 3 hours later I got an SMS saying that my account was ready. Awesome.

Step 2: Router with QoS support

My home Internet is ADSL 1.5MBit, connected through a dodgy old Linksys router that I found under a pile of computer junk. It does the job, but it was hardly up to the task of providing QoS support for my network. That was until I realised I also had a Linksys WRT54G and that I could flash the firmware with dd-wrt which supports QoS and lots of other cool stuff.

I set my WAG54G to bridged mode and connected it to the WRT54G, now running the open firmware. Once I had configured my Internet connection I then started to play with VoIP specific settings. I enabled the following:

Port triggering - I enabled this for Ekiga. Outgoing port 3478 opens 5000-5100

QoS - I turned this on and set both the up and down links to 85% of my connection speed (218 and 1306 respectively). I also added “SIP” to the service priority.

Well, that wasn’t so hard!

Step 3: Software to connect to provider

As I don’t yet have an ATA to connect my phone, I wanted to test the service out with a software solution. Naturally I chose Ekiga for the task. I recently built Ekiga version 3.0 for my Debian box, so I thought this would be a good test

Configuring Ekiga was pretty simple. I just entered the provider sip.internode.on.net, my username (which is my VoIP number), and the password they gave me.

This registered successfully, so I performed an echo test which worked really well. On the other end was the SBS guy who does those voice overs, you know..

SBS advises that the following program is recommended for mature audiences. It contains adult themes.

Sweet.

Step 4: Make a call

Now that I had my provider, hardware and software all configured and working, it was time to test it out! I added Mendy’s Macau mobile into the Ekiga contact list and told it to call with “Nodephone”.

Ring.. ring..

“Wai?”

“Hey, it’s me.”

“Hey!”

“I’m calling you from my computer. How does it sound?”

“Really good! Very clear..”

In conclusion

So there you have it. I was impressed by the quality and ease of configuration. I think I’ll stick with Internode, as I much prefer a dedicated VoIP service direct from my ISP rather than a cheaper one somewhere else that’s lower quality.

Now it’s time to go and buy an ATA device and get my home phone working. Thanks to everyone for their advice, it has been really helpful and I appreciate it!

Mary Gardiner: Writing helpful reviews

Fri, 03 Oct 2008 21:51:53 +0000

I outlined the style of good academic reviews to Jonathan in light of our impending OSDC review responsibilities, and it's worth noting here too.

For information's sake, my authority, such as it is, on reviewing comes from being the editorial assistant of Computational Linguistics, which is a journal with a hardworking editor and conscientious reviewers. Not all academic reviews are of the quality I discuss below. They should be.

Begin with stating the title of the paper you are reviewing. Then spend one to three paragraphs summarising its content, particularly what you perceive as its major findings and conclusions.

This has a couple of purposes. The first is that if the reviews have got mixed up in the system the author finds out as soon as possible and doesn't have to slog through a review that (perhaps) is a partial match for their paper and (especially in academic circles) a privacy problem to boot. The second is so that they know in what light to read the rest of the review. If they see that you have understood its fundamentals they will be inclined to take the entire review seriously. If they see you have misunderstood it, they can do one of two things. One is to realise that their paper is confusing, and to make its focus clearer. The other is to discount your review. The decision here may be affected by the following section.

The main body of the review is a discussion of how to improve the paper. Both the tone and discussion will vary considerably depending on certain factors:

is the paper already accepted?
is this the only reviewing round or will you or another reviewer be checking the changes?

For OSDC, both factors hold. For almost all conferences, there is only (at most) one reviewing round for full papers. This makes reviews more limited in scope than journal reviews, where substantial changes are often recommended even (or perhaps especially) to articles the reviewer fundamentally likes. Journal reviewers can have a role which is not far from being anonymous co-authors. (If a colleague did as much re-reading and suggestions of additional work and additional reading as Computational Linguistics reviewers do, many people would consider adding them to the authors list.)

In the event that the article has been accepted, or that this is the single reviewing round, you should limit the scope of your suggestions to much more cosmetic things. Someone who has had an article accepted is just going to be annoyed that you want it to have a whole new body of work incorporated, and they will ignore you. (And if it's rejected after a single reviewing round, they are probably ill-placed to revise much!) In the OSDC scenario, reviewers are going to be mostly limited to suggestions as to how to structure the argument and the paper better, and not really able to productively suggest changes to the argument or the work described in the paper.

As you write your review and this section in particular, keep in mind the key factor of providing useful critiques: how could this work be better on its own terms? That is, don't provide a review that is, fundamentally, about how the paper would have been better if you'd written it... about your pet topic. This is a subtle, tempting and common mistake, and if you have never caught yourself in it, you are likely to be the worst affected. Remember: What is the paper trying to do? How can it do it better? Avoid the temptation to suggest that it would be a better paper if it was doing something different from its current aim. (There is a little more leeway for this in journal reviews, but even in that case, generally what happens if a reviewer thinks this is that they review the article on its current form and recommend a fate suited to its current aims, and additionally comment that they would be interested in seeing further work in the additional direction should the authors choose.)

As a recipient of reviews, I do have a couple of things to add. One is to respect page limits. If you are reviewing for a work with a page limit, especially a conference, and you do really want to see a longer discussion of foo, please suggest which bar could be shortened or cut. Otherwise it is close to impossible for an author to consider your suggestion. Also, if you are making suggestions for future work that you think the authors should consider but which you do not actually want to see in the article, make this clear in the text of your review. I would probably recommend a whole separate section for this if you're going to do it.

A review may conclude with a list of typos, spelling mistakes, suggested rephrasings, etc. Mistakes that affect the reading of the paper (eg mislabeled figures and sections) go right at the start of this list. A sufficiently ill-proofread paper may go back with a suggestion that the authors find the mistakes themselves.

Jeremy Malcolm: Our cross-country trip wraps up unexpectedly...

Fri, 03 Oct 2008 20:10:00 +0000

The second leg of our cross-country road trip began in inner-city Melbourne, where we took in some new sights since our last visit (such as new Docklands developments and the DFO outlet mall), and had a meet-up of Australian Hufflepuffs from Hogwarts Elite (I'm not a member; Dom is)...

Matt Bottrell: A visit to Ballarat this week

mbottrell@gmail.com (Matt Bottrell) — Fri, 03 Oct 2008 18:35:00 +0000

This week saw me head to the Victorian regional city of Ballarat for a few days to undertake some RHEL5 x86_64 builds for work.

Whilst it's not as exotic a destination as an unnamed colleague visited nor did it see me returning without my pants or shoes ( sorry Jon, I couldn't resist! ), it was a productive visit.

Traditionally we undertake builds remotely, however these servers are likely not to appear on our network or even administered by ourselves, so it made sense to go onsite to undertake the work.

I stayed in the Doherty Ballarat Hotel and Convention Centre which literally is on the opposite side of the road to Sovereign Hill (and no I didn't have time to go there!)

View Larger Map

Whilst I spent most of my time in the office battling with hardware that had been incorrectly ordered (ie: missing hardware and/or non-compatible hardware (ie: 3.5" SATA drives ordered instead of 2.5" SAS drives for the internal enclosure)), I did venture back to the hotel in the evenings for some R&R.

The hotel room was quite pleasant, and what something a corporate traveller expects. There was no surprises out of the ordinary (which you really don't need when travelling), though I did have two strange nights of sleep due to external factors:

1^st night: Apparantly it's Koala mating season. Staying in a hotel room surrounded by forest is lovely to look at during the day, but it's also the playground for randy koalas of a night. So thumbs up to the 'big boy' who kept me up for several hours in the early hours whilst he ravaged the local female population. Koalas aren't dying out in this region whilst this bloke is still kicking! Seriously, I thought those types of noises I may have heard from other hotel rooms, not above my hotel room.

2^nd night: Local Football season must have finished. Seems some of the local players didn't head over to Bali to celebrate but decided our hotel was a great place to party.

In depth conversations at yelling level could be heard at around 3am. Answers to lifes biggest questions such as "Can you scull beer or bourbon faster?" were asked repeatedly. Finally after about an hour when it took that long for the gorillas to realise that they couldn't order a pizza to their room (and yes their was going to be surcharge for anything delivered to their room), the group of 6 decided that bundling into a car and heading down to the local McDonalds whilst tanked was the sensible thing to do.

I passed out after that, though it appears they were successful (if not foolish) as their car was there in the morning without any visible signs of damage. It seems the drink and driving campaigns don't apply to footballers.

I made sure I made enough noise as I passed their room at 7am heading off for breakfast, however in their drunken state the night previously, I doubt I would of woken them. It did bring me mild relief to my simmering frustration/anger from the previous nights interruption however.

The crazy and overpriced Internet costs in the room ($0.20/1MB with both uploads/downloads charged or $0.15/min) ensured I had a relatively Internet free week night. I really didn't miss being connected, though it did remind me how often I relied on just 'quickly looking up something'. Items like news, weather and even a TV guide via the Internet seems to have become the 'standard' method of information retrieval for me. It became more of an inconvenience than anything. I realised then that there was indeed a TV guide in the room. I gathered a quick round-up of news/weather using local radio. So this is what life was like prior to the Internet? (It seems so long ago now!)

Traditionally I use Google Reader to keep abreast of my feeds (as it allows me to bookmark, tag and share feeds easily), but being disconnected from the Internet was going to make that impossible.

I did however utilise a RSS reader, and was pleasantly surprised how easily it actually integrated into my Gnome desktop (particularly as it's a KDE app). The tray-icon and minimize/restore from tray all worked flawlessly (mostly due to the fact it's probably coded using FreeDesktop specs). I went with Akregator over Liferea after hearing reports that recent versions were prone to 100% CPU usage.

One thing I did notice, is that current RSS readers do allow for 'offline' mode - though it would be nice if it gave you the option of a 'full download' of certain feeds. Offline mode that only sucks down text is quite frustrating at times, as the associated image (being discussed in the article) isn't available. It feels like over-hearing a discussion you weren't meant to hear!

Whilst it's probably prudent allowing it to be set as a 'per feed' setting as I suspect many people may not want to download a full page for every article on every feed. (Particularly those feeds you only 'scan', and read a small amount of, or those that are overly heavy with graphics/flash, and embedded video.)

Since returning I did notice that Gnome also has the Straw Reader. Hopefully that comes along nicely as it appears to be in early development.

Ideally I would love to see an RSS reader that integrated WebKit with a good OPML import/export feature, as well as good support for RSS (0.91, 1.0 and 2.0), RDF as well as ATOM formats. Having a feature that allowed you to selectively do a full download of certain feeds for offline mode would be fantastic. I'll whack that down on my ever growing todo list (I think I'll need to live to 800 to ever complete it!)

One nice feature I discovered is the room had direct dial-in, and due to poor mobile reception in the room it was much more cost effective to have Pauline call me direct over VOIP. It also meant we could speak for much longer.

The staff I met at the Ballarat office were extremely friendly and actually quite enjoyable to be around. ( Sorry MrPointy, they were more cheery then you! ) They appreciated the effort I put in whilst down there (it's always nice to get some recognition) and appreciated the attention to detail. It was quite good timing as I will relocate offices to this one once I move into our new home (probably 4-6 weeks away). Heading into the Ballarat office is much closer than the Melbourne offices from Bannockburn. I've already made some good inroads into getting to know people there. I'll be the only member from my team at this office location, so it makes sense to build some bridges.

I did add two things to my 'travel-list' when travelling for work:

A cheap mini 10/100 switch.

This is invaluable when you're stuck in a room with one Ethernet port attempting multiple server builds, and also wishing to have your laptop online at the same time. I headed out the first day a picked up a cheapie TP-Link 5-port 10/100 Switch. Cost me $20 from a local computer store, but saved hours of frustration.

A decent power board with surge protection

In hindsight I should have also taken/purchased a decent power board. The room only had 4 power outlets which meant I could only have one server up at a time. It would have been nice that I could have multiple machines up at the same time, as these were identical builds (besides their hostnames), I could have used something like clusterssh to undertake the same tasks across all the machines at the same time -- ensuring I could have finished the job in a fraction of the time.

We live and learn.

Simon Rumble: Note about Same origin policy

simon@rumble.net (Simon Rumble) — Fri, 03 Oct 2008 16:59:59 +0000

JavaScript's same origin policy security model means that a script can't (directly... there are sneaky ways aroun dit) request data from another site. It's quite strict. foo.site.com is different to site.com.

One thing I'd always assumed, but now I see how it wasn't a sensible assumption, was that the piece of code calling the data had to be served by the same domain name it was going to be calling. I've just done a little test of this, and discovered that's not true.

So if you're dead keen to see it, have a look at this test which loads the script from stout.rumble.net and actually loads Prototype from somewhere completely different. It has no trouble pulling data from www.rumble.net, but won't let me drag data from stout.rumble.net.

Contact me

LCA2009 News: First miniconf Call for papers open

Fri, 03 Oct 2008 12:27:14 +0000

First miniconf calls go out: Miniconfs call for papers!

This week, first calls have come from the successful miniconfs to submit papers for their programs. These specialist conferences run in the days before linux.conf.au and bring in high quality speakers to cover topics in their field of endeavor.

"Miniconfs are always a hit with conference goers, and many delegates who just come to the main conference lament missing the minconf program" said Ben Powell, organiser for LCA09. "These miniconfs cover specialist topics that are of interest to the FOSS community - there's really something for everyone and although some of the traditional miniconfs missed out this year, the inclusion of new topics give conference goers the opportunity to see a wide range of exciting topics" said Mr. Powell.

To fill their programmes the miniconf organisers have started to announce a call for participation. Each miniconf is looking for a select range of talks from industry leaders and professionals alike. To read each particular call for participation, please read on below:

With more miniconfs announcing their call for participation soon, Linux.conf.au can expect a great line up of talks covering a vast amount of topics.

Benjamin Carlyle: Model-View-Controller with REST

benjamincarlyle at soundadvice.id.au (Benjamin Carlyle) — Fri, 03 Oct 2008 12:17:00 +0000

One of the big questions I think the REST software community are facing at the moment relates to what REST support we should expect from vendors, and what good REST support looks like. I have been taking a lead from web frameworks such as Ruby on Rails, and Gears.

Model-View-Controller means many things to many people. Most of those people will agree that it is a good idea, even if they aren't quite sure what it means for them in their environment.

A classical GUI application would employ application logic in a model, and its view would be a set of widgets on the screen. A controller object either intercedes when events come in from one or the other, or links the two together at display call-up time.

The prevailing wisdom in the world of server-side Web frameworks is more something along the lines of:

The model is a set of objects that live within a database
The view is a set of templates for documents that will be returned as part of requests
The controller is a mapping out of the URL space, which calls out to the model and view components as required

This works well in the common environs of web servers today. Anything that is strictly application logic that you might want to port to another framework one day appears in the model. Any routing of requests, filtering, or other request handling operations appear in the controller. Views allow complex web pages to be built in a template form that the controller can populate with model data.

Success on the Server side: The Controller

I have to say that I am a fan of request routing of the form found in Rails. It is appealing to me to map a GET of http://example.com/objects/{id} easily and efficiently to the getObject(id) function within the controller. It gives a reasonable degree of flexibility in relating URLs to internal controller objects, and places the list of URLs exposed by the application and their methods in an single easily-accessible and readable place. The rails implementation might include a little bit too much magic under the covers, but the central idea to my mind is clearly valuable.

Success on the Server side: The View

The views component is also useful in talking about how we move from RESTful web sites to RESTful services. The biggest change I would make here is to look at it as a two way set of transformations. It is just as important to easily process the document from a PUT request as it is to be able to produce a document in response to a GET request. Machine-to-machine communications common to services also tend to be more structured than HTML. The concept of a template per se is of diminishing use as we move headlong into structured XML payloads, so it is probably more important to include a generic data mapping tool something along the lines of an Altova Mapforce.

Success on the Server side: The Model

The model is perhaps the most interesting component of the MVC pattern. In many ways the model could be the whole of a classical SOA service. Previously you would have had (say) an EJB Session Bean as the front-end of your service. You would expose methods on this bean that operate as a thin controller object over the entities in your model.

It seems like writing a well-designed REST service involves writing an additional layer on top of the classical SOA service. You add a controller and its set of data mappings to what you might otherwise have exposed directly to the network. In doing so you gain the benefits of a uniform interface. The implementation is still Object-Oriented, even if the interface is uniform. Just like the implementation of an object's method is still rooted in the structured programming world while the interface to the object is clearly O-O. It is the vendor products that ease the development of this additional layer of abstraction that will be most useful for REST service development.

Success on the Client Side

The client-side interfaces that provide the most leverage will be the ones that cleanly separate data mapping from communication patterns, and most cleanly expose the patterns themselves to users. Built into these patterns will be any retry mechanisms, redirection, content negotiation, etc. The client won't have to deal with individual HTTP response codes. These will be baked in.

Consider a standard GET pattern. The client application code will simply say: I want data from that URL over there, and I'm using this data mapping to interpret the results. The data mapping will inform the pattern which document types it supports as input, and the pattern will do the rest. It will either reach a success or fail terminal state, and inform the application code client of that fact.

Benjamin

Chris Samuel: Neo Freerunner QT Extended Firmware Fixed

Fri, 03 Oct 2008 11:49:23 +0000

Kudos to Lorn from Trolltech/Nokia on being responsive to the timezone problem I blogged about this morning, he’s released a new version of the QT Extended 4.4.1 firmware which fixes the timezone issue and seems to have made the touchscreen behave properly again!

Only downside at the moment is that it doesn’t seem to want to read the battery information, which is going to make life a little hard (and might be an issue with this kernel instead), but this time I’m happy to stick with it.

Neo Freerunner QT Extended Firmware Fixed

Mark Greenaway

Fri, 03 Oct 2008 11:17:14 +0000

Thanks to the generous sebthecat and the helpful people at Venue Music, my Strat now has Sperzel locking tuners fitted and the string tree removed. I haven't tried any wild vibrato arm moves yet, but it seems noticeably more stable. I got them to intonate the guitar as well, and they also saw the action was a little high and lowered it. This makes leads a lot easier to play.

I've read on various guitar web sites that fitting different machine heads and removing string trees changes the tone of the guitar. And it seems to, the tone is noticeably darker, although that could be from the lower action? In any case, it's a welcome change, and I still get those beautiful chiming bell like clean tones that all Strat players love.

Russell Coker: DKIM and Mailing Lists

Fri, 03 Oct 2008 09:00:54 +0000

Currently we have a problem with the Debian list server and Gmail. Gmail signs all mail that it sends with both DKIM and DomainKeys (DomainKeys has been obsoleted by DKIM so most mail servers implement only one of the two standards although apart from space there is no reason not to use both). The Debian list servers change the message body without removing the signatures, and therefore send out mail with invalid signatures.

DKIM has an option to specify the length of the body part that it signs. If that option is used then an intermediate system can append data to the body without breaking the signature. This could be bad if a hostile party could intercept messages and append something damaging, but has the advantage that mailing list footers will not affect the signature. Of course if the list server modifies the Subject to include the list name in brackets at the start of the subject line then it will still break the signature. However Gmail is configured to not use the length field, and a Gmail user has no option to change this (AFAIK - if anyone knows how to make Gmail use the DKIM length field for their own account then please let me know).

I believe that the ideal functionality of a sending mail server would be to have the configuration of it’s DKIM milter allow specifying which addresses should have the length field used. For example I would like to have all mail sent to an address matching @lists\. have the length field used (as well as some other list servers that don’t match that naming scheme), and I would also like to be able to specify which recipient addresses should have no DKIM signatures (for example list servers that modify the subject line). I have filed Debian bug #500967 against the dkim-filter package requesting this feature [1].

For correct operation of a list server, the minimal functionality is implemented in the Mailman package in Lenny. That is to strip off DKIM and DomainKey signatures. The ideal functionality of a list server would be that for lists that are not configured to modify the Subject line it would leave a DKIM header that uses the length field and otherwise remove the DKIM header. I have filed Debian bug #500965 against lists.debian.org requesting that the configuration be changed to strip the headers in question in a similar manner [2] (the Debian list servers appear to use SmartList - I have not checked whether the latest version of SmartList does the right thing in this regard - if not it deserves another bug report).

I have also filed Debian bug report #500966 requesting that the list servers sign all outbound mail with DKIM [3]. I believe that protecting the integrity of the Debian mail infrastructure is important, preventing forged mail is a good thing, and that the small amount of CPU time needed for this is worth the effort.

Also the Debian project is in a position of leadership in the community. We should adopt new technologies that benefit society first to help encourage others to do the same and also to help find and fix bugs.

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=500967

[2] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=500965

[3] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=500966

Michael Still: Blathering for Friday, 03 October 2008

Fri, 03 Oct 2008 08:00:00 +0000

11:47: Mikal shared: Marketplace: Credit Crisis Confusion: the one-act play

Finally I understand the subprime crisis!

"Seller: Right. So, I think the Q-grades are dumped and leveraged upwards across 25 underplummeries? Our unicorn gives it a kick, and presto: you've got 300 percent annual growth.

Buyer: Now, you just said "unicorn." There is such a thing?

Seller: Uhhh. Kind of? Honestly, I don't know. Don't care!

Buyer: Well, you also said "300 percent." So, I'm sold!"

18:00: Mikal shared: Consumerist Forced To Cut Staff [PSAs]

Its sad that consumerist is cutting two staff. I read Consumerist all the time, and its a great site. The economic downturn is quite noticable at the moment -- a couple of car dealers on El Camino Real have closed already...

Tags for this post: blather() Comment RSS with no blather

Ian Wienand: Poking around in AUXV, part 2

Fri, 03 Oct 2008 06:08:00 +0000

I've written about AUXV previously, focusing on one of its most interesting applications -- its role in helping find linux-gate.so.1.

If you're starting your program, you can get the dynamic loader to echo out the AUXV fields with the environment variable LD_SHOW_AUXV, but if your process has started you'll need to pull the values out of /proc/pid/auxv directly.

This is pretty internal stuff for the dynamic loader and is probably only useful if you're writing a debugger or doing some other low-level tricks (such as debugging!). However, should you need to, here is some sample code which does just that. Hopefully it will save someone else some time!

Unlocking IP: Resale Royalty Rights - Coming Soon

noreply@blogger.com (Abi Paramaguru) — Fri, 03 Oct 2008 05:22:00 +0000

According to this media release (via the Minister for the Environment, Heritage and the Arts, Peter Garrett) we can expect the implementation of a resale royalty rights scheme for Australian visual artists by 1 July next year.

More info is available here.

Sophia has previously blogged about the proposed scheme here.

Michael Still: MythNetTV release 6

Fri, 03 Oct 2008 03:09:00 +0000

New things in this release:

Better testing:

r114: Unit tests for video.py
r148: Updated unit test now that we create more tables
r151: Start work on syndication unit tests
r153: Unit test for bad syndication dates, as well as more flexibility in db connections needed for this unit test
r157: Nicer unit test failure output, and refactor GetVideoDir() into the utility module
r181: Slightly improve syndication unit tests
r183: Add flag parsing to unit tests, and fix some more flag name changes

Better documentation:

r117: Added a man page for video.py -- I'm not 100% happy with its name though

Bug fixes:

r114: Fixed a bug where the new filename for video transcode could be the same as the input filename, resulting in video corruption. This was found with one of the new video.py unit tests
r116: The logic for the --prompt flag was the wrong way around. Fixed.
r119: Nicer download status messages
r121: Handle 404s in feed updates better
r129: Slight tweak to SVN submit script
r131: More accurate tracking of proxy usage (update during download, instead of just at the end)
r137: Proxy budget being exceeded doesn't count as a failed download attempt
r143: Subscribe now renables inactive subscriptions
r146: Add support to decimals to utility byte printer, fix a bug in the check for video directories
r155: Have users send problems to the mailing list, instead of me personally
r161: Don't throw exceptions for the videodir command line
r167, 169: Display friendly sizes in records_tool output
r171: Move verbose update arg into a flag
r173: Add "-vo null" to mplayer invocation per Ryan Lutz. This improves support on machines without X, and speeds up the identify operation
r175: Import patch from Thomas Mashos which fixes subscription removal, started work on syndication unit test improvements
r177: Fix character escaping bug in show subtitles during import
r179: Renamed --datadirdefault to --datadir. If set this will now change your data directory, regardless of if there was a previous value set.
r190: Recording_tool now prompts for deletes
r192: Improved explainvideodir output
r194: Don't crash in explainvideodir if there is no video directory
r197: Order results by subtitle in recordings_tool output

New features:

r115: Upgraded schema to version 15 to support http_proxies for subscriptions. Added http_proxy command line, which allows you to use HTTP proxies for specified URLs. Moved HTTP accesses to use the proxy code.
r127: Bump schema to v17, and add proxy use tracking including the "proxyusage" command
r133: Allow daily budgets for proxy usage
r115: Provide a user agent for HTTP requests, instead of just Python-urllib/1.17
r117: Users will now be prompted to subscribe to an announcements video feed. This will happen exactly once. This behavior may be disabled with the --nopromptforannounce command line flag.
r125: Add a full info dump command to video.py's command line interface
r139: Bump schema to 19, and implement categories with the "category" command
r141: Implement recording group support, and clarify category support
r151: Implement a helper (recordings_tool) for handling the MythTV recordings table, this is useful for testing.
r159: Add videodir and explainvideodir debugging commands, and update man page
r163: Add title list feature to recordings_tool
r165: Include recording count in title list
r185: Add the resetattempts command

Development changes:

r123: Added a submit script to automate putting the revision number into the ChangeLog
r135: Tweak to new ChangeLog auto logging formatting

Release 6 continues the tradition of better testing, improves the documentation (a little, there is more work to be done there), fixes a bunch of bugs, and implements some new features which will hopefully be useful to others. Please grab your copy here.

Tags for this post: mythtv() mythnettv() Comment

Mark Greenaway

Thu, 02 Oct 2008 22:00:32 +0000

Getting solos right is a lot of work!

Edit: Or not ... cracked it. Guitar solos can be like Rubik's Cube, a puzzle. You eventually find a way to drape the notes across the strings so that it's not so hard after all. So far doing this I've gotten rid of the tapping and some tricky string skips. The sweep in the band number, however, is immovable.

Jon Oxer: Reunited with my clothes

Thu, 02 Oct 2008 21:31:37 +0000

By the time we wrapped at the final location in Tasmania on the weekend there was barely time for me to get to the airport to make my flight home, so I went straight there and didn't go via the hotel to grab my suitcase.

The end result was that I arrived home without suit pants, black shoes, razor, and various other things I needed for Monday morning so I turned up at work looking rather casual. Luckily one of the Directors happens to be a Melbourne boy and was coming back the next day, so my clothes and I are now in the same state again.

Russell Coker: How Many Singularities?

Thu, 02 Oct 2008 21:00:34 +0000

There is a lot of discussion and speculation about The Singularity. The term seems to be defined by Ray Kurzweil’s book “The Singularity Is Near” [1] which focuses on a near-future technological singularity defined by significant increases in medical science (life extension and methods to increase mental capacity) and an accelerating rate of scientific advance.

In popular culture the idea that there will only be one singularity seems to be well accepted, so the discussion is based on when it will happen. One of the definitions for a singularity is that it is a set of events that change society in significant ways such that predictions are impossible - based on the concept of the Gravitational Singularity (black hole) [2]. Science fiction abounds with stories about what happens after someone enters a black hole, so the concept of a singularity not being a single event (sic) is not unknown, but it seems to me that based on our knowledge of science no-one considers there to be a black hole with multiple singularities - not even when confusing the event horizon with the singularity.

If we consider a singularity to merely consist of a significant technological change (or set of changes) that change society in ways that could not have been predicted (not merely changes that were not predicted) then it seems that there have been several already, here are the ones that seem to be likely candidates:

0) The development of speech was a significant change for our species (and a significant change OF our species). Maybe we should consider that to be singularity 0 as hominids that can’t speak probably can’t be considered human.

1) The adoption of significant tool use and training children in making and using tools (as opposed to just letting them learn by observation) made a significant change to human society. I don’t think that with the knowledge available to bands of humans without tools it would have been possible to imagine that making stone axes and spears would enable them to dominate the environment and immediately become the top of the food chain. In fact as pre-tool hominids were generally not near the top of the food chain they probably would have had difficulty imagining being rulers of the world. I’m sure that it led to an immediate arms race too.

2) The development of agriculture was a significant change to society that seems to have greatly exceeded the expectations that anyone could have had at the time. I’m sure that people started farming as merely a way of ensuring that the next time they migrated to an area there was food available (just sowing seeds along traditional migration routes for a hunter-gatherer existence). They could not have expected that the result would be a significant increase in the ability to support children and a significant increase in the number of people who could be sustained by a given land area, massive population growth, new political structures to deal with greater population density, and then wiping out hunter-gatherer societies in surrounding regions. It seems likely to me that the mental processes needed to predict the actions of a domestic animal (in terms of making it a friend, worker, or docile source of food) differ from those needed to predict the actions of other humans (who’s mental processes are similar) and from those needed to predict the actions of prey that is being hunted (you only need to understand enough to kill it).

3) The invention of writing allowed the creation of larger empires through better administration. All manner of scientific and political development was permitted by writing.

4) The work of Louis Pasteur sparked a significant development in biology which led to much greater medical technology [3]. This permitted much greater population densities (both in cities and in armies) without the the limitation of significant disease problems. It seems that among other things the world-wars depended on developments in preventing disease which were linked to Louis’ work. Large populations densely congregated in urban areas permit larger universities and a better exchange of knowledge which permitted further significant developments in technology. It seems unlikely that a population suffering the health problems that were common in 1850 could have simultaneously supported large-scale industrial warfare and major research projects such as the Manhattan Project.

5) The latest significant change in society has been the development of the Internet and mobile phones. Mobile phones were fairly obvious in concept, but have made structural changes to society. For example I doubt that hand-writing is going to be needed to any great extent in the future [4], the traditional letter has disappeared, and “Dates” are now based on “I’ll call your mobile when I’m in the area” instead of meeting at a precise time - but this is the trivial stuff. Scientific development and education have dramatically increased due to using the Internet and business now moves a lot faster due to mobile phones. It seems that nowadays any young person who doesn’t want to be single and unemployed needs to have either a mobile phone or Internet access - and preferably both. When mobile phones were first released I never expected that almost everyone would feel compelled to have one, and when I first started using the Internet in 1992 I never expected it to have the rich collaborative environment of Wikipedia, blogging, social networking, etc (I didn’t imagine anything much more advanced than file exchange and email).

Of these changes the latest (Internet and mobile phones) seems at first glance to be the least significant - but let’s not forget that it’s still an ongoing process. The other changes became standard parts of society long ago. So it seems that we could count as many as six singularities, but it seems that even the most conservative count would have three singularities (tool use, agriculture, and writing).

It seems to me that the major factors for a singularity are an increased population density (through couples being able to support more children, through medical technology extending the life expectancy, through greater food supplies permitting more people to live in an area, or through social structures which manage the disputes that arise when there is a great population density) and increased mental abilities (which includes better education and communication). Research into education methods is continuing, so even without genetically modified humans, surgically connecting computers to human brains, or AI we can expect intelligent beings with a significant incremental advance over current humans in the near future. Communications technology is continually being improved, with some significant advances in the user-interfaces. Even if we don’t get surgically attached communications devices giving something similar to “telepathy” (which is not far from current technology), there are possibilities for significant increments in communication ability through 3D video-conferencing, better time management of communication (inappropriate instant communication destroys productivity), and increased communication skills (they really should replace some of the time-filler subjects at high-school with something useful like how to write effective diagrams).

It seems to me that going from the current situation of something significantly less than one billion people with current (poor) education and limited communications access (which most people don’t know how to use properly) to six billion people with devices that are more user-friendly and powerful than today’s computers and mobile phones combined with better education as to how to use them has the potential to increase the overall rate of scientific development by more than an order of magnitude. This in itself might comprise a singularity depending on the criteria you use to assess it. Of course that would take at least a generation to implement, a significant advance in medical technology or AI could bring about a singularity much sooner.

But I feel safe in predicting that people who expect the world to remain as it is forever will be proven wrong yet again, and I also feel safe in predicting that most of them will still be alive to see it.

I believe that we will have a technological singularity (which will be nothing like the “rapture” which was invented by some of the most imaginative interpretations of the bible). I don’t believe that it will be the final singularity unless we happen to make our species extinct (in which case there will most likely be another species to take over the Earth and have it’s own singularities).

[1] http://en.wikipedia.org/wiki/The_Singularity_is_Near

[2] http://en.wikipedia.org/wiki/Gravitational_singularity

[3] http://en.wikipedia.org/wiki/Pasteur

[4] http://etbe.coker.com.au/2007/08/11/is-hand-writing-necessary/

Robert Thomson: Will work for food

Thu, 02 Oct 2008 13:06:21 +0000

.. if I'm ever that desperate. But seriously, since I'm going to Italy for 10 months, and quitting my job to do it, I will probably look for some projects to keep me busy, and with the current state of the share market, I'd feel better earning money instead of digging into my savings. :-) So if anybody has any 100% remote part-time work for me (python, perl, linux, whatever) at realistic rates, starting November, let me know. The Django remote work job-board looks like an interesting option, if you weed out the adverts wanting something-for-nothing.