Planet Linux Australia
Celebrating Australians & Kiwis in the Linux and Free/Open-Source community...

May 16, 2012

EVGA GTX570 dies

Looks like my EVGA GTX570 video card just died. Cannot play anything game wise without it producing lots of artifacts.

Updated my RMA request with EVGA, as I submitted one 10 days ago. Have performed what they requested and it’s gotten worse, now just can’t use it on gaming at all.

Guess no Battlefield3 for now until they respond and confirm where/how to RMA it.

Mirror team report

The LA mirror subcommittee has decided its time to change how we work... Now that there are plenty of good ISP mirrors for distributions, we intend to get out of the business of mirroring Ubuntu and Debian, and instead focus on being the seed mirror for things which are unique to LA and Australia in general. These include LCA and other conference videos, LUG videos, and member projects that would benefit from high capacity hosting.

We keep y'all informed as we progress that plan, but we're mostly planning the transition at the moment.

For subcommittee: 

linux.conf.au 2013 report

linux.conf.au 2013 organization is chugging along nicely, with the ghosts meeting being held a little over a week ago and the zookeepr hackfest scheduled for early June. We've signed our first venue contract, and have identified venues for most of the social functions. We'll be talking more about these once they're locked in stone.

The most tangible progress for LA members is that we're very close to opening the call for proposals. That will happen at the start of June, so if you're intending to submit a proposal this year, you should put on your thinking cap now.

Cheers,

Michael Still

For subcommittee: 

Patch for Modules to use shell functions with BASH, not aliases

Whilst the Modules system is awesome in making life easy to maintain multiple versions of packages and their dependencies (and is heavily used in HPC centres like VLSCI) it can have some annoyances (and seems to be fairly half-heartedly maintained looking at the bugtracker on SourceForge). One thing that’s bitten us from time to time is that you can’t really use its “set-alias” functionality as the bash shell does not expand aliases in non-interactive shells and that includes jobs that are launched from an HPC queuing system like Torque, PBSPro, etc.

It does have the compile time option “--disable-shell-alias” but annoyingly the condition is only applied when your shell is “sh“, not “bash“, so I’ve ended up having to patch Modules to make this work for bash as well. This patch is against 3.2.9c:

--- utility.c.orig      2011-11-29 08:27:13.000000000 +1100
+++ utility.c   2012-05-16 15:08:34.012038000 +1000
@@ -1422,7 +1422,7 @@
         **  Shells supporting extended bourne shell syntax ....
         **/
        if( (!strcmp( shell_name, "sh") && bourne_alias)
-               ||  !strcmp( shell_name, "bash")
+               || ( !strcmp( shell_name, "bash") && bourne_alias )
                 ||  !strcmp( shell_name, "zsh" )
                 ||  !strcmp( shell_name, "ksh")) {
            /**
@@ -1471,7 +1471,7 @@

            fprintf( aliasfile, "'%c", alias_separator);

-        } else if( !strcmp( shell_name, "sh")
+        } else if( ( !strcmp( shell_name, "sh") || !strcmp( shell_name, "bash") )
                &&   bourne_funcs) {
        /**

Hopefully this patch will be of use to people..

This item originally posted here:



Patch for Modules to use shell functions with BASH, not aliases

Matso Ginger Beer

Finally, an alcoholic Ginger Beer that actually tastes like it has ginger in it!

image

Much LOL

At least here in Australia there is this drink called LOL. I saw much LOL on the shelves in the supermarket the other day. I’ve been trying to save this picture up to send to someone… but there hasn’t (yet) been the opportunity.

image

May 15, 2012

Links: safecrackers, media, olympics, biography, singularity

  1. Interviews With People Who Have Interesting or Unusual Jobs: Ken Doyle, Safecracker
  2. Fungible A treatise on fungibility, or, a framework for understanding the mess the news industry is in and the opportunities that lie ahead
  3. Dear New York Times & Wall Street Journal: How About Some Sensible Digital Subscription Pricing?
  4. Can London Afford the $14.5 Billion Price Tag of the Summer 2012 Olympic Games? I think Vanity Fair gets a bad rap, it has good articles and lots of pictures of pretty people
  5. I the multi-volume biography dead I’ll admit I have attempted the 8-part Winston Churchill biography but ran out of stream with a couple of parts to go.
  6. Welcome to Life « Tom Scott A science fiction story about what you see when you die. Or: the Singularity, ruined by lawyers.

May 14, 2012

Excellent resource for open source sounds

In my youth I ran a BBS, and I had a shareware cd that contained thousands of samples of different things. Quotes from movies, sounds of various things. Thought it was great, however must of lost it a long time ago.

Meanwhile, looks like a site was set up to make such things available to the general public under an open source license.

Check out FreeSound, as posted by Cool Tools website (which is where I seen it posted).

sswam

to Mr. Adam Bandt, MP for Melbourne

Thank-you for your great work as the Greens MP for Melbourne.  You must be happy to have achieved such a great victory for the Greens, by getting elected to the so-called “house of representatives”.

I want to complain about the electoral system, and I’m going to use some strong language, which is in no way directed to your good self.

If 40% of all people in each electorate vote Labour, and 51% vote Liberal/National, for the “house of representatives”, then Liberal/National would win 100% of the seats, and no one else would win any seats.

This is FUCKING CORRUPT, if I may get angry. Is it “REPRESENTATIVE” that minority parties such as the Greens are almost totally unrepresented, due to a moronic system of election?

As you know, this is a big problem not just in theory – the Greens won 11.76% of vote in the 2010 elections, which should entitle them to 18 seats out of 150 in the ‘lower house’, but they have only one seat which is yours.  I’m proud to live in Melbourne where the Greens appear to be the majority party, but where the hell are our other 17 seats?

Can we do something about this?  I suppose a referendum would be needed, but with a decent public exposee, it could hardly fail.

It would even be better to forget about the electorates entirely, and say Greens won 13% = 20 seats, Labour won 50 seats, liberal won 50 seats, now please figure out amicably which of the seats you would prefer to have, guided by which seats had the highest number of votes for each party.

I am going to contact several online petition orgs to raise this issue, it really makes me furious.

Any mathematician would laugh or cry at the stupidity of the electoral system for our house of representatives.  I’m ignorant, but I wouldn’t be surprised if most other countries have equally idiotic systems for electing their government leaders.

Thank your sir for reading my rant.  I will also send it to your email for your convenience, if you wish to respond by email!

Sam Watkins

Melbourne Australia.



Open Media Vault USB boot corruption

Looks like the Open Media Vault appliance I had running via the HP N40L Microserver had some issues.

I left it overnight to copy 400+Gb of files and woke to find it had failed half way. Host was up, but networking wasn’t working quite so well. Rebooted and immediately saw the root file system failed to mount and caused the usual panic.

I suspect the USB key I had the OS installed on has corrupted. Which means my data should be okay, although hasn’t impressed me much.

Think I will go back to my Debian install and just do everything via it. So it’s not an appliance, but at least I know it will work day in and day out when ran from the hard drive.

 

May 13, 2012

Twitter Weekly Updates for 2012-05-13

  • Spent afternoon at friends 80th birthday party, great time. No need to eat for a while! :-) http://t.co/aTYgcl9J #
  • #archaeology wrap up blog about dig at St Lythan's burial chamber in South #Wales http://t.co/1SrGACK8 #
  • Performing summon $COURIER spell, making a pot of tea thereby ensuring being called to a different building for delivery #
  • http://t.co/P5iJVm9R MT @sijoe I can replace my ridiculously complex Apache log parser logic with a single Regex … WOOT!!! #
  • Summon $COURIER spell worked, though they (again) ignored the instruction to phone 15 mins before getting here.. #
  • Interesting to see that the European Synchrotron is migrating from RHEL/Centos to @debian #Linux https://t.co/7eIITu4p #
  • Anyone know how to get updatenode in #xCAT to copy dangling symlinks to managed nodes? It uses rsync under the covers.. #
  • .@MobiCity friend of mine has ordered a pair of GN's from you last night because you're firmware flash friendly, well done! #
  • #Melbourne urban sunset from a @metrotrains train http://t.co/p9v6VlRS #
  • Channelling query from #lwloc developers, has "inline" always been a part of C++? Need good source to confirm/refute. #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-05-13

Twitter Weekly Updates for 2012-05-13

  • Spent afternoon at friends 80th birthday party, great time. No need to eat for a while! :-) http://t.co/aTYgcl9J #
  • #archaeology wrap up blog about dig at St Lythan's burial chamber in South #Wales http://t.co/1SrGACK8 #
  • Performing summon $COURIER spell, making a pot of tea thereby ensuring being called to a different building for delivery #
  • http://t.co/P5iJVm9R MT @sijoe I can replace my ridiculously complex Apache log parser logic with a single Regex … WOOT!!! #
  • Summon $COURIER spell worked, though they (again) ignored the instruction to phone 15 mins before getting here.. #
  • Interesting to see that the European Synchrotron is migrating from RHEL/Centos to @debian #Linux https://t.co/7eIITu4p #
  • Anyone know how to get updatenode in #xCAT to copy dangling symlinks to managed nodes? It uses rsync under the covers.. #
  • .@MobiCity friend of mine has ordered a pair of GN's from you last night because you're firmware flash friendly, well done! #
  • #Melbourne urban sunset from a @metrotrains train http://t.co/p9v6VlRS #
  • Channelling query from #lwloc developers, has "inline" always been a part of C++? Need good source to confirm/refute. #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-05-13

Twitter Weekly Updates for 2012-05-13

  • Aww, thx! :) #
  • .@John_Hanna Hmmm, not really that I know of. Was international list. Most people in the industry are great, men & women alike. A few tools #
  • .@williamparry don't hate, then the terrorists win ;) #
  • . @John_Hanna The argument devolves too quickly into one about free speech. I want to understand why anyone would think this way & address. #
  • Wow, amazing that some ppl actually speak & think this way. A minority but a loud minority. http://t.co/nkBxqqKe #womenhaters #
  • "i find out who I am when I'm climbing the mountains of mars" http://t.co/kEWY8Xze /cc @kelisha @swearyanthony #
  • Check out these guys, they do "literal trailers" for movies.and games. Much lols. #ac3 For @swearyanthony http://t.co/KcT6ND2C #
  • "If it bleeds we can kill it". So much lols. http://t.co/KG7vEY06 #
  • Ohcrap, thx @Wittylama RT Klout http://t.co/lHJGzhPi #
  • OK, I'm really at the end of my tether, anyone in the Canberra area able to loan me a fast laptop for a week or two? Just need web browsers. #
  • Some #govhack media, thanks for the coverage :) @jamie_kirk http://t.co/CqCOKYK6 & a kinda-shout-out from @stilgherrian http://t.co/nKnAQ85U #
  • HAH! Your mum must be proud ;) RT @FakePaulKeating: @piawaugh pfft. Lightweight. http://t.co/uaGZLifw "Facepalm central" #
  • MWAH hahaha! Thanks @akshatj_96 & @purserj for Klouchebag lulz. My score is 27, or 'mostly alright', hah! http://t.co/9WPFx619 #
  • Wow, just joined @Klout. Fascinating! Wish it included Soundcloud ;) "According to @klout, my Klout score is 45." http://t.co/qtrphA1x #
  • An awesome song to start work to. Enjoy http://t.co/3TK8YN8s #music #
  • "Where to ebooks go when you do?" Great article, tackles the major challenge of digital culture wrapped up in DRM http://t.co/fyu0i3pV #
  • It's funny how we feel most safe in motion, but only when still are we able to really see what's around us. Kung Fu thought for the day. #fb #
  • Dammit! RT @ACTwonkdrinks: Not enough interest for a Budget Edition of #actwonkdrinks Why kind of wonks are you lot? #fail #
  • Have had to limit the size of Sydney #GovHack due to venue, so get in quick before we hit capacity! http://t.co/YS5Zwh26 28 seats left atm #
  • Right, thanks all, looks like CC-BY is on individual #budget papers, but not on copyright page http://t.co/lqwgNZn0 Be good to fix that :) #
  • What happened to the CC-BY for the Budget papers? http://t.co/9ZHNJThm That's quite unfortunate, seems a backwards step. #
  • Went to write democracy and typo'd demoncracy. I think there's something in that for all of us. #
  • Kudos to the Adobe http://t.co/EZzCyBnH tool. It''s a bit slow but works really nicely :) #
  • I lie, it works in IE6 and IE9, but not well in IE8 and not at all in IE7. Wonderful. #
  • Internet Explorer, the bane of web developers everywhere. How could it work in IE6, 7 & 9, but not 8. FFS. #backtodrawingboard #
  • cool, done :) #
  • Nice, now over 100 ppl registered for #govhack Hoping for 300 so go register (Sydney or Canberra). Will be awesome :) http://t.co/aXgjAzLL #
  • . @gavintapp Shiny, but seriously, promoting a "developers" laptop and then saying "key tools and utilities (emacs, Vim, Chromium etc)" ;) #
  • Funny. I get so much spam on Google+. Facebook and Twitter are wonderful by comparison. #
  • Wow, the @SensisAPI zombie app challenge is teh awesome! http://t.co/LjdrcZT8 #gov2au #
  • Putting together a #GovHack team to compete 1-3 June? Check out some previous mashups/hacks on http://t.co/ab5WZIGh http://t.co/p04DDIz4 #
  • Interesting RT @wtfsheep: @parisba I thought this was an awesome open data mashup: http://t.co/YWFaporR #CHI2012 #govhack #
  • So, going to see Smashing Pumpkins and Tea Party in July, all I need is Tool, Kyuss, Stabbing Westward, NIN, Lamb and I'd be in heaven :) #
  • I know, crazy hey, but I tweeted a new song of theirs earlier today which was surprisingly good. So am excited :) #
  • WOW! I am now also going to Smashing Pumpkins with @alexmyoug, looks like July will be #relivemyyouth month! :) SOOOOO EXCITED! #music #
  • Just received the Tea Party tickets in the post!!! Hey @alexmyoung, I can't wait! #
  • Fascinating RT @OZloop: APS - The perfect storm. Cutbacks, culture and abuse http://t.co/ZKJitFQS #gov2au #opengov #innovationweek #govcamp #
  • . @1159 Heh :) Saw Tool in '97, still one of my favourite gigs ever. They covered Hurt too, double perfect! :) #
  • "See my shadow changing, stretching up and over me. Soften this old armour, hoping I can clear the way". Tool is helping me work today :) #
  • I love Linux. Some useful ffmpeg for converting and manipulating video/audio without having to open video editing suite http://t.co/uBGQ8dUn #
  • Have you registered for #GovHack yet? Over $30k in prize funding for awesome apps, mashups & datavis, so come get some! http://t.co/aXgjAzLL #
  • Wow. New Smashing Pumpkins, and surprisingly awesome. http://t.co/gHWBu29J #music #

Twitter Weekly Updates for 2012-05-13

  • Aww, thx! :) #
  • .@John_Hanna Hmmm, not really that I know of. Was international list. Most people in the industry are great, men & women alike. A few tools #
  • .@williamparry don't hate, then the terrorists win ;) #
  • . @John_Hanna The argument devolves too quickly into one about free speech. I want to understand why anyone would think this way & address. #
  • Wow, amazing that some ppl actually speak & think this way. A minority but a loud minority. http://t.co/nkBxqqKe #womenhaters #
  • "i find out who I am when I'm climbing the mountains of mars" http://t.co/kEWY8Xze /cc @kelisha @swearyanthony #
  • Check out these guys, they do "literal trailers" for movies.and games. Much lols. #ac3 For @swearyanthony http://t.co/KcT6ND2C #
  • "If it bleeds we can kill it". So much lols. http://t.co/KG7vEY06 #
  • Ohcrap, thx @Wittylama RT Klout http://t.co/lHJGzhPi #
  • OK, I'm really at the end of my tether, anyone in the Canberra area able to loan me a fast laptop for a week or two? Just need web browsers. #
  • Some #govhack media, thanks for the coverage :) @jamie_kirk http://t.co/CqCOKYK6 & a kinda-shout-out from @stilgherrian http://t.co/nKnAQ85U #
  • HAH! Your mum must be proud ;) RT @FakePaulKeating: @piawaugh pfft. Lightweight. http://t.co/uaGZLifw "Facepalm central" #
  • MWAH hahaha! Thanks @akshatj_96 & @purserj for Klouchebag lulz. My score is 27, or 'mostly alright', hah! http://t.co/9WPFx619 #
  • Wow, just joined @Klout. Fascinating! Wish it included Soundcloud ;) "According to @klout, my Klout score is 45." http://t.co/qtrphA1x #
  • An awesome song to start work to. Enjoy http://t.co/3TK8YN8s #music #
  • "Where to ebooks go when you do?" Great article, tackles the major challenge of digital culture wrapped up in DRM http://t.co/fyu0i3pV #
  • It's funny how we feel most safe in motion, but only when still are we able to really see what's around us. Kung Fu thought for the day. #fb #
  • Dammit! RT @ACTwonkdrinks: Not enough interest for a Budget Edition of #actwonkdrinks Why kind of wonks are you lot? #fail #
  • Have had to limit the size of Sydney #GovHack due to venue, so get in quick before we hit capacity! http://t.co/YS5Zwh26 28 seats left atm #
  • Right, thanks all, looks like CC-BY is on individual #budget papers, but not on copyright page http://t.co/lqwgNZn0 Be good to fix that :) #
  • What happened to the CC-BY for the Budget papers? http://t.co/9ZHNJThm That's quite unfortunate, seems a backwards step. #
  • Went to write democracy and typo'd demoncracy. I think there's something in that for all of us. #
  • Kudos to the Adobe http://t.co/EZzCyBnH tool. It''s a bit slow but works really nicely :) #
  • I lie, it works in IE6 and IE9, but not well in IE8 and not at all in IE7. Wonderful. #
  • Internet Explorer, the bane of web developers everywhere. How could it work in IE6, 7 & 9, but not 8. FFS. #backtodrawingboard #
  • cool, done :) #
  • Nice, now over 100 ppl registered for #govhack Hoping for 300 so go register (Sydney or Canberra). Will be awesome :) http://t.co/aXgjAzLL #
  • . @gavintapp Shiny, but seriously, promoting a "developers" laptop and then saying "key tools and utilities (emacs, Vim, Chromium etc)" ;) #
  • Funny. I get so much spam on Google+. Facebook and Twitter are wonderful by comparison. #
  • Wow, the @SensisAPI zombie app challenge is teh awesome! http://t.co/LjdrcZT8 #gov2au #
  • Putting together a #GovHack team to compete 1-3 June? Check out some previous mashups/hacks on http://t.co/ab5WZIGh http://t.co/p04DDIz4 #
  • Interesting RT @wtfsheep: @parisba I thought this was an awesome open data mashup: http://t.co/YWFaporR #CHI2012 #govhack #
  • So, going to see Smashing Pumpkins and Tea Party in July, all I need is Tool, Kyuss, Stabbing Westward, NIN, Lamb and I'd be in heaven :) #
  • I know, crazy hey, but I tweeted a new song of theirs earlier today which was surprisingly good. So am excited :) #
  • WOW! I am now also going to Smashing Pumpkins with @alexmyoug, looks like July will be #relivemyyouth month! :) SOOOOO EXCITED! #music #
  • Just received the Tea Party tickets in the post!!! Hey @alexmyoung, I can't wait! #
  • Fascinating RT @OZloop: APS - The perfect storm. Cutbacks, culture and abuse http://t.co/ZKJitFQS #gov2au #opengov #innovationweek #govcamp #
  • . @1159 Heh :) Saw Tool in '97, still one of my favourite gigs ever. They covered Hurt too, double perfect! :) #
  • "See my shadow changing, stretching up and over me. Soften this old armour, hoping I can clear the way". Tool is helping me work today :) #
  • I love Linux. Some useful ffmpeg for converting and manipulating video/audio without having to open video editing suite http://t.co/uBGQ8dUn #
  • Have you registered for #GovHack yet? Over $30k in prize funding for awesome apps, mashups & datavis, so come get some! http://t.co/aXgjAzLL #
  • Wow. New Smashing Pumpkins, and surprisingly awesome. http://t.co/gHWBu29J #music #

What I REALLY Want from the NBN

Generally I haven’t had a positive attitude towards the NBN. It doesn’t seem likely to fulfill the claims of commercial success and would be a really bad thing to privatise anyway. Also it hasn’t seemed to offer any great benefits either. The claim that it will enable lots of new technical developments which we can’t even imagine yet that aren’t possible with 25Mb/s ADSL but which also don’t require more than the 100Mb/s speed of the NBN never convinced me.

But one thing it could really do well is to give better Internet access in remote areas. Ideally with static or near-static IPv6 addresses (because we have already run out of IPv4 addresses). Currently 3G networks do all sorts of nasty NAT things to deal with the lack of IPv4 addresses which causes a lot of needless pain if you have a server connected via 3G. One of the NBN plans is for wireless net access to remote homes, with some sanity among the people designing the network such NBN connections would all have static IPv6 subnets as long as they don’t move.

I’m currently working on a project that involves servers on 3G links. I don’t have a lot of options on implementation due to hardware and software constraints. So if the ISPs using the NBN and the NBN itself (for the wireless part) could just give us all IPv6 static ranges then lots of problems would be solved.

Of course I don’t have high hopes for this. One of the many ways that the NBN has been messed up is in allowing the provision of lower speed connections. As having an ADSL2+ speed NBN connection is the cheapest option a lot of people will choose it. Therefore the organisations providing services will have to do so with the expectation that most NBN customers have ADSL2+ speed and thus they won’t provide services to take advantage of higher speeds.

Related posts:

  1. RPC and SE Linux One ongoing problem with TCP networking is the combination of...
  2. A New Strategy for Xen MAC Allocation When installing Xen servers one issue that arises is how...
  3. New Net Connections On Thursday my new InterNode ADSL2+ service was connected [1]....

vmdksync helps you escape from VMware

When I wrote lvmsync late last year, I didn’t realise I was being typecast. Before too long, I realised that the logic that I’d implemented for lvmsync would also help me with a separate migration project I’d been dreading – getting the day job off VMware.

Back in the early days of virtualisation, management made the decision to run VMware, for all the usual reasons (“commercially supported!”, “industry standard!”, and so on). Unsurprisingly (to me, anyway) it didn’t take too long for management to realise that it wasn’t the best choice for us. When you’ve got umpty-billion dollars to spend on hardware, software, and support, VMware might be the right option (although Amazon doesn’t seem to think so). Anchor’s company culture, on the other hand, is build around “smart staff, simple systems” over “dumb staff, smart vendors”, because no vendor is ever going to care about our customers as much as we do. So VMware was never going to work for us.

Unfortunately, as happens all too often, once VMware was in place, there was very little motivation to get rid of it and move those customers onto the chosen replacement (that we were deploying all new customers on). I happen to think this is a terrible attitude in general – one that makes life so much harder in the long term. I believe strongly in retrofitting old systems to keep them up-to-date with the current state of the art, and keeping technical debt under control. But, I wasn’t running the show back when we stopped putting new customers on VMware, so the few VMware servers we had stayed around far longer than they should have.

Recently, though, bad things started to happen. The VMware servers were starting to fall apart. The Windows machine we had to keep around to use the VMware management console started crapping out, and when the choice was between doing unspeakable things to Windows, and just ditching VMware… well, it wasn’t much of a choice. The only remaining question was how to do the migration off VMware with the least amount of downtime to our customers.

I was really quite surprised that nobody out in Internet land appeared to have come up with a simple, robust tool to do this. Sure, some vendors had all-singing, all-dancing toolkits that cost ridiculous amounts of money, required you to install their agent on the machine involved, and promised the earth, but it all smelt of snakeoil and bullshit.

In true hacker style, then, I decided to write something myself. The model I came up with mirrored lvmsync’s quite closely – because that one worked, and it turned out to be surprisingly easy to implement once I managed to reverse-engineer the file format (VMware has a PDF spec of a bunch of it’s file formats, but whoever wrote it was enough of an evil genius to make it utterly incomprehensible to anyone who doesn’t already know the file format, whilst making perfect sense to anyone who already does).

The result: vmdksync. It is nothing but 80-odd lines of ruby whose sole purpose is to take a delta.vmdk file and write the changes that are stored in that file to a file or block device that is a copy of the flat.vmdk file that you can copy while the VM is still running (after you’ve made a snapshot, of course). It helped me provide a painless migration path away from VMware, and I’d be really pleased if it helped some other people do the same. Share and enjoy!

May 12, 2012

More from Bruce…

Hello! Have you missed me? It’s been a while since I’ve updated you all on what’s happening in the world of PyCon Australia, so I figure it’s probably about time we did that. And it’s good that I’m doing so, because a lot of things have happened since the last time I did so!

Talks, talks, and more talks

Our Call for Proposals closed its doors on Friday 4 May, and we’ve been absolutely blown away by the level of response that we’ve got from Python developers around Australia and the rest of the world. We received 59 proposals to speak, across three categories of presentation, which is far and away the biggest response this conference has had in its short history. So, to all of you who proposed presentations, give yourselves a pat on the back.

To put this into perspective — we have approximately 30 positions that we can fit presentations into. Our review team (who are, by the way, doing an incredibly awesome job) have the mammoth task of figuring out which talks will actually make it into the conference: they’ll need to cull approximately half of the proposals that we’ve received. It’s something that we really weren’t expecting, but I think the review team are up to the task.

With this in mind, it’s going to take us a few days more than expected to sort through the proposals, and we won’t be meeting our deadline of 18 May for sending out notifications. It won’t be too much later than that, but we still apologise for the delay. If you did submit a talk, don’t worry too much about missing out on Early Bird registration rates — we’ll be extending Early Bird pricing through to June 30 for everyone who has submitted a talk. I hope this is OK by you!

We want you to come to our conference

And the best way to make sure that you can actually come to the conference is by signing up for our Early Bird registrations! These have been open for just over two weeks now; and with three weeks left to go, just over a third of our available early bird tickets have been sold. Our early bird registrations represent a substantial discount on our normal registration rates, and they also guarantee you a spot at our conference dinner (which is both space-limited, and is seriously not to be missed).

Early bird regos are available for the first 60 Enthusiast or Professional tickets sold; all the relevant info is at our website.

… and that means all of you!

We’ll be reprising the very successful gender diversity grants programme that PyCon Australia launched with Google last year; in a much-expanded form. Last year these grants helped many deserving women attend the conference with subsidised registration, and some travel allowance. This year, the grants programme will offer travel assistance to many more deserving female delegates. We’ll reveal more details later, but needless to say, we’re very excited.

fin.

Well, that’s it for now, I hope you’re as excited about the conference as I am. It’s shaping up to be really quite special, and I can’t wait to share more of our plans with you. See you in August, and get registering!

Python-iView update: fixes downloading for metered users

Just pushed out an update to Python-iView that fixes issues with “StreamNotFound” errors when downloading from a metered (and/or Akamai-based) connection.

The problem occurred due to a minor change in the iView API, which contradicted an assumption I had made.

Bazaar users can type the following to update to the latest version:

$ bzr pull

PPA users can simply update to the latest version by running the following:

$ sudo apt-get update
$ sudo apt-get upgrade

Ubuntu 11.10 (Oneiric) users may need to run the following instead:

$ sudo apt-get install python-iview=0.2-1~bzr73~oneiric

I am not providing packages for Ubuntu 10.10 or 11.04 any more. (Actually binary-wise the packages do not differ between distros, so you can quite easily use another distro’s name in your sources.list.)

Value Added Tax?

Could it be? No! Never!

A few weeks ago I saw a tweet by the Drupal Assciation asking for feedback on, amongst other things, how it could make being a DA member more attractive. The survey appears to still be open, so if nothing else please go fill it out.

However, before you do so I would draw your attention to question 15:

How much would the following benefits interest you in becoming a Member (or renewing your Membership)?

and in particular one of the options listed there:

"Discounts on software (e.g., something like Adobe)"

When I read that I was flabberghasted. That would effectively mean the Drupal Association subsidising a proprietary software company, helping them to expand their market share. That's not why I use and support an open source project. In fact, I use and support open source software for the completely opposite reason!

Ah, I hear you say, but the Drupal Association has no control over the software project, so this doesn't matter. I beg to disagree. I think this would send a strong signal that the open source philosophy isn't actually an important part of Drupal.

I am amongst the first to admit that there may be a place for proprietary products in the world. In fact, I have an iPhone, which is a completely closed and proprietary device. Some (though not all) designers probably have perfectly valid reasons for needing to use Adobe's software.

However, I very much don't want to be part of an organisation or project that would support these proprietary software companies by providing them with extra sales. Especially not companies that try to sting their customers for as much money as they possibly can.

ps: The attached image was inexpertly created using The Gimp. Both I and it can do much better when not rushed ;-)

[life] Zoe at 24 months

Once again, another 3 months have flown by.

Honestly, it's been a total blur. What on earth has happened since February?

Well, heaps more travel of course.

There was the trip to New Orleans, which I already wrote about.

The other big trip we did was to Washington D.C. for a week, which I'm yet to write up. We stayed with friends in Alexandria for the week. Unfortunately, Zoe came down with conjunctivitis before Sarah's very eyes on the flight over (I'd been in North Carolina for work the week before, so they flew without me and I flew up to D.C. to meet them). She also developed an ear infection and a general cough/cold.

Fortunately, we caught it all pretty much as soon as it happened, and spent Easter Sunday at an Urgent Care in Virginia getting antibiotics for her, but her sleep was not so great the whole time we were in D.C.

Other than that, we've just been doing the normal stuff. Swimming classes have continued (she's very confident in the water now). Her speech continues developing well. Toilet training is continuing to go very well. There haven't been very many accidents at all. She's really loving the "new" day care. It was definitely the right thing to do to move her.

Physically, she's in the 80th percentile for height and the 50th percentile for weight, so I guess she's currently tracking to be on the tall and thin side of things.

We had another birthday party in the park behind our home, and this year the weather was much nicer, so we had people hanging around until about 7pm, and a good time was had by all. Zoe even successfully blew out her candle on the first attempt.

Her two-year molars still haven't come through. They seem to go through various stages of giving her a lot of grief, and then they stop messing with her sleep. I was beginning to think we'd never get back to an uninterrupted night's sleep ever again. It's still a bit hit and miss, but she usually settles down again fairly quickly if she does wake up, and doesn't always require intervention from us.

I've arrived at the theory that the "terrible twos" are largely the fault of trying to cut these two year molars. Her tantrums are so much worse when her teeth are actively giving her trouble. If she's had a good night's sleep and her teeth aren't driving her crazy, she's still pretty well behaved. If she's been awake half the night and her teeth are hurting during the day, she can be quite difficult. We seem to be currently at a point where they're not causing too much trouble at night, but she'll chew on her fingers like crazy if given the chance during the day.

I think seasonal allergies are also messing with her a bit. After she got over the cold she came down with in Washington D.C., she's still had a runny nose and a bit of a cough, but I don't think she's actually "sick" I think it's more related to allergies. She passed on whatever she had to me, and since I've recovered from it I'm also not feeling quite right.

I read somewhere that pollen counts are off the charts this year due to the weird weather, and Sarah's having a bit of a hard time as well, so I think we're all going to just have to ride it out with the judicious use of antihistamines.

Speaking of allergies, it's about time to re-test her for her egg allergy. We'll have to try that at a point when she's not regularly taking antihistamines for seasonal allergies, so it might be a while yet before we can do that.

I guess the next big development will be graduating from the crib to a toddler bed. I'm not in any hurry to do that though, so as long as she's not escaping the crib, her sleeping arrangements can stay the way they are. It'd be nice if she can cut her teeth first and then once she's sleeping properly, we can try converting the crib into a toddler bed.

Zoe walking down Castro Street in Mountain View

Links: China vs UK, glasses, unlimited flying & Online Ed.

  1. What My 11 Year Old’s Stanford Course Taught Me About Online Education
  2. Recycling Eyeglasses Is a Feel-Good Waste of Money If nothing else I need to pay less when I get some new glasses.
  3. A Tale of Two Terminals comparison between the launches of Terminal 3 at Beijing Capital Airport and Heathrow Terminal 5 from the perspective of someone used to handling complex software implementations
  4. The frequent fliers who flew too much This reminds me a lot of Internet (especially ISP) based unlimited accounts that fail to take account of people’s usage patterns when additional usage is free
  5. When half a million Americans died and nobody noticed Has the death toll from the drug Vioxx been greatly underestimated?

May 11, 2012

How the Others View DVDs

AttachmentSize
GxzeV.jpg163.41 KB

This picture was interesting. Not having watched a DVD on anything other than Linux, it was truly eye opening to see what the media companies put those who don't use Free Software, the majority of the population, through. What's more amazing is that people accept this:

Thanks Bruce!

Blog topics: 

Brew Dog Punk IPA

This is a delicious beer. The text on the side of the bottle is rather worth reading too. Basically, it has flavour. It’s not a “we’re going to hop your brains out” IPA, rather a “we’re making an IPA with flavour that’s very yummy”.

image

May 10, 2012

Web Site Updates

I am about to head to Dayton for the Hamvention, so took the opportunity to bring my web site up to date. The Codec 2 page includes our latest plans on building a Linux/Windows GUI Application for HF Digital Voice, has updates on recent algorithm developments, links to conference videos, and lots of figures explaining how Codec 2 works. I have added a FDMDV modem page, and updated my About Page to reflect my current projects and motivation. The Media page includes updates on recent conferences and radio interviews, and has a section on how this post on busting my daughters party with a Fluksometer went viral around the world, ending up back at my local newspaper!

Catching Fire







ISBN: 9781407109367

LibraryThing

This is the second book in the Hunger Games trilogy after the first book of the same name. This book is very similar to the first in terms of style, although I think the start is a lot slower. Once you plow through the first 150 pages or so the book rapidly improves though, and I was happy with where it went overall. Good teen fiction.



Tags for this post: book suzanne_collins combat hunting post_apocalypse hunger_games

Related posts: The Hunger Games; Death Bringer; Battlefields Beyond Tomorrow ; East of the Sun, West of the Moon; Canned hunting; Bolos 1: Honor of the Regiment; Iron Master; Cloud Warrior; Amtrak Wars; Earth Thunder; First Family; Emerald Sea; Body Armor: 2000; Without Warning; Blood River; Against the Tide; The Stars Must Wait; Bolos 2: The Unconquerable; There Will Be Dragons
Comment Recommend a book

Advertising

Gumtree working fine - more to come.

For subcommittee: 

May 09, 2012

Blue Hackers Seattle Meet Up, Sunday 5/13

Blue Hackers MeetUp

Sunday, May 13th

TIME CHANGE–2 to 3pm

Jigsaw Renaissance

(at the Inscape Building, 815 Seattle Blvd S, 98134)

Frank and open discussion about mental health in hacker/maker communities. Join us to get or give support, or just to know you are not alone with your challenges (mood and anxiety disorders, ADHD, Asperger’s/PDD, suicide, self harm, and more).

Twitter Weekly Updates for 2012-05-09

  • moved to new flat, bought furniture and appliances, set up utilities, got engaged, enrolled at uni, went to jury service… all in 2 weeks! #
  • $11.7m for OLPC Australia in the federal govt budget! Huzzah! http://t.co/zgmqAnMP #

Drupal Melbourne Talk: Webform MySQL Views

Don't abandon the kittens!

These are the resource links for my Webbform MySQL View talk at the meetup on May 8th.

Modules used on the demo site:

SSD acquired

I’ve been tempted to purchase an SSD for my home desktop system for sometime. This is the same system I use to run a number of virtual machines and for playing battlefield3.

Got the SSD on Monday. Purchased a Corsair 120Gb Series 3 and reinstalled Windows. The time it takes for Windows to boot to logon screen is only about 12 or so seconds.

Below is how my Windows Index scores changed, and I knew this change for my boot drive would bring the score up quite a lot. So knew it was worth a purchase.

Game play via BF3 seems better too, although only early days. I need more time to play some more. I installed BF3 onto the SSD too, and have installed VMware Workstation to a 7200rpm traditional internal drive.

Heroku's new $50 per month production database

With a $150 per month reduction in the minimum dedicated database price, Heroku suddenly becomes viable for many more of our apps.

Permalink

The Darker Side of Software Development

Jim Whimpey read my last post and shared exactly how he feels when consumed with a software problem:

After hours without progress you go to bed thinking about it. Thinking about it makes you angry and frustrated but you can’t stop thinking about it. You spend so long at the computer you lose all perspective and follow paths that take you further from the solution. With each minute you spend banging your head against the problem you think less clearly and the possibility of solving the problem decreases. It is frustrating.

This is exactly what happened with me. I think there’s only so many times you can work like this before you burn out. It’s important to learn from those moments and develop a sense for when they begin to arise, so you know when it’s time to step back and short-circuit the cycle.

Permalink

May 08, 2012

ISIF award for Dili Village Telco

This is somewhat old news but last September Fongtil and I won an award for our work on the Dili Village Telco. Here is the ISIF press release. We were selected from a group of about 50 ISIF funded projects for the award. Here is a photo of Lemi (2nd from right) accepting the award in Nairobi, Kenya:

Although this project was my first developing world deployment and I worked hard to make sure we did a good job, soaking up experience and knowledge from others where I could.

I think we got it about 70% right. Many things (ease of use, training, enthusiasm for local assembly and installation) worked really well. However being an Engineer there are still some remaining “bugs” I feel compelled to work on, for example there are still lingering issues with Wifi link quality, and a sustainable business model. Importantly, I haven’t seen viral growth, either in Timor or other Village Telco deployments. This is important for me – my goal for the Village Telco was to help a lot of people in the developing world get telephony. This can only happen if the local people embrace the technology and grow the networks themselves without reliance on grants or 1st world technical input. I do think many of the pieces for this are in place. “The Engineer” in me would love to have a go at really fixing those remaining issues! Still, this award confirms we did pretty well over all.

Although the funded part of the project officially ended in 2011, Fongtil are continuing to work on the project using internal resources and have also secured a further ISOC grant. They continue to train people, install nodes, evangelise with government (e.g. installing Mesh Potatoes in the National Parliament building), and have plans to fix the Wifi link issues in Dili:

I have moved my time and attention to other projects for now, but would love to get back to Timor some time to help out if I can. In the mean time Village Telco development and deployment work continues in Timor Leste and around the world.

TasLUG (Hobart) May Meeting has Been Announced

TasLUG have announced the details for this month's meeting:

When: Wednesday, May 16th

Time: 18:00 for an 18:30 start

Venue: 57F Brisbane St, Hobart

Agenda:

  • Welcome!
  • New venue Review / Discussion
  • Statewide Meeting Review / Discussion
  • Distro Wars!

    Line up and give us 10 minutes on your favourite distro framed around:
    • What's the target audience / usage
    • What makes it rock
    • What could be done better
    • Current Talks:
  • Decide on what we'd like Karl to talk on in June:
    • Debian (random stuff, requests?)
    • Gnewsense (DIY ubuntu/debian derivative, requests?)
    • ITShare (computer refurbishing)
    • ITShare's work in Papua new guinea, or 'stuff' about Goroka/PNG
    • Issues from running the computers of an NGO (The Hut community centre)
    • Any other things? By request ;)
  • General Questions Session
  • Call for talks
  • Next Meeting?
  • add something else here!

Coming Attractions:

  • June: Karl will be talking on what we request :)
  • July: Julius will be giving a system administrator's over view of ZFS.
Blog topics: 

Courier IMAP and FAM

Last Friday, while tracking Debian Testing, the courier package was updated, and while authentication could be seen to be successful, actually using IMAP seemed to fail.

Turns out the FAM package was somehow to blame; installing fam and libfam0 was the solution. This uninstalled gamin for me. So if you’re pulling your hair out with a similar courier/imap issue, then perhaps have a look at the courier-imap mailing list.

May 07, 2012

PyCon AU 2012 Report #3

CFP was very well received - the number of proposals received after

'soft closing' on Friday May 4 was almost double the number of spaces

available in the conference. We're still expecting one or two late

submissions.

Early bird registrations have been open for just over a week; we've

sold just over 25% of our early bird tickets, along with a few student

tickets as well.

We hope to have the list of talks at the conference finalised by the

middle of May, along with announcements of our keynote presenters.

For subcommittee: 

FDMDV Modem Page

I have added a FDMDV modem page to this web site, plus typed up a README_fdmdv.txt that explains all the files related to the modem and how to use them.

This modem was “lost” a few years ago as the initial implementation was closed source. So I felt compelled to put a fair bit of effort into documenting the open source implementation I have been working on. Building this modem was fun, just hard enough to be challenging but no real show stopper bugs and didn’t hurt my head like Codec 2 algorithm development.

Next step is to tweak Codec 2 to make it interface cleanly to the modem for some initial on-air tests.

Orbital

Saturday night, went to see Orbital. Awesome. A two hour set of a whole bunch of fun and a crowd that was really into it. Luckily there was a mirror nearby to capture the stage and the crowd.

image

Hargraves Hill Extra Special Bitter

I think this has to be one of my favourites. I  don’t get to have it often, but it has such a wonderful flavour: strong, not overwhelming and a wonderful example of what an Extra Special Bitter should be. I know you can get it on tap at Mrs Parmas in Melbourne, although I swear I’ve found it at other places too – and anyone who doesn’t mind an ESB should certainly give this one a try.

image

brendanscott

More on AusGOAL lack of Understanding of Formats:

The AusGOAL’s statement on open formats is confused.  While the examples given are good, the criteria for identifying whether a format is open or not are of no use.  They say:

An open format is a specification for storing and manipulating content, that is usually maintained by a standards organisation.  In contrast, a proprietary format is usually maintained by a company, with a view to exploiting the format by incorporating it into other products they sell, such as software.

“Which is like, ‘no’”

The fact that a format is “usually maintained by a standards organisation” is hardly a useful criteria.  In that case, every ISO standard would be open, despite being subject to patent claims or permit the inclusion of unspecified binary blobs. The only good thing about this is that it hints at the fact that the standard must have some transparency to its creation.  Compare this to the US government’s description:

“An open format is one that is platform independent, machine readable, and made available to the public without restrictions that would impede the re-use of that information”

This, at least, says something about the format (“is” vs “usually”).

Assessing a format for openness is quite difficult.  I think the criteria should be focussed on the practical implications of the standard.  In particular, if it creates a gatekeeper that people wishing to implement the standard must go to for any reason at any time (eg for getting a reasonable, nondiscriminatory licence), then it is not open.  Other criteria, such as the transparency with which it is developed/maintained go not to whether it is open per se, but to whether it is advisable to adopt it.

Frankly, I think the best test  of whether a format is open or not is whether it has a free software implementation.  This, of course, is not a definition – an open format is still open immediately before the free software implementation is finished.  The existence of such an implementation implies that the rules of the format are now discernable from the code and that there is freedom to implement those rules.

 

 



May 06, 2012

Money, Stress and The Cloud

Or: How to Save Your Relationship with Rails Streaming Responses on Heroku

This is as much a personal story as one about software development. It’s about dealing with problems in a production app, handling credit card payments on Heroku, working with customers, implementing custom HTTP streaming responses in Rails 3.0, all while living a happy life alongside.

I failed at the last part, which has actually made this article somewhat painful to write. Although the events occurred in November and December 2011, it’s taken this long to share, not only because it’s a long story, but also because I was happy to have this stuff behind me. However, I believe it is illustrative on both a technical and personal level, and contains some useful lessons, so here you go!

The Problem

I’m at a mountain resort, palm trees shading the warm tropical sun, with a cold bottle of San Miguel Light in my hand. My wife and I have just arrived in the Philippines, where we’ll be spending the next eight months. I’m hanging out for a week during her volunteering orientation, before we head out to our new home town.

Towards the end of that week, a rather disturbing email came to me: a double payment in an e-commerce app that we maintain. It’s the first time we’ve been told about something like this happening. I fixed it for the customer right away and let them know. Handling the money properly is pretty much the most important thing for this kind of app, so I interrogated the database and, surprisingly, found several more instances of the problem, taking a few different forms. Some payments were even missing data. This was not good.

My first step was to step through the codebase again and look for opportunities to make the payment process more robust. I made a few changes and deployed them. For a little while, things seemed better, but it wasn’t too much longer before we heard about some further occurances of the problem. After some more debugging, I isolated the problem: some transactions with our credit card gateway (SecurePay) were very slow.

The credit card transactions were taking longer than 30 seconds, which is the default timeout period for web requests on Heroku. This resulted in the user seeing their request timeout – but the actual payment might or might not still complete in the background, detached from the user’s browser request. In the meantime, the user might just pay again or leave altogether, creating these inconsistencies in the data.

By this time, we’ve moved from Manila to our new home city, Bacolod. But we haven’t actually moved moved yet. We’re staying in a pension house. It’s basic, but comfortable, and in a good location to start exploring the city. But I have this big technical problem that’s inconveniencing people. So I spend the first couple of days half-thinking about what needs to be done, and the first couple of nights perched on the bed (the only seating in our room) working on code to get a solution ready.

This first attempt at a solution was pushing the payment into a background job, then having the frontend check on the progress of the job before showing the “payment complete” message and order receipt. Pretty straightforward. I’d done this kind of thing plenty of times before. So I put my head down and got to work building a solution using delayed_job, which we already had in place for background jobs. After a couple of nights, I had things ready to test. I tried it and everything worked except the payment. What had happened? The credit card number wasn’t being passed on to the payment gateway. Then it struck me: the credit card number! Naturally, I had ensured my payment model never persisted the full credit card number. We didn’t want to store credit card numbers. This was a problem for delayed_job, because it relied on the persistence of all a record’s information in order for the asynchronous method to work properly. If only I had remembered this first!

This led to a big discussion with my co-workers and Twitter network about ways to make this work. I really didn’t want to persist the credit card number in any way, but could it be done acceptibly, somehow? Perhaps persisting it only momentarily? Maybe we could move to Resque for the background jobs and use an in-memory-only Redis instance for the job persistence? I wasn’t happy with any of these options. I really didn’t want to have to worry about storing credit card numbers. (In my desparation, I briefly looked into what PCI compliance might involve, but that quickly steeled my resolve to avoid it completely.) Back to the drawing board.

My wife has just started her first couple of days at her new workplace. There’s a lot of meeting people and other formalities. I join her for these. I’m still thinking about fixing this problem, but there’s not much time to translate the thought into action. Soon after, we’re taken around the town looking at places to rent. We pick a place and now it’s time to start a final move.

With the delayed_job option no longer possible, I thought about other simple ways to fix the problem. I found this <timeoutValue> option in the SecurePay API docs! Perhaps we could just set that to 30 and all would be good! Turns out it wasn’t so. After some largely unconstructive back and forth with SecurePay’s developer support team, I realised that the timeout value was only intended for batches of transactions, not the individual transactions that we were using.

Meanwhile, I’d written a script that took a day’s transaction data from SecurePay, compared it to the payment records in our app, and reported on any inconsistencies. I ran it each night with bated breath. Many nights, things were OK, and I would feel a massive relief. But my heart was in my mouth everytime I pasted the code into the Heroku console and hit the return key. And at the times when it did reveal problems, I knew I had to then look into things, rectify them, and sheepishly pass on the news to our customer, so they could make things right for their users. Fortunately, our customer was actually very gracious about this. They knew that new software undergoes teething issues, and they were grateful we were staying on top of things for them.

As an aside, one thing that was very helpful for ensuring we didn’t lose any critical payment information was Papertrail, a cloud-based application logging service. I threw a whole bunch of specially marked logger.info messages around the payment events in the app, which would safely store all the request parameters in on Papertrail (after removing the full credit card details), where I could easily find them and review them with a saved search. I highly recommend it. In this case, it ensured we could always recover from any instances of the problem.

But this was just a measure to help us cope while the problem still existed. We still needed a solution. It was clear that the only way forward was to accept that the credit card transactions could take more than 30 seconds, and then find a way to handle it gracefully.

I had an inkling that an asynchronous web server or app framework might help. I didn’t know much about this stuff, so I started madly researching them. Goliath. Rainbows. Cramp. EventMachine and its family of libraries. They’re a whole a different world, and I didn’t really know where to start. I didn’t want to shift the whole app to different underlying tech. Maybe I could extract the payments into an asynchronous mini-app? But that would still have the same response timeout problems, just shifted around a little. Perhaps this approach wasn’t going to help at all.

It’s the weekend now, and we’re in our new place with a minimal set of furniture. Just some bamboo chairs. No bed or table yet. We’ll do some shopping later in the day for the rest of our essentials, but it’s the morning now and I need to keep pushing with this work. I’m perched on the edge of the bamboo chair, my laptop tethered to a delicately placed iPhone, tenuously receiving a 3G signal. The connection was fine when I tested it outside the house, but not so within its concrete walls. We don’t have air conditioning yet. It’s 35 degrees inside and the air is still. I’m sweating and the laptop is hot and I have tab upon tab of slowly-loaded web pages open about a topic I’m not sure will help me and I’m worried about just how on earth I will be able to work in this environment and I’m not getting anywhere and I’m more irritable and tightly wound than I can remember. This is not how things should be.

The Solutions

After some research, an async web server didn’t seem to offer a straightforward answer to my problems. A simpler approach was to leverage a feature of the Heroku Cedar platform: its handling of HTTP streaming responses. If Heroku detects that you’re streaming a response, then its timeout window is extended by 55 seconds every time a byte of data is sent down the line.

Streaming responses are a flagship feature of Rails 3.1, but they are primarily for passing on the start of your page layout while it assembles the rest of the page’s content for rendering. This means the browser can start downloading the site’s JS and CSS assets sooner. This is useful for overall performance, but it wasn’t what I needed. My app was also only on Rails 3.0, and didn’t want to upgrade to a new Rails at the same time as fixing this problem.

Fortunately, some support for manually handling streaming exists in Rails 3.0, via assigning an object to the controller’s response_body. All it needs to do is response to each and accept a block, which it can call with the data to be streamed:

class StreamResponse
  def each(&block)
    # Send some data to the client
    block.call("This is some data")
    
    # And send a bit more. It's a stream, after all
    block.call("This is some more data")
  end
end

class OrdersController < ApplicationController
  def create
    # Do your normal controller things here

    # Then stream the response
    self.response_body = StreamResponse.new
  end
end

This object is sent the each message once as the controller action runs. Inside this method, whatever you output is streamed immediately to the client browser. Here, finally, I could create some custom streaming logic and support these slow payments.

What I did was split the response into four phases:

  • Immediately stream the content for a loading screen
  • Run the slow payment action in a separate Ruby thread
  • Regularly check the thread’s progress, and while it is still running, stream a bite-sized chunk of data to keep the connection alive
  • When the thread completes, stream a final success or failure message

Here’s how the controller looks:

class OrdersController < ApplicationController
  respond_to :html
    
  def create
    @order = Order.new(params[:order])

    if !@order.valid?
      # Don't bother streaming and talking to the payment gateway if we know there are already things missing or invalid
      render 'show' and return
    end

    stream_response = ThreadedOrderPurchaseResponse.new
    stream_response.start_content     = render_to_string('purchasing')
    stream_response.keepalive_content = ' '.html_safe
    stream_response.success_content   = render_to_string('purchasing_complete', :layout => false, :locals => {:success => true})
    stream_response.failure_content   = render_to_string('purchasing_complete', :layout => false, :locals => {:success => false})

    stream_response.slow_process = lambda do
      if @order.save
        if @order.purchaser_email.present?
          Resque.enqueue(DeliverOrderMailerOrderCompletedJob, order.id)
        end

        Thread.current['success'] = true
      else
        Thread.current['success'] = false
      end
    end

    self.response_body = stream_response
  end
end

The logic for handling the threaded payment in the background is in the ThreadedOrderPurchaseResponse class, which is based in part on the template_streaming gem for Rails 2.3. For this particular case, I added a simple Ruby thread:

class ThreadedOrderPurchaseResponse
  attr_accessor :bytes_to_threshold,
                :slow_process,
                :start_content,
                :keepalive_content,
                :success_content,
                :failure_content,

  def initialize
    @bytes_to_threshold = 2048
  end

  def each(&block)
    @response_stream = block

    push(@start_content)
    
    if @payment_thread.blank?
      @payment_thread = create_payment_thread
    end

    # OK, let's sort this out. We're going to let the transaction run for up to 180 seconds.
    # And we only have 190 before unicorn kills us anyway.
    # Everytime we send a bit of data in a streamed response, they give us 55 seconds.
    # So we only only need to check on the thread 3 times.

    90.times do |i|
      if @payment_thread.status.nil?
        # An exception occured in the thread
        push(failure_response)
        @payment_thread.join
        return
      elsif @payment_thread['complete']
        if @payment_thread['success']
          push(@success_content)
        else
          push(@failure_content)
        end
        @payment_thread.join
        return
      else
        push(@keepalive_content)
      end

      sleep 2
    end

    # We only get here if the thread hasn't completed within 180 seconds.
    @payment_thread.join
    push(@failure_content)
  end

  def push(data)
    if @bytes_to_threshold > 0
      @response_stream.call(data + padding(@bytes_to_threshold - data.length))
      @bytes_to_threshold = 0
    else
      @response_stream.call(data)
    end
  end

  private

  def create_payment_thread
    Thread.new do
      @slow_process.call
      Thread.current['complete'] = true
    end
  end

  def padding(length)
    return '' if length <= 0
    content_length = [length - 7, 0].max
    "<!--#{'+'*content_length}-->".html_safe
  end
end

Some things to note here. Firstly, with a streaming response, browsers won’t start rendering your content until you’ve returned a certain number of bytes. So the first time we send data back, we pad it with an HTML comment string to ensure we pass this threshold. 2048 bytes for the threshold is enough for all the browsers to render the initial content.

We also use two thread-local variables (complete and success booleans) to record the status of the operation we’re running inside the thread. Since these can be inspected from outside the thread, the supervising loop in the each method can know when the operation has completed and return the appropriate data to the client browser.

Because we’re using threads here, it’s important to set config.threadsafe! in config/application.rb to ensure things run properly (though I keep it mostly turned off in development mode because having to restart the app for every change is quite slow).

Finally, here’s how the views look. First, we have a partial HTML page for the loading screen:

<% page_title 'Purchase in progress' -%>

<div class="public-page">
  <div class="content">
    <div class="wrapper">
      <div class="purchasing">
        <div class="purchasing-inner">
          <div id="purchasing-progress">
            <div><!-- Needed to ensure streaming DOM forms how we want it -->
            <div id="spinner"></div>
            <h2>Your payment is in progress</h2>
            <p>We&rsquo;re talking to the bank to confirm your payment. This can take up to a minute or two so hang tight!</p>
            <script src="/javascripts/vendor/spin.min.js"></script>
            <script>
              (function() {
                var opts = {
                  lines: 16,
                  length: 12,
                  width: 4,
                  radius: 19,
                  color: '#000',
                  speed: 1.2,
                  trail: 60,
                  shadow: false
                };
                var spinner = new Spinner(opts).spin(document.getElementById('spinner'));
              })();
            </script>
          </div>
          <!-- just inside .purchasing-progress -->

Even though this is not a complete HTML document, browsers still render it as expected, showing the user a nice spinner and loading message. Then, then the operation (hopefully) succeeds, we send the following along the response stream:

              <div id="purchasing-complete">
                <% if success -%>
                  <h2>Payment successful</h2>
                  <p>Thanks! Your payment was successful and your order is complete. <a href="/order">View your purchase receipt</a>.</p>
                <% else -%>
                  <h2>Payment error</h2>
                  <p>Sorry, something went wrong and we couldn&rsquo;t complete your purchase. <a href="/order?errors=true">Please review your details and try your purchase again</a>.</p>
                <% end -%>
                <script>
                  (function() {
                    document.getElementById('purchasing-progress').style.display = 'none';
                    var t = setTimeout(function() {
                      document.getElementById('purchasing-complete').style.visibility = 'visible';
                    }, 500);
                  })();
                </script>
              </div>
              <script type="text/javascript">
                (function() {
                  window.location.href = '/order<%= "#{'?errors=true' unless success}" %>';
                })();
              </script>
              <meta http-equiv="refresh" content="1; url=<%= "#{request.protocol}#{request.server_name}/order#{'?errors=true' unless success}" %>">
            </div><!-- .purchasing-inner -->
          </div>
        </div>
      </div>
    </div><!-- .public-page -->
  </body>
</html>

When the purchase is complete, we want to send the user back to an /order page, where they see either their purchase receipt, or a form for fixing any errors if the purchase failed. We use a number of methods to ensure this happens. Firstly, for the browser with no support for JavaScript, we show a descriptive paragraph with a simple link back to /order. Additionally, we throw a meta refresh tag in there that should send them along to the right place. Next, for browsers supporting JavaScript, we hide the existing loading message and send the user to /order by setting window.location. Lastly, we append ?errors=true to the URL if the purchase failed. This is necessary because the redirect results in a new GET request, and it doesn’t have the existing object around to inspect for its state.

I’m reasearching and building all of this, and I know it will no longer be something I can cram into a couple of big days alone. So I finally fall back into a regular, more sustainable work pattern, from our thankfully now furnished and air-conditioned home. I wake up, make breakfast, start working at 8am, and work solidly through to around 6pm, when I once again check for any erroneous payments that might have occurred during the day. But outside my working hours, I just want the next work day to hurry along so I can get closer to finishing this refactor. Especially know that I’ve found a workable approach, I really want this off my plate. I want to reclaim my life.

Fortunately, a forced respite comes in the form of an impromptu beach holiday in Sipalay, a 5-hour bus trip to the south of our island. There’s a weekend followed by a Philippine public holiday and also our wedding anniversary, so we get a nice break. The laptop only leaves my bag once, for a movie-viewing session. This is much-needed time away for both of us.

The Release

Finally, I thought I was ready to go. I’d built a working solution for gracefully handling payment processing of any duration. I’d also expanded the state machine around orders to include an additional “paying” state that is entered immediately before the transaction with the payment gateway, and left immediately after. This gave me some extra granularity in finding and catching any orders that don’t complete the transaction as expected.

At this point, it was time to do some thorough testing on the staging app on Heroku. I pushed up the code, eagerly looking for confirmation that the end of this problem was in sight. It didn’t come. My streaming responses, well, didn’t stream. Not in the slightest.

I was baffled and frustrated. Before embarking on this line of development, I’d already written an isolated testing app and verified that the streaming responses worked on Heroku. Now that I’d taken the time to integrate this technique into the app at large, and verified it all locally, all I had was the browser waiting until the request was complete before rendering all the content at once.

To further investigate, I introduced an isolated streaming test controller into my app. Something I could harmlessly access and test at /streams/new within the full app. First, I kept most of the logic that I was using for the purchases. Deployed and tested. Didn’t work. Then I started stripping a little out. Deployed and tested. Didn’t work. Finally, I made it as basic as possible. It was effectively the same as my test within the separate isolated app. Surely this would yield the behaviour I wanted. Deployed and tested. Still didn’t work!

Something strange was happening, and it must have been something outside of the app’s codebase. I compared the environment of my test app with the staging app. The staging app had a few Heroku add-ons enabled, so I started disabling them. It turned out that the NewRelic add-on was preventing the streaming responses from working. I dove deeply into its configuration to see if there was a way to keep it around. Perhaps something to make it thread-friendly? Nothing was immediately obvious, and removing it allowed streaming to work again. So I removed it (though later I worked out out to keep it around, and I’m glad to have it back).

By this stage, I was wary of how changes in the Heroku environment could affect the streaming. To ensure everything would work in production, I copied the remaining environmental difference into the staging app: HTTPS support. This broke everything all over again. So again, it was time for more frustration at trying to work around the problem, especially since there didn’t seem to be many options available. I was using hostname-based SSL, the only suitable option for our app, which uses a wildcard SSL certificate for secure connections across multiple subdomains. Luckily, my good colleague Hugh made me aware of Heroku’s SSL Endpoint beta add-on, which works seamlessly with Cedar’s streaming support and fixed the remaining problem with the app in staging. This improved SSL support has just now been released by Heroku.

So there it was. With the app environment problems smoothed out, the streaming refactor applied to both the public and admin areas, and a nice design put in place for both, everything was ready to go. On the 19th of December, we made the changes live. Everything went smoothly, and after six weeks, I could finally breathe a sigh of relief.

The Lessons

In those six weeks (which truly felt like a lot longer), not only did I have to solve one of the most difficult technical problems I’d encountered, but I also moved six thousand kilometers away, started living in a new culture amid a new language, found and set up a house, and did what I could to support my wife in doing all these same things as well as getting started in her new volunteering role. In trying to do all of these at once, I did all of them somewhat badly. And in placing priority on a quick fix to the technical issues, I did especially poorly at everything that happened away from the computer screen, including being the happy husband and travel partner that I wanted to be.

I won’t do this again. What would I do differently if I had the same things happen? First, I would relax. I’d spend more time thinking over the problems while away from the text editor and the Google searches. This would actually allow some room for insight and incision into the problem. There would be less flaying about. I’d be calmer and happier and would think more clearly. I’d also spend more time talking things over with my creative and resourceful teammates. In this case, the problem occurred immediately after moving overseas. While I’d previously spent quite some time successfully working remotely, the extra distance left me feeling unreasonably isolated, and I didn’t rely on my friends and colleagues as much as I should have.

What I wouldn’t have changed was how I communicated with the customer. I was quick to admit a problem, assured them that addressing it was important to me, and then kept them in the loop through the entire debugging and development process. I’d also still have devised interim measures (like the script to check for erroneous payments) to stay on top of any instances of the problem, ensuring my customer could manage their business as well as possible under the circumstances. Even if my final fix took a little longer to come, keeping the customer informed and equipped in this way would have kept them happy.

And happiness, after all, is what’s important. A happy developer will do better work and end the day satisfied. And then be happy in anything else outside of work. And all of this strengthens their ability to contribute positively to the world.

Despite the unhappiness I endured dealing with this problem, I’m happy that I’ve fixed it, moved on from it, and learnt some valuable lessons:

  • It’s important to recognise when a problem requires a serious fix. Building a serious fix requires a real plan, not an ad-hoc one.
  • Creating a real plan requires time, especially away from the computer screen.
  • Architecting serious changes is best done in collaboration with your teammates, even you’re the holder of most of the domain knowledge.
  • Customers can remain satisfied and sympathetic even during times of app instability, as long as they know you’re serious about fixing the problems and are kept involved throughout the process.
  • You need to take care of yourself. You’re not a machine, and you can’t solve problems when you’re feeling like shit.

I hope you find these helpful too.

A Quick Review of the Mac Mini with OS/X Lion compared to Linux

A client just lent me a new Mac Mini with OS/X Lion to play with. I think it’s interesting to compare it with regular PCs running Linux.

Hardware

The Mac Mini is tiny. It’s volume can be compared to that of a laptop. The entire outside apart from the base is made from aluminium which helps dissipate heat, it’s not as effective as copper but a lot better than plastic. The ports on the system are sound input/output, 4*USB, Ethernet, Firewire, Thunderbolt (replacement for Firewire), SDXC, and HDMI. It ships with a HDMI to DVI-D adapter which is convenient if you have an older monitor (or if you have a recent monitor but no HDMI cable as I do).

To open the case you unscrew the bottom, this is much like opening a watch. Also like opening a watch it’s not particularly easy to screw it back on tightly, I will probably return the Mac Mini without managing to completely screw the base in.

The hardware is very stylish and intricately designed, what we expect from Apple. It’s also quiet. In every way it’s a much better system than the workstation I’m using to write this blog post. The difference of course is that this workstation was free and the Mac Mini cost just over $1000 including the RAM upgrade. A Mac Mini could be a decent Linux workstation and if I see one about to be recycled I’ll be sure to grab it!

Installation

The Mac OS comes pre-installed so I didn’t get to do a full installation. When I first booted it up it asked me if I wanted to migrate the configuration from an existing server, I don’t know how well this works as I don’t have a second Mac system but the concept is a good one. Maybe having full support for such a migration process would be a good release goal for a Linux distribution.

After determining that the installation is a fresh one I was asked for a mac.com email address or other form of registration. I skipped this step as I don’t have such an email address, but it could be useful. Red Hat has “Kickstart” to allow configuration of an OS install based on a file from a server (via NFS or HTTP). Debian supports “preseeding” to take OS configuration options from a file at install time [1] and the same option can be used for later stages of OS autoconfiguration.

One thing that would be really useful is to allow the user to enter a URL for configuration data for an individual account or for all accounts, so someone with an account on one workstation could upload the configuration (which would be either encrypted or sanitised to not have secret data) and then download it when first logging in to a new system. I can easily take a tar archive of my home directory to a new system, but people like my parents don’t have the skill to do that.

One of the final stages of system configuration was to identify the keyboard. The system asked me to press the key to the right of the left shift key and then the key to the left of the right shift key and then offered me three choices of keyboard. That was an interesting way of reducing the list of possible keyboards offered to the user and thus preventing the user from selecting one that is grossly incorrect.

Cloud Storage

When first logging in I was asked for an iCloud [2] login. iCloud doesn’t seem like a service that should be trusted, it’s based in the US and has been designed to facilitate access by government agencies. Ubuntu One [3] is a similar service that is run by a more reputable organisation, but the data is still stored by Amazon (a US corporation) which seems like a security risk. Ubuntu One isn’t in Debian (which is strange as Ubuntu is based on Debian) so it was too much effort for me to determine whether it encrypts data in a way that protects the users against US surveillance.

The cost of Ubuntu One storage is $4 per month with music streaming. A better option is to use a self-hosted OwnCloud installation for a private or semi-private cloud [4]. A cheap server from someone like Hetzner (E49 per month for 3TB of RAID-1 storage) [5] is a good option for OwnCloud hosting. A cheap Hetzner server is about $US64 per month (at current conversion rates) which is equivalent to about 16 users of Ubuntu One for music streaming. So if 20 people shared a Hetzner server they could save money when compared to Ubuntu One while also getting a lot more storage. I’ve got about 300G of unused disk space on the Hetzner server that hosts my blog and when the system is migrated to a newer Hetzner server with 3TB disks it will have 2.5TB of unused space, I could store a lot of cloud data in that!

The main features of iCloud and Ubuntu One seem to be distribution of random data files (anything you wish), streaming music to various playing systems, and copying pictures from phones as soon as they are taken. These are all great features but it’s a pity that they don’t appear to support distributed document storage. Apple Pages apparently allows documents to be immediately saved to the cloud. I’d like to be able to save a file with Libre Office at home and then access it from my netbook using the cloud, of course that would require encryption for secret files but that’s not so hard to do. One advantage with such distributed storage is that when combined with offline-IMAP for email it would almost entirely remove the need for backups of the desktop systems I maintain for my relatives. I could have all their pictures and documents go to the cloud and all their email stay on the server so if their desktop PC dies I could just give them a new PC and get it all back from the cloud! OwnCloud supports replication, so if I got two servers I would be covered against a server failure. But I think that for a small server with less than a dozen users it’s probably better to just take some down-time when things go wrong and do regular backups to an array of cheap SATA disks.

App Store

Apple has an “App Store” in the OS. The use of such a store on a desktop OS is a new thing for me. It’s basically the same as the Android Market (Google Play) but on the desktop. I think that there is a real scope for an organisation such as Canonical to provide such a market service for Linux. I think that there is a lot of potential for apps to be sold for less than $10 to a reasonable number of Linux users. A small payment would be inconvenient for the seller if they have to interact with the customer in any way and also inconvenient for the buyer if they are entering all their credit card details into a web site for the sale. But for repeat sales with one company being an intermediary it would be convenient for everyone. A market program for a desktop Linux system could provide a friendly interface to selecting free apps from repositories (for Debian, Ubuntu, Fedora, or other distributions) and also have the same interface used for selecting paid applications.

Conclusion

This isn’t much of a review of Apple OS/X or the Mac Mini. Thinking about ways of implementing the best features of Lion on Linux is a lot more interesting. I admire Apple in the same way that I admire sharks, they are really good at what they do but they don’t care about my best interests any more than a hungry shark cares about me.

Update

I got the currency conversion wrong in the first version of this article. It seems that to save money via a shared Hetzner server instead of Ubuntu One about 20 users would be needed instead of 10. But that’s still not too many and would still give a lot more storage. It would be a little more difficult to arrange though, probably anyone who is seriously into computers knows 10 people who would want to share such a service (including people like their parents who want things to just work and don’t understand what’s happening). But getting 20 people would be more difficult.

Related posts:

  1. Xen and SE Linux – EWeek review of RHEL5 The online magazine EWeek has done a review of RHEL5....
  2. Servers vs Phones Hetzner have recently updated their offerings to include servers with...
  3. Modern Laptops Suck One of the reasons why I’m moving from a laptop...

Twitter Weekly Updates for 2012-05-06

  • "RcvRemotePhysErrors – This indicates a problem ELSEWHERE in the fabric." #UselessErrors #infiniband #
  • .@BBCBreaking usually "making something safe" is lingo for stopping it going off – so they want to stop the Olympics? :-) #
  • Just updated @donna_williams #art gallery with a set of mini-paintings http://t.co/lte1TM11 #autism #
  • .@David_Speers "inappropriate" not good enough Grahame Morris, how about "wrong", "offensive", "stupid" and "incitement"? #
  • So tomorrows @TheTimeTeam is Dunwich, I wonder if @FluffCthulhu and relatives will make a guest appearance? :-) #
  • Just came across http://t.co/oIC7NMfC by @siovene, a sort of @Flickr for amateur astronomers – lovely! #space #astronomy #
  • Yeah, just before 5pm on a Friday is an ideal time to phone up to say you want to deliver a 50kg package to work tonight #
  • Reflashing my Samsung Galaxy Nexus with @Google's #Android 4.0.4 "takju" factory image – rebooting now – fingers crossed! #
  • Phone currently restoring all apps via the saved Google settings, looking good so far. #
  • Looks like #GNOfficialUpdate needs an update itself, doesn't recognise the new @googlenexus "takju" build as official :) #
  • Right, time for an "adb restore" to recover the rest. #
  • Nice to see New Model Army back tweeting after a gap of 2 years! They're at @officialnma (with a new bass player) #
  • It appears @Google hide or remove Google Wallet from takju #Android @googlenexus firmware if you're not in the US.. :-( #
  • Went with @donna_williams to see "Contact, the netball opera" – great fun, first experience of both opera and netball. :) #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-05-06

Twitter Weekly Updates for 2012-05-06

  • "RcvRemotePhysErrors – This indicates a problem ELSEWHERE in the fabric." #UselessErrors #infiniband #
  • .@BBCBreaking usually "making something safe" is lingo for stopping it going off – so they want to stop the Olympics? :-) #
  • Just updated @donna_williams #art gallery with a set of mini-paintings http://t.co/lte1TM11 #autism #
  • .@David_Speers "inappropriate" not good enough Grahame Morris, how about "wrong", "offensive", "stupid" and "incitement"? #
  • So tomorrows @TheTimeTeam is Dunwich, I wonder if @FluffCthulhu and relatives will make a guest appearance? :-) #
  • Just came across http://t.co/oIC7NMfC by @siovene, a sort of @Flickr for amateur astronomers – lovely! #space #astronomy #
  • Yeah, just before 5pm on a Friday is an ideal time to phone up to say you want to deliver a 50kg package to work tonight #
  • Reflashing my Samsung Galaxy Nexus with @Google's #Android 4.0.4 "takju" factory image – rebooting now – fingers crossed! #
  • Phone currently restoring all apps via the saved Google settings, looking good so far. #
  • Looks like #GNOfficialUpdate needs an update itself, doesn't recognise the new @googlenexus "takju" build as official :) #
  • Right, time for an "adb restore" to recover the rest. #
  • Nice to see New Model Army back tweeting after a gap of 2 years! They're at @officialnma (with a new bass player) #
  • It appears @Google hide or remove Google Wallet from takju #Android @googlenexus firmware if you're not in the US.. :-( #
  • Went with @donna_williams to see "Contact, the netball opera" – great fun, first experience of both opera and netball. :) #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-05-06

Twitter Weekly Updates for 2012-05-06

May 05, 2012

Liberty and Mobile Phones

I own two mobile phones at the moment, I use a Samsung Galaxy S running Cyanogenmod [1] (Android 2.3.7) for most things, and I have a Sony Ericsson Xperia X10 running Android 2.1 that I use for taking photos, some occasional Wifi web browsing, and using some applications.

Comparing Android Hardware

The hardware for the Xperia X10 is better than that of the Galaxy S in many ways. It has a slightly higher resolution (480*854 vs 480*800), a significantly better camera (8.1MP with a “flash” vs 5MP without), and a status LED which I find really handy (although some people don’t care about it).

The only benefit of the Galaxy S hardware is that it has 16G of internal storage (of which about 2G can be used for applications) and 512M of RAM while the Xperia X10 has 1G of internal storage and 384M of RAM. These are significant issues, I have had applications run out of RAM on the Xperia X10 and I have been forced to uninstall applications to make space.

Overall I consider the Xperia X10 to be a significantly better piece of hardware as I am willing to trade off some RAM and internal storage to get a better resolution screen and a better camera. The problem is that Sony Ericsson have locked down their phones as much as possible and they don’t even give users the option of making a useful backup – they inspired my post about 5 principles of backups [2].

The fact that the Galaxy S allows installing CyanogenMod which then gives me the liberty to do whatever I want with my phone is a massive feature. It outweighs the hardware benefits of the Sony Ericsson phones over Samsung phones prior to the Galaxy Nexus and Galaxy Note.

For an individual user the ability to control their own hardware is a feature. Such an ability wouldn’t be much use if there wasn’t a community of software developers, so if you buy an Android phone that isn’t supported by CyanogenMod or another free Android distribution then whether it is locked probably won’t matter to you. But for any popular Android phone that’s sold on the mass market it seems that if it’s not locked then it will get a binary distribution of Android in a reasonable amount of time.

Comparing with Apple

It seems that Apple is the benchmark for non-free computing at the moment. The iPhone is locked down and Apple takes steps to re-lock phones that can be rooted – as opposed to the Android vendors who ship phones and then don’t bother to update the firmware for any reason. The Apple app market is more expensive and difficult to enter and if an app isn’t in the market then you have to pay if you want to install it on a small number of development/test phones. This compares to Android where the Google market is cheaper and easier to enter and anyone can distribute an app outside the market and have people use it.

But for an individual this doesn’t necessarily cause any problems. I have friends and clients who use iPhones and are very happy with them. In terms of software development it’s a real benefit to have a large number of systems running the same software. As Apple seems to have higher margins and larger volume than any other phone vendor as well as shipping only one phone at any time (compared to every other phone vendor which seems to ship at least 3 different products for different use cases) they are in a much better economic position to get the software development right. As far as I can tell the hardware and software of the iPhone is of very high quality. The iPad (which has a similar market position) is also a quality product. The fact that the Apple app market is more difficult to enter (both in terms of Apple liking the application and the cost of entry) also has it’s advantages, I get the impression that the general quality of iPhone apps is quite high as opposed to Android where there are a lot of low quality apps and many more fraudulent apps than there should be.

The lack of choice in Apple hardware (one phone and one tablet) is a disadvantage for the user. There is no option for a phone with a slide-out keyboard, a large screen (for the elderly and people with fat fingers), or any of the other features that some Android phones have. The lack of a range of sizes for the iPad is also a disadvantage. But it seems that Apple has produced hardware that is good enough for most users so there aren’t many complaints about a lack of choice.

It seems to me that the biggest disadvantage of the closed Apple ecosystem is for society in general. Anyone who wants to write a mobile app to do something which might be considered controversial would probably think twice about whether to develop for the iPhone/iPad as Apple could remove the app at a whim which would waste all the software development work that was invested in writing the app. Google seem to have much less interest in removing apps from their store and if they do remove an app then with some inconvenience it can be distributed on the web without involving them – so the work won’t be wasted.

How Much Freedom Should a Vendor Provide?

The Apple approach of locking everything down is clearly working for them at the moment. The Samsung approach of taking the Google prescribed code and allowing users to replace it is good for the users and works well. The Sony Ericsson approach of taking the Google code, adding some proprietary code, and then locking the phone down is bad for the users and I think it will be bad for Sony Ericsson. People are more likely to tell others about negative experiences and negative reviews are more likely to be noticed than positive reviews. So while many people are reasonably happy with Sony Ericsson products (until they find themselves unable to restore from a backup) it’s still not a good situation for Sony Ericsson marketing.

It seems that there are benefits to hardware vendors for being really open and for locking their users in properly. But being somewhat open isn’t a good choice, particularly for a vendor that ships poor quality proprietary apps such as the Sony Ericsson ones.

In terms of application distribution Google isn’t as nice as they appear. The Skyhook case revealed that Google will do whatever it takes to prevent apps that compete with Google apps from being installed by default [3]. Google is also trying to make money from DRM sales via Youtube which it denies to rooted phones [4]. Again it seems to me that the best options here are being more open than Google is and being as closed as Apple. Google might gain some useful benefits from applying DRM (even though everyone with technical knowledge knows that it doesn’t work) but the Skyhook shenanigans have got to be costing Google more than it’s worth.

How to make Android devices more Free

The F-droid market is an alternative to the Google App market which only has free software [5]. On it’s web site there are links to download the source for the applications, including the source and binaries for old versions. In the Google App market if an upgrade breaks your system then you just lose, with F-droid you can revert to the old version.

A self-hosted OwnCloud installation for a private or semi-private cloud [6] can be used as an alternative to the Google Music store as well as for hosting any other data that you want to store online.

The Open Street Map for Android (Osmand) project provides an alternative to the Google Map service [7]. Osmand allows you to download all the vector data for the regions you will ever visit so it can run without Internet access. But it doesn’t have the ability to search for businesses and the search for an address functionality is clunky and doesn’t accept plain text, which among other things precludes pasting data that’s copied from email or SMS. While Osmand provides some important features that Google Maps will probably never provide, it doesn’t provide some of the most used features of Google Maps so uninstalling Google Maps isn’t a good option at the moment.

The K9mail project provides a nice IMAP client for Android [8]. Use K9 with a mail server that you run and you won’t need to use Gmail.

There are alternatives to all the Google applications. It seems that apart from the lack of commercial data and search ability in Osmand an Android device that is used for many serious purposes wouldn’t lack much if it had no Google apps.

Google seems to be going too far in controlling Android. Escaping from their control and helping others to do the same seems to be good for society and good for the users who don’t need apps which are only available in proprietary form.

Related posts:

  1. Dual SIM Phones vs Amaysim vs Contract for Mobile Phones Currently Dick Smith is offering two dual-SIM mobile phones for...
  2. My Ideal Mobile Phone Based on my experience testing the IBM Seer software on...
  3. Old Mobile Phones as Toys In the past I have had parents ask for advice...
 
I’m pleased to note that you can lose your Internet connection partway through upgrading Oneiric Ocelot Kubuntu to Precise Pangolin at the point where the GUI doesn’t actually work, then reboot it into text mode (RunLevel 1), bring up a network connection by hand, apt-get upgrade then hand-install the numerous things it held back on... (dpkg does an interactive dance routine a few times, catching up on half-done things) & it all works!



Better than that, the final recompile of dvdauthor is not suffering the lack-of-menu-backgrounds in the final product when invoked via devede which several previous versions did.
 
I am in the habit of collecting knowledge. In walk-a-mile see-a-mile mode, I can just about glimpse — in the very farthest distance — hints of the horizons of mortal knowledge.



In short words, we don’t know nuffin’! :-)

Interesting links for May 5th 2012

  1. A Relevant Tale: How Google Killed Inktomi – Overview on how early search engine Inktomi was knocked out by Google. The Hacker News discussion is quite good and includes a link to a video talk by Inktomi’s co-founder.
  2. Reddit interview  – IAmA Part Time Hooker in New ZealandRaw Interview or Summary of Q/A . Prostitution is legal in New Zealand so some people in other countries find it interesting how it works. NSFW obviously.
  3. Are Shakespeare’s Plays Encoded within Pi? – YouTube . The full text is in the text section of the page.
  4. Gather – Auckland BarCamp has decided to rebrand itself as “Gather”. Not sure of the point especially since they don’t even own gather.co.nz (or gathernz.com or something). Anyway it’s a pretty good unconference that happens each year. This year it is one June 30th

May 04, 2012

Kiuchi Brewery’s Espresso Stout

My first beer from Japan! This pours nice and thick. This Espresso Stout is black like my heart and is thick like the phrase “Guinness is a meal” would lead you to believe (rather than the sad reality that is Guinness these days).

While I can’t really detect the espresso in the smell of this one (although once it warmed up a bit I could), I can in the taste. Halfway through a mouthful you’re certainly going “mmm… espresso-y”.

This was quite enjoyable. I’m not sure if I could drink more than one (I probably couldn’t), and it doesn’t match the whatever it was I had through a coffee bean filled Randy at Mountain Goat a while ago, but it’s certainly nice.

Yes folks, there can be good beer come out of Japan.

image

brendanscott

Australian Government and Open Data: They just don’t get it

Having a look through the AusGOAL site – you know, AusGOAL, the Australian Governments Open Access and Licensing Framework’-  to see what Australian Government is doing in the area.  Did you know that they have a video explaining open data and why it’s so important?

The video is here:

Guess what?

It doesn’t comply with their own guidelines.  When I go to that page I get a black box where the video should be, along with a message:

This video can’t be played with your current setup.
Please switch to a browser that provides native H.264 support or install Adobe Flash Player

Errr… here’s a video telling you how great open formats are but the video is in a closed format?? It’s not at all comforting to know that the very people charged with enabling open formats are using a closed format for their videos. There’s no reason H.264 video can’t be transcoded into an open format and offered (even offered as an alternative).

Knock me down with a feather!

 



SLUG Monthly report - April 2012

1. Continued community support and discussion available through SLUG mailing list.

2. Meeting held on last Friday of month including key talk and five lightning talks.

3. Attendance comprised 28 persons

For subcommittee: 

May 03, 2012

Barons Black Wattle

In which I attempt to make beer photos more interesting by hipster-izing them as much as humanly possible without using Instagram.

This is Barons Black Wattle Original Ale. As far as anyone can work out, there isn’t much of this left. The Barons Brewing website is no more and it seems the company went away after not sticking to their core business – which was brewing beer.

It’s got Wattle in the name as the beer has roasted wattle seeds. The Wattle is an Australian native, that is – to me, this beer is distinctly Australian. It’s not a flavour you’d come up with elsewhere. While it is not my most favourite beer of all time, being something that you simply couldn’t really come up with anywhere else, I hold it in a special place.

Oh, and yes it is true that I’m attempting to bribe people at work (Percona) with offers of me sending them a bottle.

When we heard that the brewery was no more, we stockpiled. If you find any in a store, grab it – you probably won’t get another chance to try this brew.

It’s 5.8%, with a rich amber colour and a good rounding of malt flavours – you can certainly taste the Wattle and that’s what makes it distinctive. I’ve never found this a session beer, but I do enjoy a few of them.

image

Acoustiblok/Thermablok

Acoustiblok is an interesting product for blocking sound, it works by dissipating sound energy through friction within the sound barrier materiel [1]. They sell it in varieties that are designed for use within walls and for use as fences. As it isn’t solid it won’t reflect sound so it can be used to line the walls to stop sound being reflected back at you. It’s design is based on NASA research.

The web site claims that a 3mm sheet of Acoustiblok gives a greater noise reduction than 12 inches (30.7cm) of poured concrete. I am a little dubious about that claim as I’ve read a report of someone using three layers of Acoustiblok to make a quiet room for recording music (and to be used as a play-room for an Autistic child). I find it difficult to imagine someone needing a meter of concrete to stop any sort of noise that they might encounter in a residential area so the fact that someone needed three layers of Acoustiblok is an indication that it might not be quite as good as they claim (although there is the possibility that Acoustiblok was badly installed). I wonder whether the claims about concrete concern particular frequencies. The technical specifications and product comparisons page [2] shows that Acoustiblok is least effective at 130Hz where it only reduces noise by 12dB and that it’s effectiveness increases to 38dB at 5KHz. So perhaps a concrete wall to stop low frequencies and Acoustiblok to stop high frequencies would be the best solution.

The Australian distributor for Acoustiblok is based in Brisbane [3].

The same company also sells Thermablok [4] which is the first aerogel based insulation that I’ve seen being advertised for commercial sale. I guess that it must be rather expensive as they are mostly advertising it for use as thin strips to cover stud faces (steel studs conduct heat well and can cause a lot of heat loss). A note in their FAQ says that it’s available in rolls for insulating entire walls or floors. The FAQ also indicates that they sell samples suitable for science classes. They are also apparently looking for retailers, it would be nice if someone wanted to sell this in Australia.

Related posts:

  1. Noise Canceling Headphones My patience with the noise of airlines has run out,...
  2. Testing Noise Canceling Headphones This evening I tested some Noise Canceling Headphones (as described...
  3. I Bought the Bose QC-15 I bought the Bose QC15 noise canceling headphones for my...

May 02, 2012

Harviestoun Old Engine Oil Porter

This beer heralds from a craft brewery in Scotland. At 6% and with good strong flavours, it’s strong all around. A good solid porter with (again, as the bottle says) notes of chocolate and coffee (both in smell and taste) and a bittersweet aftertaste that is just perfect on a cold evening like this.

image

offlineimap syncing

Just finishing up an offlineimap(1) sync to a new laptop. 5 years of email has taken 2.5 days to sync. And that's over a fast network. I'm thinking twice about syncing the larger corpus of older mail.

Upgrading to Precise

The latest release of Ubuntu, version 12.04 aka Precise, has a lot of updates we’ve been waiting on for a while — GNOME 3.4, Haskell 7.4.1, and a huge stack of bugfixes. On the desktop side, quite a number of Linux kernel vs X video modes vs suspend glitches have gone away. That’s fantastic. During most of Oneiric, my laptop was freezing and needing a hard reset at least once a day. Tedious. So I’m quite pleased to report that running Precise, Linux 3.2, gdm, and GNOME 3.4, things are vastly more stable.

Getting upgraded to Precise, however, has not been a pleasant experience.

First we’ve had unattended-upgrades overwriting any configuration stating “no automatic upgrades”. The number of non-technical friends who were set to “security updates only” calling in wondering why a “big upgrade” happened and now their computers don’t work has been staggering. Needless to say we nuked unattended-upgrades from all of our systems a hurry, but for those people it was already too late.

Several desktop upgrades failed half-way through because dpkg suddenly had unresolved symbol errors. Fortunately I was able to work out the missing library binary and manually copy it in from another machine, which was enough to get package system working. Hardly auspicious.

Server side was fraught with difficulty. You cannot yet upgrade from Lucid to Precise. It breaks horribly.

E: Could not perform immediate configuration on 'python-minimal'. Please
see man 5 apt.conf under APT::Immediate-Configure for details. (2)

Brutal. I tried working around it on one system by manually using dpkg, but that just led me into recursive dependency hell:

# cd /var/cache/apt/archvies
# dpkg -r libc6-i686
# dpkg -i libc6_2.15-0ubuntu10_i386.deb
# dpkg -i libc-bin_2.15-0ubuntu10_i386.deb
# dpkg -i multiarch-support_2.15-0ubuntu10_i386.deb
# dpkg -i xz-utils_5.1.1alpha+20110809-3_i386.deb
# dpkg -i liblzma5_5.1.1alpha+20110809-3_i386.deb
# dpkg -i dpkg_1.16.1.2ubuntu7_i386.deb
# apt-get dist-upgrade

Huh. That actually worked on one system. But not on another. Still slammed into the python-minimal failure. For that machine I couldn’t mess around, so I had to give up and did a re-install from scratch. That’s not always feasible and certainly isn’t desirable; if I wanted to be blowing systems away all the time and re-installing them I’d be running Red Hat.

Anyway, I then located this bug about being unable to upgrade (what the hell kind of QA did these people do before “releasing”?) where, very helpfully, Stefano Rivera suggested a magic incantation that gets you past this:

# apt-get install -o APT::Immediate-Configure=false -f apt python-minimal
# apt-get dist-upgrade

(I had tried something very close to this, but didn’t think of doing both apt and python-minimal. Also, it hadn’t occurred to me to use -f. Ahh. For some reason one always sees apt-get -f install not apt-get -f install whatever-package-name).

Ta-da.

AfC

Kernel Security Talk at LinuxCon Japan

Just to let folk know — I’ll be giving a talk on the state of Linux kernel security development at LinuxCon Japan in Yokohama on June 8th. From the abstract:

In this talk, we’ll examine the current state of the Linux kernel security subsystem. Starting with a brief overview of existing features, we’ll discuss recent developments, current efforts and future directions. We’ll also discuss the evolving threat landscape, and the increasing need for mobile and cloud security. This will be a high-level technical discussion aimed at IT professionals. A good general knowledge of operating system and computer security concepts will be advantageous.

I’ll also likely be in Tokyo briefly — if any kernel security development folk there want to meet up, let me know.

May 01, 2012

WordCamp Sydney April Report

Things are well and truly underway, ticket sales are happening, would like them to be happening faster, but isn't that always the way it goes?

We've had a number of great speaker applications and are stoked to have Kate Carruthers and Joe Ortenzi on the roster great players in the Sydney Web/Social Media Scene.

The website is up and getting reasonable traffic, if your'e interested you can read more at http://2012.sydney.wordcamp.org

We have a few issues we're working through with WordCamp central - mostly around ticket price and sponsorship. Difficult to strike a balance between what's fair and reasonable and keeping that all within WordCamp's open source ideals) I've a call in with the WC organiser Andrea sometime today and will be able to report more when that's over, if of course, those issues are of interest.

Next big push is for getting sponsors, which has been difficult to do given the above issues, hopefully the conversation with Andrea will make it possible for us to push more down that path.

We're keen for international speakers and as such are conversing with a couple of other WordPressers overseas with a view to bringing someone down under. This will largely depend on budget.

Think that's about it for now...

Dee Teal

For subcommittee: 

Murray’s Punch and Judy’s Ale

On the back it describes itself as a “New World Bitter” and the word bitter is certainly true – it’s not a floral hoppy flavour but rather a bitter that tastes like a bitter should. I could drink a few of these, although more than that could get overwhelming. Bottle conditioned, only 3.9% and quite pleasant.

image

BarCampMelbourne 2012: Report: April 2012

# We've almost reached the level of sponsorship required to run BCM with a $0 ticket price

# We sold 31 of 110 tickets in the first week of opening regos

# Venue has been secured

# Still need: a sponsor or two, finalise catering and facilities

For subcommittee: 

Testing a FDMDV Modem

A key use for Codec 2 is digital voice over HF and VHF radio. A few months ago I figured we needed to get Codec 2 on the air. With a PC, codec and modem software, two sounds cards, and a Single Sideband (SSB) radio it is possible to send and receive Digital Voice (DV) signals over HF radio.

This requires a HF modem optimised for digital speech, in particular fast sync, no multi-second training sequences, the ability to recover quickly after a fade, and no automatic re-transmit of “bad” packets. FDMDV was a working system for HF Digital Voice from a few years ago, so seemed like a good starting point. It embodies a lot of experience from Digital Voice pioneers like Mel Whitten.

FDMDV stands for Frequency Division Multiplexed Digital Voice. A FDM modem is a basically a bunch of slow modems running in parallel. For example FDMDV has 14 carriers spaced 75 Hz apart, each running at 50 symbols/second. Due to multipath problems on HF this approach works better than one carrier running at 14×50 = 700 symbols/second. On each symbol is encoded two bits using differential QPSK, so the bit rate is 1400 bit/s.

A few months ago I started experimenting with GNU Octave simulations of parts of the FDMDV modem. One thing led to another and I ended up writing an open source version of the FDMDV modem, based on the FDMDV spec.

I am in the final stages of the C version of that modem, currently writing command line demo programs. I am not sure what the “best” HF DV system would look like (Codec/FEC/protocol/modem) but I feel the best way to find out is build something and iterate on it. Rather than concentrating on the Codec alone I wanted to get some real world HF DV experience to tune and evolve the system as a whole.

The cool thing about open source is it attracts the best in the field. I have been in regular contact with HF modem gurus like Peter Martinez G3PLX (PSK31), and Rick Muething KN6KB (WINMOR) who have been very helpful with suggestions and support as I re-implemented the FDMDV modem. Rick also has some great ideas for more advanced modulation schemes (trellis coded PSK) that would be nice to try later. I have also had some great help from Bill Cowley, who has 25 years of PSK modem experience.

Testing the Modem

After developing the modem algorithms for about two months using GNU Octave I was ready to test over a real HF channel. So a few days ago I sent a wave file of the modem signal to Mel and Tony (K2MO). They kindly played the tones over a 925 mile HF channel and sent me a recording of the received signal.

I ran the files through my FDMDV modem code. On the first pass the scatter diagram was a mess and the Bit Error Rate (BER) was about 10% – suspiciously high.

Then I noticed the timing offset was changing very quickly, as you can see in the plot below:

The demod estimates the best time to sample the received symbols. This is known as the “timing offset”. In the real world the sample clocks used at the transmitter and receiver tend to be a little different, for example 8000 and 8001Hz. In our case the sample clocks are in the sound device hardware used to play and record the modem signals. So we expect the timing offset to drift a little. I had been simulating just such problems during the modem development, for example testing clock differences of up to 2000 ppm (16Hz at an 8000 Hz sample rate).

Now the demod code keeps an eye on the drift in the timing estimate, and reshuffles buffers every now and again to keep them from overflowing. Hence the saw-tooth effect.

If we count how many “teeth per second” in the saw-tooth, we can estimate the difference in the transmit and receive sample clock. I estimated about 2.5, of 40 samples each. So in every second that’s 2.5×40 = 100 samples, or a 100Hz difference, or 12500ppm! It’s like the PC playing the signal was at 8000Hz and the sample rate of the PC receiving the signal was at 8100Hz.

I re-sampled the signal to correct the large sample clock offset using Sox:



sox -r 8100 -s -2 for_david.raw -s -2 for_david_8000hz.raw rate -h 8000

and the results were perfect – 0 bit errors except for when there was SSB interference across the signal! This was very exciting for me – the first verification that my modem actually worked over real HF channels.

Turns out some sound cards can’t accurately sample at 8000Hz. This was something I had been warned of by my HF modem brains trust. The solution is to use the 48000Hz sound card rate, which most soundcards seem to be better at.

FDM Modem in Action

Here are samples of the first 5 seconds of the transmit and receive modem signal. Now look at the spectrogram of the received signal:

Time is along the x axis, frequency along the y axis. The “hotter” the colour, the stronger the signal. Our FDM signal is the parallel red lines between 600 and 1700Hz. Above the modem signal is some analog SSB. You can hear this as the high frequency “Donald Duck” sound in the received signal. Now around 2.5 and 3.3 seconds there are strong bursts of SSB right on top of our signal, in the 0 to 1100 Hz range.

So how does our modem do? The “Bit errors for test frames” and “Test Frame Sync” plots below tells the story:

Look at the centre plot, it is a measure of bit errors for each test frame received by the demod. Between 2.5 and 3.5 seconds you can see several error bursts. However the demod recovered quickly after the SSB interference. The BPSK sync and test Frame sync plots are unbroken, indicating our demo didn’t “lose it” during the interfering burst. If this was Codec (digital voice) data we would hear some degraded speech, but the system would soldier on between interfering analog SSB bursts. Just what we want.

This sample also shows some of the accumulated wisdom that went into the FDMDV system design. It is a narrow signal (just 1100Hz), so less sensitive to interference to adjacent users on the busy HF bands compared to a system using a full SSB bandwidth of say 2400Hz. Narrow band means we can pack more energy into fewer carriers. The signal to noise ratio is relatively high, and the BER due to gaussian type channel noise (AWGN) practically zero. Rather bit errors come from adjacent users and mutipath fading effects (the latter not illustrated here).

Next steps are to integrate the Codec and Modem into an easy to use GUI program for Windows and Linux. This will help us obtain some real world experience which we can use to tune and further develop the entire system.

April 30, 2012

The Royal Caribbean Official Android app

I’ve just played with the official Android app [1] for the Royal Carribean cruise line [2]. The cruise line is apparently great (I’ve never been on one of their ships but the reviews are good) but the Android app isn’t.

Net Access

The most obvious and significant problem with this app is that it’s entirely useless without net access. All data of note comes from the Internet which means that the program is useless in any location where Internet access is unavailable (or unreasonably expensive). They wrote an app about cruising that can’t be used on a cruise ship! Did they even think about what they were doing?

The correct thing to do when writing such an application is to have all basic data about all ships included in the app. This means that when they change the deck plan of a ship they need to release a new version of the app and have people download it. Having done a lot of software development I understand that forcing software updates (even updates to included data files) involves some effort and expense. But when they spend $20,000,000 to update a ship (which is about the minimum that is spent for a major ship in dry-dock according to TV documentaries I’ve watched) it seems quite reasonable to budget $10,000 to release new software. Also one benefit of updating the software is that it can promote the changes, after spending tens of millions of dollars improving a ship they probably want to promote that to customers and pushing a new app update with adverts for the improved ship seems like a good way of doing that.

There is some data that can’t reasonably be included in the app due to size constraints with photos of ships being the most notable example. The solution to this is to provide an option for the user to cache the data that interests them. For example if I was meeting some people to discuss the possibility of a group cruise on a RCL ship then I could download all the pictures of that ship on my home Wifi network and then have them all available with no delay or 3G costs.

Also the app seems to hang if net access is temporarily interrupted. As phones are expected to have unreliable net access this is also a significant flaw.

Maps

The maps of the ships are comprised of a series of pictures which each show one deck. In addition to being downloaded (not cached or included in the app) they aren’t scalable (they should be SVG or at least allow zooming the high resolution pictures) and they don’t allow a 3D view. The paper maps used to promote cruises (including RCI cruises) and which are given to all passengers on Princess cruises (I’m not sure about other lines) show a side cutaway view of the ship which is handy for working out which things are near where you are. It seems that an ideal cruise ship mapping program would have some sort of 3D component, maybe X3D.

My experience is that a two night cruise isn’t long enough to become familiar with one of the smaller cruise ships. Using a map is essential and a smart phone is a good way of managing such a map as typical 2D paper maps just aren’t good enough for such a large and complex structure.

Photos

One of the significant things that is wrong with the app is a lack of care in displaying the photos of ships. They display three pictures of the Allure of the Seas (one of the two newest, biggest, and most luxurious ships in their fleet), but one of those three photos is actually of the Oasis of the Seas. The fact that the two ships are almost identical is no excuse, there is a principle at stake! Also only having three pictures is pretty poor, there is no way that less than 50 pictures could do justice to such a big ship!

A Google search for the words cruise and photos turns up many sites with pictures of cruise ships and it’s not particularly difficult to find pictures of any particular ship. Photographs by customers are often of high quality as some of the better DSLR cameras are in the same price range as some of the cheaper tickets for cruise ships. Probably the best thing that RCI could do is to run a contest and allow their customers to enter photos and vote towards the winning entries. That would get them photos that aren’t as sterile as the official photos and which include the things that are of most interest to customers.

Finally in terms of caching, pictures are the most easily cached source of data and as phones get higher resolution they keep getting bigger. The storage space for a modern phone is equivalent to the entire 3G download quota for about a year on an affordable Australian 3G plan. When dealing with photos downloaded from the net the default should be to cache everything.

Navigation

Navigating a smart-phone app is a lot more difficult than navigating the same data on a desktop system (which would be in a web browser). Users can compensate for some deficiencies with web site organisation by using a large monitor and having several web browser windows with multiple tabs. But with a phone it should be possible to switch between things quickly.

The main menu has a “View Our Ships” option (which allows viewing deck plans and pictures) and a “View Our Staterooms” option which offers a list of ships and then describes the state rooms available for each one. This means that you can’t see all the information about a ship in one place and even worse you can’t easily compare ships. As it seems likely that people will want to use this app for selecting a cruise it should be possible to select a few ships that are of interest and then quickly flip between them. For example the Rhapsody of the Seas and the Voyager of the Seas are cruising in my part of the world so it would be nice if I could tell the app to compare those ships and then allow me to view a page about one ship and then flick to the equivalent page about the other ship.

Another notable problem is that the ships are listed in alphabetical order. The sensible thing to do is to list them by class going from biggest to smallest.

Lessons to be Learned

These problems aren’t specific to the RCI app, many other Android apps have the same flaws. For example the Google Play market app doesn’t cache the icons of the installed apps so every time I want to see a list of installed apps it goes slow and wastes some of my bandwidth. Doing something wrong in the same way as Google isn’t necessarily a great mistake, although using the Google Play market on a cruise ship is probably very uncommon.

Probably the biggest problem is a lack of testing. They should have sent the developers on a cruise as a live test. Every cruise ship has a sales desk for booking future cruises so it wouldn’t be difficult to have a dozen Android phones at the sales desk to see how real customers who really want to book a cruise find it. I presume that even if net access was available then such a test would fail dismally. If a 3D display of a ship combined with all the data management capabilities of a modern smart phone (which is a lot more powerful than the desktop systems I used prior to 2000) can’t at least be a useful supplement to a stack of paper brochures then it’s probably a failure.

I think that the RCI app is an example of how to make an Android app which doesn’t fall into the more common failings (such as being a quick and dirty port from iOS) but yet still isn’t useful to customers. I recommend that people who develop apps which have an objective of imparting information to users try it out as an example of what not to do. Try a few basic tasks like comparing the three biggest classes of RCI ships in terms of features, after failing to do that with the app you can then use Wikipedia to get the result. But don’t use the Wikipedia client apps, use a tabbed browser such as Opera Mini.

Related posts:

  1. An Introduction to Android I gave a brief introductory talk about Android at this...
  2. Choosing an Android Phone My phone contract ends in a few months, so I’m...
  3. Galaxy S vs Xperia X10 and Android Network Access Galaxy S Review I’ve just been given an indefinite loan...

Waiting…

If you’ve ever had the misfortune of breaking something that is part of yourself, you’ve probably spent a bunch of times in waiting rooms. Even though the waiting is annoying, when something isn’t going to immediately kill you, you can’t really get too frustrated at having to wait. Well, at least that’s my theory.

I’ve learned the lesson of bringing a tablet or laptop along for the up to an hour wait after the scheduled start of an appointment, although actually waiting in the consult room itself is a new one.

The exposed USB Ports to a computer that can access my medical records does raise a concern. I feel it sad that I now just assume that anybody who does want to read them could. I’ve started to think of possible ways to get anonymous health care. The doctor-patient confidentiality is something that is indispensable to a good health care system.

But here I am, waiting. Nothing interesting to note in this room with no windows (except that window into my medical records)

image

PyCon Australia 2012's Call for Proposals closes this Friday

This week marks your last opportunity to submit proposals to present at PyCon Australia 2012, the national conference for the Python programming community, to be held on August 18 and 19 in Hobart, Tasmania.

The deadline for proposal submission is Friday May 4, 2012, and more information can be found at http://pycon-au.org/cfp

PyCon Australia is an excellent opportunity to share experience and knowledge with like-minded Python developers from all walks of life -- we attract professional developers from industry, government, science and education, along with enthusiast and student developers. Presentations can be targeted at all skill levels, on any topic related to Python programming.

If you're a first-time developer, don't be afraid to submit a presentation -- we're a community-driven conference, and we have a focus on building the next generation of Python programmers.

Don't know what to present about? PyCon Australia co-organiser, Christopher Neugebauer recently listed the results of PyCon Australia 2012's Call for Topics on his blog; we'd love it if you used these suggestions to help shape your submission.

We can't wait to see your proposals.

About PyCon Australia

PyCon Australia is the national conference for the Python Programming Community. The third PyCon Australia will be held on August 18 and 19, 2012 in Hobart, Tasmania, bringing together professional, student and enthusiast developers with a love for developing with Python. PyCon Australia informs the country’s Python developers with presentations, tutorials and panel sessions by experts and core developers of Python, as well as the libraries and frameworks that they rely on.

To find out more about PyCon Australia 2012, visit our website at http://pycon-au.org or e-mail us at contact@pycon-au.org.

PyCon Australia is presented by Linux Australia (www.linux.org.au) and acknowledges the support of our Gold sponsor, Google Australia (www.google.com.au); our Event partner, Secret Lab; and our Silver sponsors, the Python Software Foundation, Anchor Systems, Red Hat, ekit, and CSIRO.

More photos

I’ve decided to try and take more photos and publish more of them. This means I have to look around for opportunities, including capturing some daily life.

This morning I’m off for hopefully my last appointment at the Alfred after a bike accident about six or seven weeks ago (injury photos posted previously). So, it means taking the train as driving with a brace on my arm doesn’t excite me.

The train is late, the 9:08 in the other direction is even later (twenty minutes). I just missed the previous one, so there I am looking at the mind the gap paint.

image

April 29, 2012

Innis & Gunn Original

This was a really nice beer. Sweet with toffee and vanilla (a the bottle says) and you could even smell the oak

. This one was recommended by PatG, and I finally found it at Acland Cellars.

image

I have their rum cask one and a special Canada Day brew in the fridge.

 
Everything that was here is gone.



I could explain, but I'm not going to.

Twitter Weekly Updates for 2012-04-29

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-04-29

Twitter Weekly Updates for 2012-04-29

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-04-29

Twitter Weekly Updates for 2012-04-29

  • Turns out the Tea Party have a bunch of txt for pre-sale, so I'm in ! Am going with teh awesome @alexmyoung :) http://t.co/NRp9JbIE YAY! #
  • Gar, no @blackdogride for me, had to pull out due to stupid circumstances. Was *really* looking fwd to it! Will make up for it next weekend! #
  • YES! Will be logging on tomorrow to snag txts to Tea Party, who's with me?!? (Sydney Show) http://t.co/29Faqag9 #
  • Getting ready for @blackdogride Canberra! Very excited :) It's a beautiful day for some motorbike riding! #needcaffeine #
  • Just to ruin your afternoon productivity http://t.co/oxqi3ZYP "Dance magic dance!" #
  • Reading Lawrence Review of Australian #geospatial capability & gov response. Interesting stuff! Vital for #gov2au http://t.co/v0WNJLF3 #
  • Was there anything to it? RT @GeordieGuy: Meeting derailed while security guy goes on a five minute incomprehensible rant. ????? #
  • Walking past a closing restaurant which has the Labyrinth soundtrack on. So awesome. #
  • Hilarious. "President of the United Sates (sic) Barack Obama & Campbell High Students". No one else? OurCity mag :) http://t.co/uTK7hkr2 #
  • Have officially shifted from NSW @blackdogride to ACT. Sorry to NSW friends, but excited about ACT ride! :) http://t.co/dn2XpFo1 #
  • .@cgiffard maybe… next weekend, not this one? Who else would be up for a movie night to watch The Raid? :) #
  • Tasking some time out from an intense day! Some Shank 2 to chill :) Such a fun but simple game! #
  • Have been flat out all day today, between contract work and #govhack #govcamp preparation. Both launching Monday! :) #gov2au Very excited! #
  • Thanks all, mystery solved :) @timClicks: @nzfi @piawaugh feijoa, yum! see http://t.co/Rk0siHD1 #
  • It kills me to say but I may have to switch to ACT @blackdogride. Too many deadlines pressing in :/ @hollingsworth @moldor @MrsMoldor @Rog42 #
  • Does anyone know what this is? Fruit (?) tree where I live http://t.co/Xiz9W0qT #
  • Really looking toward to @blackdogride this weekend, just me, the open road, and a few hundred other bikes :) #shortholidayforagoodcause #
  • I know lots of you will know this, but handy way to get RSS from Twitter http://t.co/ZseB9Scw Just switch my username with yours :) #
  • Hey guys, sry I've been flat out. @moldor could you dm me phone number to coord meet spots. I've a long way to come beforehand :) /cc @Rog42 #
  • YES! RT :) @kelisha: @piawaugh damn, now I'm going to have to watch it!! LOL #
  • Yes! RT @peenydeeny: @piawaugh the start of Wayne's World , with Bohemian Rhapsody is one of the best happytime moments in cinema :-) #
  • Totally! :) RT @timl: There is so much awesome & references I'd not noticed before! Like this part, 2nd Mission Impossible reference so far! #
  • MWAH HAHAHA! Watching Wayne's World for the first time in years, had totally forgotten the Terminator cameo. "Have you seen this boy?" LOL! #

Twitter Weekly Updates for 2012-04-29

  • Turns out the Tea Party have a bunch of txt for pre-sale, so I'm in ! Am going with teh awesome @alexmyoung :) http://t.co/NRp9JbIE YAY! #
  • Gar, no @blackdogride for me, had to pull out due to stupid circumstances. Was *really* looking fwd to it! Will make up for it next weekend! #
  • YES! Will be logging on tomorrow to snag txts to Tea Party, who's with me?!? (Sydney Show) http://t.co/29Faqag9 #
  • Getting ready for @blackdogride Canberra! Very excited :) It's a beautiful day for some motorbike riding! #needcaffeine #
  • Just to ruin your afternoon productivity http://t.co/oxqi3ZYP "Dance magic dance!" #
  • Reading Lawrence Review of Australian #geospatial capability & gov response. Interesting stuff! Vital for #gov2au http://t.co/v0WNJLF3 #
  • Was there anything to it? RT @GeordieGuy: Meeting derailed while security guy goes on a five minute incomprehensible rant. ????? #
  • Walking past a closing restaurant which has the Labyrinth soundtrack on. So awesome. #
  • Hilarious. "President of the United Sates (sic) Barack Obama & Campbell High Students". No one else? OurCity mag :) http://t.co/uTK7hkr2 #
  • Have officially shifted from NSW @blackdogride to ACT. Sorry to NSW friends, but excited about ACT ride! :) http://t.co/dn2XpFo1 #
  • .@cgiffard maybe… next weekend, not this one? Who else would be up for a movie night to watch The Raid? :) #
  • Tasking some time out from an intense day! Some Shank 2 to chill :) Such a fun but simple game! #
  • Have been flat out all day today, between contract work and #govhack #govcamp preparation. Both launching Monday! :) #gov2au Very excited! #
  • Thanks all, mystery solved :) @timClicks: @nzfi @piawaugh feijoa, yum! see http://t.co/Rk0siHD1 #
  • It kills me to say but I may have to switch to ACT @blackdogride. Too many deadlines pressing in :/ @hollingsworth @moldor @MrsMoldor @Rog42 #
  • Does anyone know what this is? Fruit (?) tree where I live http://t.co/Xiz9W0qT #
  • Really looking toward to @blackdogride this weekend, just me, the open road, and a few hundred other bikes :) #shortholidayforagoodcause #
  • I know lots of you will know this, but handy way to get RSS from Twitter http://t.co/ZseB9Scw Just switch my username with yours :) #
  • Hey guys, sry I've been flat out. @moldor could you dm me phone number to coord meet spots. I've a long way to come beforehand :) /cc @Rog42 #
  • YES! RT :) @kelisha: @piawaugh damn, now I'm going to have to watch it!! LOL #
  • Yes! RT @peenydeeny: @piawaugh the start of Wayne's World , with Bohemian Rhapsody is one of the best happytime moments in cinema :-) #
  • Totally! :) RT @timl: There is so much awesome & references I'd not noticed before! Like this part, 2nd Mission Impossible reference so far! #
  • MWAH HAHAHA! Watching Wayne's World for the first time in years, had totally forgotten the Terminator cameo. "Have you seen this boy?" LOL! #

Captain Cook by Vanessa Collingridge

A while back I dusted off a book about Captain Cook at a market second hand book store and added it to my Australian history collection.

Captain Cook: Obsession and Betrayal in the New World by Vanessa Collingridge is as the title indicates about Australia’s founder and perhaps most revered explorer James Cook, but it mixes in a good amount of history of the early (1500-1770) European discovery – and philosophy – of Australia as it was known back then (Java la Grande, Terra Australis Incognita, etc).

The book chronicles the life and times of two quite distinct figures – the eighteenth century mariner Captain James Cook and nineteenth century British adventurer, historian and polyglot (and distant relation to Vanessa) George Collingridge.

It was at this point I thought I was going to hate this because the book begins by juxtaposing these two seemingly incompatible people, person by person, chapter by chapter. I am glad I persevered, however, as there a very interesting passages of Australia’s history intertwined in both stories.

Cook would cement himself in history as the discoverer of the then unknown great southern land and Collingridge would postulate the earliest European discovery of Australia belonged to the Portuguese some centuries earlier. The scandal at the time was the very suggestion that the discovery of Australia had not belonged to Cook who was enshrined into the national psyche as the father of the continent.

Collingridge’s work would go on to inspire debate on the pre-Cook discovery of Australia and in particular that of the Portuguese. This school of thought culminates in the recent work Beyond Capricorn by Peter Trickett. [See my review of that book here exactly one year ago. I definitely need to write more book reviews].

While the two ends don’t officially ‘meet’ the history of both adventures makes the book very interesting reading. Full credit to Collingridge (Vanessa) for writing about both in the same volume.

As I’ve blogged about in the past, pre-Cook European discovery of Australia is a fascinating topic that deserves more recognition within the context of the events that led to the eventual British colonisation of the southern land.

For example, this book tells the story of Spanish explorers led by de Queiros who first called aboriginal people “Australians” in what was perhaps the first ever use of the word “Australian”. The word Australia was first coined in Spanish.

And in Cook’s travels he branded one James Mario Matra as “one of those men who could be spared”. That man who could be spared went on to recommend to the British government that Botany Bay (also named Stingray Bay) would make an ideal location for a penal colony. The Sydney suburb of Matraville is named in his honor.

Regarding the puzzling Portuguese discovery George Collingridge was convinced it was possible, but like those after him, was forced to rely on the Dieppe Maps and other historical documents and remained taunted by a lack of coroborating hard evidence as to whether it actually did happen to a level of more than sightings of the Kimberley coast and the occasional landfall.

Cook, having sailed through the South Pacific, Tahiti and as far north as Alaska met his end when he was clubbed to death by the aborigines of the Sandwich Islands (Hawaii).

Captain Cook collates a number of key themes in Australian history and once you get used to the oscillatory nature of the parallel histories of the two main protagonists makes for very interesting reading on the age of discovery in the Southern Hemisphere.

April 28, 2012

Freedom From Choice

At today's statewide meeting of the Tasmanian Linux Users Group a little distro wardiscussion broke out (sorry!) which quickly diverged into "WTF is with Unity/GNOME3" etc. I made the point that while I choose to use Enlightenment on Debian, I think that the direction that GNOME3/Unity are heading in is spot on for mass as adoption, if everyone hasn't moved to some cloud paradigm before we finish :)

My primary argument was that Devo had it right in Freedom of Choice with lines like "Freedom of choice, is what you've got. Freedom from choice, is what you want".

This video is for those who smiled in recognition, those who stared blankly at the reference and for those who like rollerball, Roman role play and bad costume aliens in their music videos...

Blog topics: 

TasLUG...Striding Into the Future

I had a great deal of pleasure meeting a stack of Freedom Lovers from across Tasmania today at the Ross Town Hall in the Tasmanian Midlands. Josh had scored us a surprisingly well fitted out venue that we might well end up using in the future again.

The venue was in the middle of Tassie, so as to equally inconvenience everyone ;) but it was a beautiful colonial town with good food, great scenery and a damn fine venue (did I mention I liked it?).

View TasLUG Statewide Gathering 2012 in a larger map

Thomas proved himself a dab hand at minute taking and posted some minutes here. The main thrust though is that we're going to poll the TasLUG community to determine which of the structures, if any, are appropriate for the community going forward. The meeting appointed a steering committee (see minutes) to publish the three proposed options and the pros/cons related to each so that everyone can be as informed as possible about which direction they'd prefer us too take.

There's also some enthusiasm around holding a Software Freedom Day event in Hobart to complement the one already being hosted successfully in Launceston and in having further statewide meetups, which would be just brilliant, IMHO :)

Nando’s Voucher Interpretation

Every year my parents buy a book of vouchers for various businesses in Victoria. It’s one of those deals where businesses (mostly restaurants) pay for advertising space to have their tear-off vouchers in the book (which typically allow a discount of between $5 and $30) and the customers buy the book for something like $40 (I’m not really sure as I don’t pay). Every year I take my pick of the vouchers that don’t suit my parents, the Nando’s chain of chicken and chips restaurants that specialises in Peri-Peri spicy sauce [1] is one that doesn’t suit my parents (they prefer the traditional English-Australian food).

The Nando’s vouchers say “Enjoy one complimentary 1/4 flame-grilled peri-peri chicken item when another 1/4 flame-grilled peri-peri chicken item is purchased” with no explanation of exactly what an “item” is. Every Nando’s store that I’ve been to in the past has interpreted “item” as chicken and chips, usually they include the drink that comes with the “quarter chicken meal” in the “item” that is free. I can do without a second soda as it’s really cheap from the supermarket and I’m not going to drink two at one meal anyway so I’m not bothered when someone interprets the voucher as not involving a free drink. But the lack of chips is annoying.

At the Nando’s store on Swanston St between La Trobe St and Little Lonsdale St they interpret “item” as being just the 1/4 chicken. I think that most people would regard this as an unusual interpretation. If the intent was to only offer 1/4 chicken then the voucher could have stated that a free 1/4 chicken was offered and removed all doubt. The fact that the voucher says a free “1/4 flame-grilled peri-peri chicken item” instead of offering a free “Quarter Chicken” (which is the description for chicken on it’s own on the Nandos menus) seems to be a reasonable indication that more than just the free chicken is offered.

I won’t be attending the Nandos store on Swanston St again and recommend that others avoid it too. Failing to offer the full value on the voucher is annoying, it decreases the value for money (which is a problem given how expensive Nandos is), and it makes me wonder what other cost-saving measures might be used at that store. I’ve got a stack of vouchers (many of which will expire before being used) and the Melbourne CBD has many places to eat.

No related posts.

Come to AdaCamp DC, July 10–11

From the Ada Initiative blog:

Applications now open for AdaCamp DC

Capitol Building in Washington, D.C. by Bernt Rostad, CC-Attribution

© Bernt Rostad, CC Attribution

Applications for AdaCamp DC are now open – apply now!

AdaCamp DC will be July 10 – 11, 2012, in Washington DC, co-located with Wikimania 2012. We are likely to have more applications than available slots, so apply now to have the best chance of attending. Applications close June 15 (May 11 for those requesting travel assistance).

Who should apply

AdaCamp DC will bring together a wide variety of people from open technology and culture, all of whom are working to support women in open tech/culture. We’re looking for people who:

  • Participate in open technology and culture: any field involving open/grassroots/community participation and sharing the results of your work for free: open data, open source software, wikis, open government, open libraries, remix/fan culture, open video, and more
  • Can share information about women’s experiences in that field, including talking about women’s achievements and the challenges they face
  • Want to work together and share strategies to support and promote women in the field
  • Share the Ada Initiative’s feminist approach to supporting and promoting women in open technology and culture
  • Are young and old; students, professionals and hobbyists; from a diverse range of backgrounds; and reflect the breadth of the open technology and culture field

AdaCamp is open to people of all genders. However, since AdaCamp and the Ada Initiative exist to support and promote women in open technology and culture, prospective attendees who are not themselves women will need to demonstrate a high level of prior engagement and experience with the issues faced by women in those fields in order to be invited.

Find out more about AdaCamp DC at the event webpage.

Sponsors

The Ada Initiative thanks our generous AdaCamp sponsors for making this event possible. Our current sponsors are the Linux Foundation, Intel, Facebook, Red Hat, Collabora, and GitHub.

Interested in becoming an AdaCamp sponsor? Email us at sponsors@adainitiative.org and we will send you more information on the benefits of sponsorship.

Homeostasis Check: HALT

The OS works best when the hardware has stability within certain environmental parameters… not to hot, not too cold, not too many yaks yodeling in the alleyway outside.

This is as true for our bodies as it is for computers.

HALT is a handy mnemonic I learned when I was working with kids in a residential treatment program to check parameters of their operating environment. It works by noticing that you feel dysregulated, down, or otherwise kind of out of sorts or off kilter. Then asking yourself:

  • Am I HUNGRY?
  • Am I ANGRY?
  • Am I LONELY?
  • Am I TIRED?

Trouble shooting these four items can solve a significant amount of distress.

A mentor of mine also added some other basic functions like

  • Do I need to pee or poop?
  • Should I drink some water?
  • Do I need to take a break from what I’m doing right now?
  • When was the last time I moved my body?

Those questions still get at the same basic principles as HALT. They just don’t fit as nicely into the mnemonic.

But I think you get the idea.

Humans don’t come with indicator panels, or alarms that go “bing” when we get low on fuel or full of information. But with a bit of practice, we can learn to recognize the early indicators of dysregulation to prevent a full-scale meltdown.

Especially when at hack-a-thons, all night coding parties, or other projects where you are working intensely for long periods of time, having an easy way to remind yourself about basic needs can be a lifesaver.

So make time to check in with yourself. Scan the environmental parameters of your operating system. It will function a lot more smoothly when you pay attention to it.

April 27, 2012

Mormon Flow Chart for Your Soul

It's a guilty pleasure poking fun at the superstitions of religious folk and I do try to restrain myself but here's a classic post that provides a Mormon flow chart for your soul.

Guaranteed to have you snorting in your cereal. I've posted an excerpt below, click through for the whole thing. Spoiler alert for the religious:

I like jwz's idea of turning this into a board game....

AttachmentSize
Sample of the Mormon Soul Flowchart48.37 KB

Raspberry Pi debian notes

So one of the guys at our office somehow ended up with two Raspberry Pi's from the first batch, as one was enough for him to play with he offered the other one up, and I turned out to be the only person in the office who wasn't so lazy as to not walk over to the other building where he was to borrow for the weekend.









Here's a bunch of useful things that you probably want to do with the default Debian installation to make it more usable.



First, please don't give the foundation guys flack for any of these issues, a decent distro is hard, and I've paid hundreds of times more then this and gotten a horrific hack-job of (usually) debian (often with a kernel already years out of date, istead of one from this year). This really isn't too bad for a first go.



Security



If you're using the pi on a network, or in a public place there are a few things to consider, it's actually pretty good compared to most embedded images I've seen.

Regenerate SSH keys



The pi already has SSH keys on the image, this is a security issue as it makes you a very easy target for MITM attacks.

As root run:

rm /etc/ssh/ssh_host_*key
dpkg-reconfigure openssh-server


Note this enables SSH server on boot, so disable it if you want, see the note below about NFS, just use "ssh" as the service. If you've used SSH before this you'll need to delete your existing entry on your client before SSH will let you connect due to the new keys.

Consider disabling NFS client (the sole open services by default)



Other then the ports being open this has no security implication, but it does save a lot of boot time.

update-rc.d portmap disable
update-rc.d nfs-common disable


Delete the pi user



Or at least change its password. If you create another admin user consider removing pi from sudoers.

Minor bits



"root" has an invalid password (same as Mac OS, Ubuntu, etc.). The users "tli" and "pnd" exist in /etc/shadow with passwords (but not /etc/passwd). The user "suse" also has full root by sudo, but doesn't exist.

Keyboard layout



Most of us don't use UK keyboards, you can switch to your local layout by running "dpkg-reconfigure keyboard-configuration". you may want at least a qwerty (if not UK English) layout keyboard for this step, will be hard without one.

Time zone



I think the concept of a "British Summer" is an oxymoron so I want to change the timezone to something more relevant to me.



You can do that by running "dpkg-reconfigure tzset" (again, sudo for root if needed)

Console Blanking



If you're using a pi as a server you might want to disable console blanking so if you connect a monitor you don't need to hit a key to wake it up (which you might not be able to do if you've somehow crashed it).



To do this edit /etc/kbd/config and change BLANK_TIME to 0>.

Debian Mirror



You may wish to change to a local debian mirror by editing /etc/apt/sources.list and changing "uk" to the appropriate two letter code (debian mirror list), then as with all apt based systems, "apt-get update" to find new packages, apt-get dist-upgrade to upgrade to them (you should be careful what you install unless you've expanded the filesystem as there's not much free space).



I'd actually suggest the following as a good base debian apt set, these include security updates:

# Main, the core of debian
deb http://ftp.us.debian.org/debian/ squeeze main contrib non-free
#deb-src http://ftp.us.debian.org/debian/ squeeze main contrib non-free

# Security updates
deb http://security.debian.org/ squeeze/updates main contrib non-free
#deb-src http://security.debian.org/ squeeze/updates main contrib non-free

# Other important updates before point releases
deb http://ftp.us.debian.org/debian/ squeeze-updates main contrib non-free
#deb-src http://ftp.us.debian.org/debian/ squeeze-updates main contrib non-free




The commented out lines are for source packages, unless you plan to do debian package development on the board itself they're not worth it

Swap



You can (but probably shouldn't unless you like killing SD cards) enable swap by uncommenting the swap line in /etc/fstab and rebooting or running "swapon -a"

Expanding the filesystem to use all (or just more) of your SD card



*WARNING* This is only applicable to the 19/April/2012 Debian build, it's very easy to destroy data by doing this wrong.





I installed on an 8GB card (as it was all I had lying about) and wanted to use all the space available. If you're going to expand the filesystem I'd suggest doing it straight away so you won't feel bad if you stuff up and destroy the OS on the card.



All of this procedure needs to be run as root.



First, change the partition size:

fdisk /dev/mmcblk0


Inside fdisk use these commands:
  • Type "p" and press enter, note the "Start" number of p2 (in this image, 1233)
  • Delete the swap partition with "d" then "3"
  • Delete the root partition with "d" then "2"
  • Recreate the root partition with "n" then "2", then start cylinder (1233 for mine), then either press enter for all the card, or follow the instructions for otherwise (using anything less then the old End cylinder of p2 will break your system)</li>
  • Verify things look ok by printing the table again ("p")
  • If they're all good use "w" to finish.




Now reboot



Once the system is back to finish expansion run:

resize2fs /dev/root


(This took several minutes on my 8GB card)



You can verify the result with "df -h"

BTRFS and ZFS as Layering Violations

LWN has an interesting article comparing recent developments in the Linux world to the “Unix Wars” that essentially killed every proprietary Unix system [1]. The article is really interesting and I recommend reading it, it’s probably only available to subscribers at the moment but should be generally available in a week or so (I used my Debian access sponsored by HP to read it).

A comment on that article cites my previous post about the reliability of RAID [2] and then goes on to disagree with my conclusion that using the filesystem for everything is the right thing to do.

The Benefits of Layers

I don’t believe as strongly in the BTRFS/ZFS design as the commentator probably thinks. The current way my servers (and a huge number of other Linux systems) work of having RAID to form a reliable array of disks from a set of cheap disks for the purpose of reliability and often capacity or performance is a good thing. I have storage on top of the RAID array and can fix the RAID without bothering about the filesystem(s) – and have done so in the past. I can also test the RAID array without involving any filesystem specific code. Then I have LVM running on top of the RAID array in exactly the same way that it runs on top of a single hard drive or SSD in the case of a laptop or netbook. So Linux on a laptop is much the same as Linux on a server in terms of storage once we get past the issue of whether a single disk or a RAID array is used for the LVM PV, among other things this means that the same code paths are used and I’m less likely to encounter a bug when I install a new system.

LVM provides multiple LVs which can be used for filesystems, swap, or anything else that uses storage. So if a filesystem gets badly corrupted I can umount it, create an LVM snapshot, and then take appropriate measures to try and fix it – without interfering with other filesystems.

When using layered storage I can easily add or change layers when it’s appropriate. For example I have encryption on only some LVs on my laptop and netbook systems (there is no point encrypting the filesystem used for .iso files of Linux distributions) and on some servers I use RAID-0 for cached data.

When using a filesystem like BTRFS or ZFS which includes subvolumes (similar in result to LVM in some cases) and internal RAID you can’t separate the layers. So if something gets corrupted then you have to deal with all the complexity of BTRFS or ZFS instead of just fixing the one layer that has a problem.

Update: One thing I forgot to mention when I first published this is the benefits of layering for some uncommon cases such as network devices. I can run an Ext4 filesystem over a RAID-1 array which has one device on NBD on another system. That’s a bit unusual but it is apparently working well for some people. The internal RAID on ZFS and BTRFS doesn’t support such things and using software RAID underneath ZFS or BTRFS loses some features.

When using DRBD you might have two servers with local RAID arrays, DRBD on top of that, and then an Ext4 filesystem. As any form of RAID other than internal RAID loses reliability features for ZFS and BTRFS that means that no matter how you might implement those filesystems with DRBD it seems that you will lose somehow. It seems that neither BTRFS nor ZFS supports a disconnected RAID mode (like a Linux software RAID with a bitmap so it can resync only the parts that didn’t change) so it’s not possible to use BTRFS or ZFS RAID-1 with an NBD device.

The only viable way of combining ZFS data integrity features with DRBD replication seems to be using a zvol for DRBD and then running Ext4 on top of that.

The Benefits of Integration

When RAID and the filesystem are separate things (with some added abstraction from LVM) it’s difficult to optimise the filesystem for RAID performance at the best of times and impossible in many cases. When the filesystem manages RAID it can optimise it’s operation to match the details of the RAID layout. I believe that in some situations ZFS will use mirroring instead of RAID-Z for small writes to reduce the load and that ZFS will combine writes into a single RAID-Z stripe (or set of contiguous RAID-Z stripes) to improve write performance.

It would be possible to have a RAID driver that includes checksums for all blocks, it could then read from another device when a checksum fails and give some of the reliability features that ZFS and BTRFS offer. Then to provide all the reliability benefits of ZFS you would at least need a filesystem that stores multiple copies of the data which would of course need checksums (because the filesystem could be used on a less reliable block device) and therefore you would end up with two checksums on the same data. Note that if you want to have a RAID array with checksums on all blocks then ZFS has a volume management feature (which is well described by Mark Round) [3]. Such a zvol could be used for a block device in a virtual machine and in an ideal world it would be possible to use one as swap space. But the zvol is apparently managed with all the regular ZFS mechanisms so it’s not a direct list of blocks on disk and thus can’t be extracted if there is a problem with ZFS.

Snapshots are an essential feature by today’s standards. The ability to create lots of snapshots with low overhead is a significant feature of filesystems like BTRFS and ZFS. Now it is possible to run BTRFS or ZFS on top of a volume manager like LVM which does snapshots to cover the case of the filesystem getting corrupted. But again that would end up with two sets of overhead.

The way that ZFS supports snapshots which inherit encryption keys is also interesting.

Conclusion

It’s technically possible to implement some of the ZFS features as separate layers, such as a software RAID implementation that put checksums on all blocks. But it appears that there isn’t much interest in developing such things. So while people would use it (and people are using ZFS ZVols as block devices for other filesystems as described in a comment on Mark Round’s blog) it’s probably not going to be implemented.

Therefore we have a choice of all the complexity and features of BTRFS or ZFS, or the current RAID+LVM+Ext4 option. While the complexity of BTRFS and ZFS is a concern for me (particularly as BTRFS is new and ZFS is really complex and not well supported on Linux) it seems that there is no other option for certain types of large storage at the moment.

ZFS on Linux isn’t a great option for me, but for some of my clients it seems to be the only option. ZFS on Solaris would be a better option in some ways, but that’s not possible when you have important Linux software that needs fast access to the storage.

Related posts:

  1. Starting with BTRFS Based on my investigation of RAID reliability [1] I have...
  2. ZFS vs BTRFS on Cheap Dell Servers I previously wrote about my first experiences with BTRFS [1]....
  3. Reliability of RAID ZDNet has an insightful article by Robin Harris predicting the...

April Statewide TasLUG Gathering

As announced here on the TasLUG website, tomorrow is the state-wide Tasmanian Linux users meetup.

So come along Freedom Lovers and catch-up with some like-minded Freedom enthusiasts here in Tassie and participate in the decision making around the direction TasLUG will take over the next few years! The details in brief are:

Saturday April 28th

Ross Town Hall

  • 12:00pm - 2:00pm: Mill around, get to know each other, show off any gear/projects we may have brought with us
  • 2:00pm - 3:30pm: Meeting (covering voting on an organisational structure for TasLUG, maybe discussing SFD 2012 and anything else that gets raised),
  • 3:30pm - 4:00pm: Pack up
  • 4:00pm onwards: Go home/BBQ/whatever anybody is up for
    • Google Maps link

      The rough agenda we have for the meeting part is:

      • TasLUG Organisational Structure Vote (and nominate some people to move us forward in whatever direction we end up going)
      • SFD Planning
      • Next Year's Meetup,

      For more complete details, go here for the full announcement

Blog topics: 

Editing XML and PostgreSQL with ferris REST.

With a few minor tweaks, one can now edit the contents of XML files and PostgreSQL tables with the libferris REST interface. The YUI web interface has been updated to allow that to easily happen. In the below video, example.xml is first shown and then instead of viewing it as a file, I choose to "read" it as a directory, causing libferris to sniff it out and work out that it can mount that file as a directory for you. The same can be done using "ferrisls example.xml" at the command line.

Notice that the read link is only offered on the XML files. This is because libferris tells the client that those are not natively "directories" but can be seen that way if you like.

I then simply click down to the "barry" XML element in the mounted XML file. Editing the barry entry will write the data back to the server asynchronously. The terminal is used to verify that things went on back.

The second browser tab shows a mounted PostgreSQL table which has ID and Message columns. If I select edit on the message column I get to update the tuple as desired. Notice that the data grid in the browser is updated when the data is saved on back by watching the "message" column in the browser. I then use psql to verify the update from the command line.

Other interesting possibilities include mounted log files being split into columns in the web view, or grabbing some data from plasma on the server, streaming data from gstreamer or zoneminder, or anything else that ferris can do as a VFS. And as I tend to want ferris to mount it all^TM, the sky should be the limit! Heh, so I got the catchphrase in there emes.

April 26, 2012

PyCon Australia early bird registrations now open!

For fear of spamming EVERYWHERE with the news, I include just the tl;dr:

tl;dr: PyCon Australia early bird registrations are now open! Find out more at http://2012.pycon-au.org/register/prices, including details of our accommodation programme.

The full media release on the opening of registration can be found at http://2012.pycon-au.org/media/news/15

Hope we see you all registered soon!

PyCon Australia 2012 Early Bird registration and Accommodation deals now available!

tl;dr: PyCon Australia early bird registrations are now open! Find out more at http://2012.pycon-au.org/register/prices, including details of our accommodation programme.

PyCon Australia is excited to announce that early bird conference registrations are now available for our 2012 conference, to be held on Saturday 18 and Sunday 19 August in Hobart, Tasmania. Early bird registration will be extended to the first 60 confirmed conference registrations, or until Friday 1 June, whichever comes first.

PyCon Australia is the national conference for students, enthusiasts and professionals working with the Python programming language; it represents a unique opportunity for Python developers to meet fellow developers, and gain knowledge from experts and core Python developers from around Australia and the world. Securing your registration during the early bird period ensures your place at all of the events that PyCon Australia has to offer.

Early bird registration comes with a substantial discount for tickets at our "Enthusiast" and "Professional" rates. Early bird tickets at both the "Enthusiast" and "Professional" level are guaranteed a seat at our conference dinner. All tickets include access to the CodeWars event on Friday 17 August, and the post-conference sprints on Monday 20 and Tuesday 21 August.

Early bird registration starts at $44 for full-time students; $168 for enthusiasts and $420 for professionals.

PyCon Australia has been working closely with our venue to provide a great conference experience; we're very pleased to be able to offer accommodation to delegates for the duration of the conference. We've secured an allocation of rooms within the Wrest Point complex, many of which are in the same building as the conference venue. Rooms available to delegates start at $124 per night; rooms inside the conference building start at $146 per night.

Information on conference registration, including details on how to book delegate accommodation through our preferred provider can be found at the PyCon Australia website (http://2012.pycon-au.org).

Our conference Call for Proposals is still open, and will close on Friday 4 May.

We can't wait to see you in Hobart in August!

About PyCon Australia

PyCon Australia is the national conference for the Python Programming Community. The third PyCon Australia will be held on August 18 and 19, 2012 in Hobart, Tasmania, bringing together professional, student and enthusiast developers with a love for developing with Python. PyCon Australia informs the country’s Python developers with presentations, tutorials and panel sessions by experts and core developers of Python, as well as the libraries and frameworks that they rely on.

To find out more about PyCon Australia 2012, visit our website at http://pycon-au.org or e-mail us at contact@pycon-au.org.

PyCon Australia is presented by Linux Australia (www.linux.org.au) and acknowledges the support of our Gold sponsor, Google Australia (www.google.com.au).

April 25, 2012

brendanscott

Meaning of Gotye “Somebody that I used to know”

Challenge accepted…

Summary:

He hasn’t come to terms with his break up with his former lover.   He is particularly affronted that what he called love was nothing to his former lover and is scared that it will happen again.[1]  He becomes involved in a new relationship.  His new lover explains how he has hurt her because he hasn’t “let it go”.  He recognises that he’s being a heel.  He signals to her that he has faith in her/is ready to “let it go”.  She signals her faith in him/the relationship.  There is an opportunity for reconciliation.

Discussion

Throughout the video Gotye sings to the camera.  Kimbra sings to Gotye.  The thing which Gotye and Kimbra have in common is the painting.   The painting starts after the end of the first stanza.  By then we know:

(i) he has already broken up with whoever he is addressing;

(ii) he knew the old relationship was empty;

(iii) he feels hurt that his old lover thought it was nothing (/less than  nothing?).

The fact that his old lover cuts him off (even though the break up was amicable) suggests that his old relationship was not just empty, but was illusory – and that he had committed to this illusion.  He thought it was love.

The painting is the start of his new relationship with Kimbra.  That relationship develops through the second stanza.  It is only after the relationship has developed that we are introduced to the Kimbra character.  (He has his back to the wall, but she has her front to the wall -  I assume there is nothing in this other than the desire to have a G rating for the video.  Neither character acknowledges the existence of the wall/painting [see also note 1])

As Kimbra starts singing she turns her head so that:

(i) the green paint on her cheek aligns with the rest of the painting/relationship

(ii) she is addressing Gotye.

When she says he “screwed [her] over” he sighs noticeably.  When she says he hurt her (“believing it was something I had done”) he closes his eyes.

Just before she sings “I don’t want to live that way” she motions noticeably to break herself from the painting/relationship before approaching Gotye.  She is still painted at this point.  They are almost touching – she addresses him but he continues to address the camera.   When she says that he had promised her that he was over his old lover Gotye closes his eyes/looks down, away from Kimbra.  When he starts singing again he still talks about how he was hurt by break up of the old relationship/confused by his realisation it was nothing.  Perhaps he is scared that his new lover will also think he’s nothing/that the new relationship might also not be real.  What if he commits to this new relationship and discovers she too thinks he/it is nothing? [Kierkegaard/Existentialism - how can you commit your whole being to something which you know can end - if that happens, won't it destroy you too?]

While he is singing about his hurt, she seems to be imploring him to acknowledge she is there.  He doesn’t look at her.  She starts to move off.  He realises his new relationship is not an illusion/not nothing.  He starts to look towards her (he has not done so earlier).  She returns to her original position and the painting/relationship begins to come off her. As this happens his singing becomes increasingly more subdued.  He spends more time looking to her.

As the painting/relationship comes off her, he sings partly to the camera, but also partly to her.  When he is saying “you” in “you’re just somebody that I used to know” he addresses the camera.  But when he is facing Kimbra he’s singing “somebody” and “used to know”.  After she has moved away from him his singing doesn’t return to his earlier hurt.  All he sings about is how the old lover is somebody that he used to know – he is over her/her opinion of him now.  He also increasingly looks towards her – while she walks away there are cuts with him looking towards her and at the camera but he spends all of the end of the video (the last 15 seconds or so) looking at her.  His subdued singing, the content of what he is singing and the fact that he is looking to Kimbra is signaling that he’s ready to have faith in the new relationship.

While he does this, their stances are reversed.  She faces the wall/painting/relationship while he is looking to her.  As the song ends, she stops looking at the wall and turns her head to look at him.  Both of them sing the last word (“somebody”) to each other (they “harmonise”/”in harmony”).  This is the only time that they sing together – in fact it’s the only time they communicate with each other directly.  It is also the only time they make eye contact.  They are signaling faith in each other to each other.  She has faith that he has put his past experience behind him, he has faith that she believes in him/the relationship.  When the video ends they are both in frame, he is painted and she is not – she is ready to start anew.[2] The painting on the wall/relationship is still there.  They are looking at each other.

Whatcha think?

Conclusion

He hasn’t come to terms with his break up with his former lover.   He is particularly affronted that what he called love was nothing to his former lover and is scared that it will happen again.  He becomes involved in a new relationship.  His new lover explains how he has hurt her because he hasn’t “let it go”.  He recognises that he’s being a heel.  He signals to her that he has faith in her/is ready to “let it go”.  She signals her faith in him/the relationship.  There is an opportunity for reconciliation.

Note:

Natasha Pincus, the film maker behind the music video has a different view, implying Gotye is singing about his current relationship with the Kimbra character.  Michael Cathcart describes it as a break up song/tiff song.  It is not either.

End Notes:

1: The issue is not that Gotye is upset at losing the previous relationship.  He admits that the relationship was empty, is happy they split up and asserts that the old lover is just somebody that he used to know.  The issue is that the old lover really is “just somebody that [he] used to know”.  That is, the relationship was nothing when he thought it was something.  We know his new relationship is “something” because it’s all over the wall – it literally (err… in a figurative sense) exists outside of him/her.  However, he doesn’t seem to see it so can’t be sure about it, and this is what is hurting her.

It may be that Kimbra faces the wall because she can see that their relationship is meaningful, while he has his back to the wall because he hasn’t seen it.  She never turns fully with her back to the wall.

2: The removal of the paint from Kimbra is actually pretty hard to parse.  If she is leaving the relationship, why does she stay in the picture?  Why does she face the wall/relationship?  Why does she sing to/with Gotye? Why doesn’t the paint cover her face as it does Gotye’s? (and why does she have such heavy eye makeup??)

From her point of view, she can’t see the paint on her own back.  Does the removal of the paint mean that this is how Gotye sees her – she sees him as being part of the relationship, but he doesn’t see her that way?  Does it mean that Gotye sees her as a person/for what she is rather than as an adjunct to the relationship?



The ZaTab from @ZaReason – a fully open source Android (or whatever you want to load) tablet (UPDATED)

ZaReason are a US company who only make Linux based computers and have recently been tweeting about building a completely open source tablet device, shipping with Android but unlocked so you can install whatever you would wish on it. They have even been working with the Software Freedom Conservancy to ensure that it passes muster as an open source device.

ZaReason ZaTab

However, other than some photos of it on Twitter details have been a little lacking, but now the ZaReason CEO (who is in New Zealand working on setting up a store there) tweeted the URL for pre-orders which includes the details about it:

  • Pure Android
  • Allwinner A10 SoC
  • 9.7″ IPS 1024×768 display
  • 5 point capacitive touchscreen
  • 16 GB internal storage + microSD for additional storage
  • 1 GB ram
  • b/g/n WiFi
  • Front and Back cameras
  • Sturdy metal back
  • High-capacity 8000 mAh battery
  • Ultra-light 630 grams

Ports:

  • Headphone
  • microSD card slot
  • mini-HDMI video out
  • 2x micro-USB ports

The device is shipping with CyanogenMod 9 (so based on Google’s Android Open Source Programme – AOSP – Ice Cream Sandwich release) and yes, it has root access available (CyanogenMod 9 doesn’t enable it by default, but it is just a configuration option). I would suspect this means it won’t ship with the Google Apps package which are not open source and so you won’t have access to the Google Play Store (formerly the Android Market), but you could still access the F-Droid open source application repository from it and should you feel the need for the proprietary Google Apps then you could reflash CyanogenMod9 with the Google Apps package available from their site. Most importantly it ships with all the source code.

The Socket on a Chip (SoC) is the Allwinner A10 which has a 1GHz ARM Coretex A8 and a MALI 400 MP1 GPU. Whilst the GPU manufacturer releases GPL driver code you apparently need a proprietary DDK to be able to produce a functional driver and so the Lima project has been born to create a fully open driver. Quite how the ZaReason people are dealing with this is unclear, if they are really shipping a fully open tablet then perhaps the Lima driver is a lot more stable than their project website claims. :-) (Clarified below in the update)

They’ve also promised the boot loader is unlocked so you can put whatever software you should wish to try out on it, on Twitter they said that they intend to try and get KDE going on it (presumably the KDE Plasma Active project, possibly using Mer as the supporting distribution). There’s even been a tongue-in-cheek reference to Gentoo.. :-)

Update:

Two updates on the ZaTab:

  • firstly that the pre-orders they are taking are limited to FOSS people, they want you to contact them via email as part of the process (they have concerns about fraud).
  • Secondly on the openness and GPU driver, they have confirmed that “Initially there may be some binary blobs. Lima isn’t far enough along at this time, but we have high hopes for it”.

This item originally posted here:



The ZaTab from @ZaReason – a fully open source Android (or whatever you want to load) tablet (UPDATED)

Getting the GLib gdb macros with your own installed GLib

So there are these really cool debugging macros for GLib/GObject that do things like let you iterate GLists and pretty print GObjects and GHashTables and stuff that should nowadays be available in most distros.

Unfortunately when you build your own GLib to develop against, it all breaks.

It turns out, gdb locates when to load these things by looking in $prefix/share/gdb/auto-load/ and matching the sonames, so if you have libraries with a different soname (cause they’re new versions), or in a different prefix, it just doesn’t work.

Damien Lespiau has a workaround for this that adds a command to load them from a path you can control with sys.path; which works. I looked a little for a generic solution to feed the autoloader from my GLib build prefix, but I haven’t found one yet.

Having now looked at this. I’m also excited about possibilities for combining this with GObject-Introspection to potentially do some useful things. I don’t know what the caveats are here yet, what’s safe to call and what’s not.

April 24, 2012

Less SPOFs: pyjunitxml, testscenarios

I’ve made the Testtools committers team own both the project and the trunk branch for both pyjunitxml and testscenarios. This removes me as a SPOF if anything needs doing in those projects – any Testtools committer can now do it. (Including code review and landing). If you are a testtools committer and need PyPI release rights, ping me and I’ll add you. (I wish PyPI had group management).



April 23, 2012

VFS in the Could? Libferris Web Interface...

I'm syndicating to planet KDE because things in the post might be of interest for KDE. I'd be overjoyed to see some of the features in KDE too, the more powerful the tools available to folks the better the future tools will be ;) So, on with the show... I decided to add REST and YUI stuff to libferris. This is still very much a work in progress in spare time... Luckily the heavy lifting is all done already in the libferris library.

The initial web interface is still fairly basic, the back and forward buttons are handled by the browser leaving only the parent button in the apps toolbar. Home and Heart are your home dir and bookmarks respectively.

Clicking on a row allows arbitrary annotation of that file. The annotations are stored in either native kernel level Extended Attributes or RDF. A feature I find very useful is that all metadata is presented via the same interface. As you can see the "Annotation" column in the listview is showing your own description of each file. You can filter or sort on annotation just as you can on the file name.

The search page allows you to find files by their text content (full text search) and/or their metadata. As I've mentioned in the past, the metadata indexing modules include many optimizations above using the native APIs. This includes indexed lookup on certain classes of regular expressions. Many naive query evaluations using regular expressions will result in a linear time complexity. And unless you have used explicit code to handle it, you are likely to enjoy this bad performance even with very advanced indexing libraries and databases.

Tags or "emblems" etc are also handled through the same metadata interface. The tagging sidepanel offers suggestions for existing tags as you type and allows you to create new tags as you attach them to files. Removal of a tag is just a click away.

Of course, clicking a filename shows you the file itself over REST. This allows you to stream video files over REST to the Nokia n9 for example. There is partial IO support and write support so I could include a fancy text editor or image editor component in there... Unfortunately YUI 3.5.0 doesn't seem to support "selections" in the datagrid, so that nice visual feedback will have to wait for yui 3.6.0.

Links April 2012

Karen Tse gave an interesting TED talk about how to stop police torture as an investigative tool [1]. Mostly it’s about training and empowering public defenders.

Phil Plait gave an interesting TED talk about how to defend the Earth from asteroids [2].

Julian Baggini wrote an interesting article for the Financial Times about the persecution of Atheists in the US [3].

Charlie Todd of Improv Everywhere gave an amusing TED talk about Improv events that he has run [4]. He is most famous for organising people to wear blue shirts and khaki pants in Best Buy, but he’s done lots of other funny things.

Paul Zak gave an interesting TED talk about trust, morality, and oxytocin [5]. One of the many interesting fact that he shared is that oxytocin levels can significantly increase when using social networking sites. So people who use Facebook etc are likely to be more trustworthy as well as more trusting.

The Occupy the Judge Rotenberg Center movement aims to stop the torture of Autistic children in the US [6], Anonymous is involved in that too.

Paul Lewis gave an insightful TED talk about the use of crowdsourced data in news reporting [7]. A lot of the analysis of “citizen journalism” is based on comparing bloggers with full-time paid journalists, but Paul describes how professional full-time journalistic work can be greatly assisted by random people filming and photographing things that seem noteworthy. Make sure your next phone has the best possible camera – phone cameras will never be great but the quality of the camera you have with you is what matters.

Sam Harris published an interesting interview with Tim Prowse who is a Baptist minister who faked belief for two years after becoming an atheist [8A]. He also references The Clergy Project – a support group for atheists who are current or former members of the clergy [8B].

Cracked has an insightful article about 6 things that rich people need to stop saying [9]. How do the 1% not understand these things?

Barack Obama and Nichelle Nichols (who played Lt Uhura in the original Star Treck) give the Vulcan Salute in the White-House [10].

Gabriel Arana wrote an insightful article about his experiences with the ex-gay movement [11]. The “therapist” who hurt him so much is still doing the same to other victims.

S#!T Ignorant People Say To Autistics is an interesting youtube video about ignorant and annoying people [12]. Strangely I’ve received little of that myself, I wonder whether women on the Autism Spectrum get a lot more of that than men.

At GManCaseFile an ex-FBI agent has written an informative post about how the TSA is failing [13].

The Nieder Family has an interesting article about how patents are being used to prevent the creation of assisted communication (AAC) devices for children [14]. Apparently the company that has the patents wants all AAC devices to be really expensive and profitable for them. This is yet another example of patents doing harm not good.

Renew Economy has an informative article by Giles Parkinson about the affect that solar power generation will have on power prices [15]. In short as solar systems produce power when it’s most needed (during the day and at the hottest time of the day for warm climates) it will dramatically reduce the auction price for wholesale power. That will hurt the business of the power companies and also allow lower prices on the retail market.

Related posts:

  1. Links March 2012 Washington’s Blog has an informative summary of recent articles about...
  2. Links February 2012 Sociological Images has an interesting article about the attempts to...
  3. Links April 2010 Sam Harris gave an interesting TED talk about whether there...

Advertising

Started to advertise the group - more to come.

For subcommittee: 

Jupiter







ISBN: 9780812579413

LibraryThing

This is the first book of Ben Bova's that I've read. Before that I've only read "The Nonmetallic Isaac or It's a Wonderful Life" in Foundation's Friends. I bought this book randomly because I had run out of things to read on a business trip, and I don't regret it. The book is well paced, interesting and fun to read. It also explores alien life in a way which is particularly believable (unlike many other SF books I encounter). This book reminds me of Dragon's Egg without being so hard-SFy. A very good book.



Tags for this post: book ben_bova alien aliens jupiter exploration religion

Related posts: Isaac Asimov's Robot City: Robots and Aliens: Humanity; Isaac Asimov's Robot City: Robots and Aliens: Maverick; Dragon's Egg; Runner; Starbound; The Accidental Time Machine ; Patron saints; Isaac Asimov's Robot City: Robots and Aliens: Changeling; Logos Run; Marsbound; Speaker For The Dead; Snow Crash ; Camouflage ; Isaac Asimov's Robot City: Robots and Aliens: Alliance; The Robot City, Robots and Aliens Series; The Coming; Isaac Asimov's Robot City: Robots and Aliens: Intruder; Isaac Asimov's Robot City: Robots and Aliens: Renegade
Comment Recommend a book

The Top of My Todo List

Paul Graham takes the five regrets of the dying and turns them into positive, affirmative statements. Things you can live by.

Permalink

April 22, 2012

Neighborhood Watch

While writing my previous post I heard a huge noise at the front of my house. I found one man being restrained in a seated position on the ground at my front door, the man who was holding him down was accusing him of theft and asking me to call the police, and a woman was hanging around and crying.

When calling the police I discovered that Optus (the Telco that provides the virtual service which Virgin Mobile uses) doesn’t accept 112 as an emergency number! This combined with the fact that CyanogenMod 7 on my phone doesn’t accept 000 as an emergency number meant that I had to unlock my phone before calling the police. Unlocking your phone late at night when there’s a situation that needs police attention isn’t as easy as you would hope. As an aside there are usually no penalties for testing the emergency service on your phone, people who install PABX systems and other significant telephony devices test emergency services calls as a matter of routine, so testing emergency calls from your phone is a really good idea. If anyone knows how to configure CyanogenMod 7 to support 000 as an emergency call then please let me know!

Anyway the man who was held down claimed that a friend of his had given him a bag containing tools that he had lugged from some place not particularly near my house. The man who was holding him down said that he witnessed the other man stealing the tools from his neighbor – not far from my house. The woman was apparently the girlfriend of the man who was accused of burglary.

The end result was that the police arrested the man who was accused of burglary and his girlfriend. He didn’t have any obvious injuries and the police said that the man who detained him did them a favor, so it seems unlikely that there will be any assault charges filed. Presumably the man who detained the burglar is explaining it all at the police station now, I hope the police gave him a chance to put on pants and shoes first.

The man who made the burglary accusation said that his house was robbed last night which is why he was more observant than usual tonight.

This makes me glad of my policy of rejecting every job offer which involves moving to the US. In Australia hand guns are really hard to get so there’s no way that a house burglary will involve a gun and there’s also no way that someone who wants to help the police will have a gun. So while it was unpleasant to have this happen at my front door it didn’t involve any risk to me. It could have ended up with someone other than me getting a beating but the probability of serious injury or death for them was quite low. As everyone knew that no-one had a gun and no-one wanted to be charged with assault it made sense for everyone to avoid excessive force. From what I saw no excessive force was used.

The police arrived fairly quickly and EVERYONE was glad to see them. All up it took a bit more than 30 minutes from the first noise to the police departing after arresting both suspects and filling out a bunch of paperwork. I was impressed by that!

Related posts:

  1. CyanogenMod and the Galaxy S Thanks to some advice from Philipp Kern I have now...

Autism as an Excuse

A Polish geek going by the handle of mmemuar has recently written a blog post claiming that people use Autism as an excuse for bad behavior [1]. He gets enough things wrong in one short post to make it worth debunking it. It seems that Google’s translation of Polish isn’t as good as some other languages, but unless Google mistranslated about a dozen sections such that they had the exact opposite meaning then mmemuar’s post has a lot of wrong ideas.

Does Anyone use Autism as an Excuse?

I’ve read a lot of blog posts written by people on the Autism Spectrum, read many forum discussions, and talked to more than a few people in person. So far I haven’t yet encountered any evidence of people using an Autism Spectrum Disorder (ASD) diagnosis as an excuse. There are probably about 70,000,000 people who meet the diagnostic criteria (most of whom have not been diagnosed due to not having access to anyone who is qualified to do an assessment). The number of people who have been diagnosed is large enough that I couldn’t claim that none of them have ever used it as an excuse. The number of self-diagnosed people is also large enough that there has to be some people who wouldn’t get diagnosed if professionally assessed. But I don’t see any evidence that using an ASD as an excuse is at all common.

I imagine that some people would take someone merely mentioning the fact that they have an ASD diagnosis in a public place (EG a mailing list or a blog post) as some sort of an excuse. One problem with such an interpretation is that for every way in which people on the Autism Spectrum annoy other people it’s the ones who aren’t diagnosed (or who reject a diagnosis) that will do it the most. Being diagnosed with an ASD is correlated with annoying other people less. Another problem is that keeping quiet about such things when they get raised for discussion so often takes a psychological toll.

When someone is diagnosed as an adult a fairly common reaction is to study Psychology and Sociology (usually through web sites such as Sociological Images – which I highly recommend [2]) and try to get a better understanding of other people. Any time you assume that everyone else thinks like you then you will get things wrong, when someone gets an ASD diagnosis they will probably take more care to avoid making such mistakes.

Is Autism Obvious?

Mmemuar says “if you were full of autistic, which is very easy to overheat the brain from excess signals at the input, it would be very obvious to all“. There are some people who are utterly incapable of acting like an NT. But the majority of people on the Autism Spectrum have some ability to pass as NT, it just takes a lot of effort. So if someone is spending all their effort to walk, talk, and make eye contact in a way that most people expect then they will have little spare effort for other things. This can result in them having little patience for other people. The solution to this is to not require people to look average.

Even apart from Autism there are people who fidget, don’t make eye contact in the way you expect, and do other things a little differently. Being tolerant of such things won’t hurt you and will generally make things easier for everyone.

TheAnMish gave a good Youtube presentation about the way that she has to act “normal” [3]. Note that while her video represents her own personal experiences (which differ slightly from those of other Aspies – particularly male Aspies) they are regarded as representative enough for her video to be shown by Tony Attwood (a world renowned expert on Asperger Syndrome) at a conference about women on the Autism Spectrum. As an aside I disagree with her use of the word “normal” without scare-quotes.

Learning about Psychology through Sci-Fi

In the end of the blog post and in some of the comments there is discussion about learning to understand people through reading sci-fi books. The first problem with this is that fiction books generally have a range of characters that is determined by the author’s understanding of people, if you read multiple books by an author then you will usually notice the same character types. The second problem is that characters in fiction books are simplified to fit into a reasonable sized book, in real life people have lots of really boring reasons for doing what they do, in fiction only stuff that is interesting ends up in print.

But the biggest problem is that fiction books just aren’t a good way of learning about people. Whatever lessons might be in a fiction book will probably be missed by a reader who is concentrating on the plot. You can learn things about people by reading an analysis of literature by a Psychologist or a Sociologist, but apart from that you probably won’t be able to learn things if you don’t already know them.

As an aside, my experience of reading sci-fi books suggests that some of the popular sci-fi authors have such a poor understanding of people that it impairs their ability to write believable fiction about human characters. If I was going to try and learn about people by reading fiction I’d choose something that’s been popular for a long time. If a book has been popular for more than 100 years and sold well in different times, cultures, and languages then it probably has something to say about the human condition.

Catching up on Youth

Mmemuar says “absolutely nothing prevents you to the age of twenty he began to catch up on some ‘of youth’“. Actually one significant thing is that the human brain develops in particular stages, the older you get the more difficult it becomes to learn things. So even if it was just a matter of learning things someone who was behind at age 20 would have some significant difficulty in catching up. But some things just can’t be learned, for example someone who has extreme discomfort in making eye contact can’t just learn to be happy with it.

Also there are some things which people would have learned if it was possible. For example people who have Prosopagnosia (an inability to recognise faces) suffer extreme bullying in school, if they could just learn to recognise people then they surely would do so, so if they complete school without learning then it’s probably going to be impossible for them. Prosopagnosia is one of many conditions which can contribute to social difficulties and therefore contribute to an ASD diagnosis.

Can Aspies get Married?

Some people think that people on the Autism Spectrum can’t get married. In fact this belief is so widely held that some people who seem very obviously Autistic are convinced that they are NT simply because they are married! The fact that there are more than few books offering advice to people who have married someone who is on the Autism Spectrum is clear proof that such claims are bogus.

The Relevance to Geek Communities

People who meet the diagnostic criteria for Asperger Syndrome will almost certainly be Geeks due to the “Restricted, repetitive patterns of behavior, interests, or activities” section of the diagnostic criteria (the proposed revision for the DSM-V is the best reference I know for this [4]) as the modern use of the term Geek applies to anyone who has an extreme interest in something. The more Geeky a community is the greater the incidence of people who could be diagnosed with an ASD.

Spreading ideas such as those of mmemuar will lead to people not being assessed for an ASD. I would have been assessed earlier if it wasn’t for hearing an influential member of the Linux community say some things which were similar in concept (although not as deliberate). Having people not be assessed is bad for the individuals in question and bad for the community.

Related posts:

  1. Autism Awareness and the Free Software Community It’s Autism Awareness Month April is Autism Awareness month, there...
  2. Autism vs Asperger Syndrome Diagnostic Changes for Autism Spectrum Disorders Currently Asperger Syndrome (AS)...
  3. Communication Shutdown and Autism The AEIOU Foundation The AEIOU Foundation [1] is a support...

Twitter Weekly Updates for 2012-04-22

  • Anybody want an @ibmhpc Power5 #HPC cluster? Here's one on E-Bay http://t.co/A403CIaJ #
  • Tonight's odd soup with @donna_Williams is chicken, swertcorn, onion, bacon and brussel sprout. :-) #
  • BTW the @ibmhpc Power5 #hpc cluster on eBay I mentioned? It's OpenPOWER 720, so only runs #Linux http://t.co/A403CIaJ #
  • I liked a @YouTube video http://t.co/n0VQpwHK The Muppets – Bohemian Rhapsody #
  • Anyone know if the Jasper Carrott sketch with "Beelzebub has a devil for a sideboard" is around on @YouTube? #
  • Hey @Internode, your survey email doesn't display properly in Thunderbird 11.0.1 #
  • For those in New York City in US, you may see space shuttle Enterprise fly over low on April 23rd http://t.co/r1rU5wwd #
  • "cybercrime surveys they have examined 90 percent of the estimate appears to come from the answers of one or two individuals." #
  • #Melbourne Australia, seen from the ISS at night #space #esa http://t.co/QquALGZ6 #
  • gpu hang means an unwanted reboot whilst trying to work from home. OS was fine, could tell KDE to logout from VC. #
  • Pray tell AFACT how will changing the law alter the fact that ISPs "had no direct technical power" to stop piracy? #iitrial #
  • Please take the Parliamentary marriage equality survey – let us all count, not just the discriminators http://t.co/nVn8SlPi #
  • Note Mr Jones bill compels celebrants to perform marriage ceremony, whereas @AdamBandt's bill does not. #equallove #
  • I cannot support anyone being compelled to perform services against their will, so can only support @AdamBandt's bill. #
  • If you use WD Green drives with #Linux you may want to look at this: http://t.co/0F3sgLRK #
  • Unfortunately that idle3ctl from idle3-tools doesn't seem to work with WD 6400AAV external USB drives. :-( #
  • Ever wondered just how much #caffeine may be lethal? http://t.co/SM8QrPZT (of course some are more susceptible than others) #
  • At Cardinia with @donna_williams watching thunderstorm pass over from safety of our car. Quite spectacular! #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-04-22

Twitter Weekly Updates for 2012-04-22

  • Anybody want an @ibmhpc Power5 #HPC cluster? Here's one on E-Bay http://t.co/A403CIaJ #
  • Tonight's odd soup with @donna_Williams is chicken, swertcorn, onion, bacon and brussel sprout. :-) #
  • BTW the @ibmhpc Power5 #hpc cluster on eBay I mentioned? It's OpenPOWER 720, so only runs #Linux http://t.co/A403CIaJ #
  • I liked a @YouTube video http://t.co/n0VQpwHK The Muppets – Bohemian Rhapsody #
  • Anyone know if the Jasper Carrott sketch with "Beelzebub has a devil for a sideboard" is around on @YouTube? #
  • Hey @Internode, your survey email doesn't display properly in Thunderbird 11.0.1 #
  • For those in New York City in US, you may see space shuttle Enterprise fly over low on April 23rd http://t.co/r1rU5wwd #
  • "cybercrime surveys they have examined 90 percent of the estimate appears to come from the answers of one or two individuals." #
  • #Melbourne Australia, seen from the ISS at night #space #esa http://t.co/QquALGZ6 #
  • gpu hang means an unwanted reboot whilst trying to work from home. OS was fine, could tell KDE to logout from VC. #
  • Pray tell AFACT how will changing the law alter the fact that ISPs "had no direct technical power" to stop piracy? #iitrial #
  • Please take the Parliamentary marriage equality survey – let us all count, not just the discriminators http://t.co/nVn8SlPi #
  • Note Mr Jones bill compels celebrants to perform marriage ceremony, whereas @AdamBandt's bill does not. #equallove #
  • I cannot support anyone being compelled to perform services against their will, so can only support @AdamBandt's bill. #
  • If you use WD Green drives with #Linux you may want to look at this: http://t.co/0F3sgLRK #
  • Unfortunately that idle3ctl from idle3-tools doesn't seem to work with WD 6400AAV external USB drives. :-( #
  • Ever wondered just how much #caffeine may be lethal? http://t.co/SM8QrPZT (of course some are more susceptible than others) #
  • At Cardinia with @donna_williams watching thunderstorm pass over from safety of our car. Quite spectacular! #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-04-22

Twitter Weekly Updates for 2012-04-22

  • Anybody want an @ibmhpc Power5 #HPC cluster? Here's one on E-Bay http://t.co/A403CIaJ #
  • Tonight's odd soup with @donna_Williams is chicken, swertcorn, onion, bacon and brussel sprout. :-) #
  • BTW the @ibmhpc Power5 #hpc cluster on eBay I mentioned? It's OpenPOWER 720, so only runs #Linux http://t.co/A403CIaJ #
  • I liked a @YouTube video http://t.co/n0VQpwHK The Muppets – Bohemian Rhapsody #
  • Anyone know if the Jasper Carrott sketch with "Beelzebub has a devil for a sideboard" is around on @YouTube? #
  • Hey @Internode, your survey email doesn't display properly in Thunderbird 11.0.1 #
  • For those in New York City in US, you may see space shuttle Enterprise fly over low on April 23rd http://t.co/r1rU5wwd #
  • "cybercrime surveys they have examined 90 percent of the estimate appears to come from the answers of one or two individuals." #
  • #Melbourne Australia, seen from the ISS at night #space #esa http://t.co/QquALGZ6 #
  • gpu hang means an unwanted reboot whilst trying to work from home. OS was fine, could tell KDE to logout from VC. #
  • Pray tell AFACT how will changing the law alter the fact that ISPs "had no direct technical power" to stop piracy? #iitrial #
  • Please take the Parliamentary marriage equality survey – let us all count, not just the discriminators http://t.co/nVn8SlPi #
  • Note Mr Jones bill compels celebrants to perform marriage ceremony, whereas @AdamBandt's bill does not. #equallove #
  • I cannot support anyone being compelled to perform services against their will, so can only support @AdamBandt's bill. #
  • If you use WD Green drives with #Linux you may want to look at this: http://t.co/0F3sgLRK #
  • Unfortunately that idle3ctl from idle3-tools doesn't seem to work with WD 6400AAV external USB drives. :-( #
  • Ever wondered just how much #caffeine may be lethal? http://t.co/SM8QrPZT (of course some are more susceptible than others) #
  • At Cardinia with @donna_williams watching thunderstorm pass over from safety of our car. Quite spectacular! #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-04-22