Planet Linux Australia
Celebrating Australians & Kiwis in the Linux and Free/Open-Source community...

May 28, 2015

Square Rock and Mount Franklin

I'm not really sure why it took me so long to write this set of walks up -- I think I just got lost in preparations for the most recent OpenStack summit and simply forgot. That said, here they are...

Tony, Steven and I mounted an expedition to Mount Franklin, which is one of the trigs I hadn't been to yet. Its right on the ACT border with NSW, and despite not being a super long walk its verging of inaccessible in winter (think several feet of snow). So, we decided to get it done while we could.


Interactive map for this route.

We also tacked on a trip to Square Rock based on the strong recommendation of a good friend. Square Rock has amazing views, highly recommended.


Interactive map for this route.

Tags for this post: blog pictures 20150426-square_rock_franklin photo canberra bushwalk trig_point

Related posts: Goodwin trig; Big Monks; Narrabundah trig and 16 geocaches; Cooleman and Arawang Trigs; One Tree and Painter; A walk around Mount Stranger


May 27, 2015

Las Vegas Style Food Recipes

We interrupt our regular blog posts with a word from our sponsor... LOL

Seriously tough, times are tough in Las Vegas so instead of resorting to standard marketing techniques they've been trying to convince food bloggers (including me) to do their work for them... Just look at the condition of the place! Why would I ever want to go there?

Anyhow, recently someone from (a company that specialises in promoting hotels, restaurants, locations, and other events in Las Vegas) contacted me and asked me to do a take on some of the dishes available in Las Vegas (A copy of the menu is included,

More precisely, dishes from the Aria, Caesars Palace, Bellagio, and The Pallazo. I'm going to take a stab at on a take of a few of these dishes in a way that is inexpensive, quick, and hopefully tasty.

The point of these is to also make them more accessible by substituting ingredients as well (A lot of these ingredients quite simply aren't easily available in other parts of the world and to be honest it's hard to be impressed by something you know little about.).

The following three desserts are designed to be eaten like sundaes.

- ice-cream (vanilla, coffee, or rum-raisan will work best for this)

- crushed peanuts or crushed roasted almonds

- chopped up chocolate bar (Snickers, Picnic, or anything which contains nougat/nuts in it's core. Tip - chop it up in a way that the temperature of the ice cream is unlikely to cause it to freeze hard. Texture/perception of the dish can be changed quite a lot by this)(optional)

- strawberries (or another berry) which have been sliced and left in the fridge in a ice/sugar syrup mix (half an hour is enough. We're just trying to get rid of the extreme tartness of many fresh berries)

- a drizzle of caramel/chocolate/coffee sauce

- cocoa/coffee powder (optional)

Scoop ice cream into bowl or cup. Drizzle other ingredients on top.

- ice-cream (vanilla, coffee, or rum-raisan will work best for this)

- raisins which have been drenched in rum overnight

- crushed peanuts or crushed roasted almonds

- drizzle of caramel/chocolate/coffee sauce

- cocoa/coffee powder (optional)

Scoop ice cream into bowl or cup. Drizzle other ingredients on top.

- ice-cream (vanilla will work best for this)

- some form of cake (can be made or purchased. My preference is towards something darker such as chocolate or coffee flavour. If cooking please cook it so that it is slightly overcooked as it will be mixed with the ice cream. This will stop it from going soggy too quickly and add a bit of texture to the dish).

- some form of alcohol/liquor (we're targetting aroma here. Use whatever you have here but I think rum, cognac, or something else suitably sweet would do well)

Scoop ice cream into bowl or cup. Break up the cake and drop it around in chunks around the ice cream. Drizzle alcohol/liquor around and over the top.

The following is a dessert which is meant to be eaten/drunk like an 'affogato'.

- ice-cream (vanilla will work best for this)

- crushed macaroon biscuits (can be made or purchased. My preference is towards chocolate or coffee flavours. Texture is to be slightly crusty with a chewy interior. Don't bother making the cream if you don't want to)

- a side drunk of coffee, cappucino, late, Milo (chocolate malt) (I'd probably go for a powdered cappucino/late drink which only requires boiling water to be added to get this done quick and tasty)

- cocoa/coffee powder (optional)

Scoop ice cream into bowl or cup. Drizzle other ingredients on top.

The following is obviously is my take on a deluxe steak sandwich.

- sandwich bread slices

- steak

- onions

- lettuce

- tomatoes

- bacon

- cheese

- egg

- tomato sauce

- balsamic vinegar (optional)

- mayonnaise (optional)

- mustard (optional) 

Toast or grill sandwich slices. Add cheese as first layer. Fry an egg and add this as the next layer. Fry some bacon and add this as the next layer. Fry off steak slices with some onion, garlic, salt, sugar, pepper, and maybe a tiny drop of balsamic vinegar (I would probably caramelise this slightly in a pan to remove some of the tartness before adding it to the sandwich or not add it at all) and add this as the next layer. Slice vegetables and add this as the next layer. Use tomato sauce (mayonnaise and/or mustard are optional depending on your taste) on the top layer as it will stop it from drenching the sandwich prior to your having completing preparing it. Season to taste.

The following is more savoury and is obviously meant to be a main meal.

- roasted chicken (can be made or purchased)

- pasta in a white sauce (the 'Bacon and Mushroom Carbonara with Pasta' recipe from, would work well here)

- asparagus

- cheese

- potatoes (use the recipes at, or and remove relevant ingredients (bacon, cream, and cheese for me) to suit the dish)

Cook pasta. Fry asparagus with garlic, butter, oil or else blanch it. Put it in a microwave for a few seconds with a slice of cheese on top to give it a bit of extra flavour (optional). Serve with roasted chicken and fried potatoes. Season dish to taste. You may need to serve this dish with a salad as it can be very rich or fatty depending on your interpretation.


Youtube has done wonders for lots of people, but frankly, my reaction to the vast majority of videos is that they are largely or wholly content free.  Those cases where a visual demonstration actually assists are exceedingly slim (some digital illustration videos for example, but even those don’t necessarily show you what you want). Watching videos of ostensibly informative topics is an exercise in entertainment and almost always a waste of my time.  If you have a transcript at least you can jump around to see if it’s got the info you’re looking for. With videos even if you jump around, you’re still pulling down info at the rate they speak (ie slowly). Next time you watch a documentary count the average number of words spoken in a minute. It’s ridiculously low.

It’s something of a farce that for my CLE requirements I can listen to some 5 year out “senior associate” um and arr through some talk at a firm or do some facile online tutorial (are there other kinds?) and get an hour’s credit, but if I read an entire book by an expert in the area or research the cases myself I get exactly 0 points.

May 26, 2015

So Bill is going to bring a Bill

So Bill Shorten has announced that he and the Deputy Leader of the Opposition, Tanya Plibersec will be putting a bill to the house to allow Same Sex Marriage.

Honestly I'm torn.

The cynical part of me thinks the whole thing is an exercise in futility. Unless the Coalition allows a free vote amongst its members the bill is doomed to die in the House of Reps. If I was going to be really cynical I'd think this was an attempt to take the wind out of the sails of the greens who were proposing a similar bill to start in the Senate.

On the other hand, this is probably the first sign I've seen of Shorten actually stepping forward on an issue that hasn't been focus grouped to death. SSM doesn't have universal support within the Labor party (hi Joe deBruyn you reactionary old fart), and by putting his name directly on the bill Shorten is showing some leadership at last.

If you support Same Sex marriage, or as it's known in other parts of the world, Marriage, I'd urge you to let your local MP know how you feel. Do it politely, do it succinctly but make sure you do it. 

If you want to find out if your local MP or Senator supports or opposes SSM this site is a great resource

Blog Catagories: 

May 25, 2015

MrBayes HPC Installation

Mr. Bayes is a program for Bayesian inference and model choice across a wide range of phylogenetic and evolutionary models.

Download, extract. Note that the developers have produced a tarbomb which will require a separate directory created before download. This has been raised as a bug.

Note that more recent versions of MrBayes make much better use of autoconfiguration tools.

cd /usr/local/src/MRBAYES

mkdir mrbayes-3.2.5

cd mrbayes-3.2.5

read more

May 24, 2015

Twitter posts: 2015-05-18 to 2015-05-24

How I Would Solve Plugin Dependencies

lol, I wouldn’t1.

1. If I absolutely had to, I wouldn’t do it the same as Ryan.

WordPress isn’t (and will never be) Linux

ZYpp is the dependency solver used by OpenSUSE (and its PHP port in Composer), it was born of the need to solve complex dependency trees. The good news is, WordPress doesn’t have the same problem, and we shouldn’t create that problem for ourselves.

One of the most common-yet-complex issues is determining how to handle different version requirements by different packages. If My Amazing Plugin requires WP-API 1.9, but Your Wonderful Plugin requires WP-API 2.0, we have a problem. There are two ways to solve it – Windows solves it by installing multiple versions of the dependency, and loading the correct version for each package. This isn’t a particularly viable option in PHP, because trying to load two different versions of the same code in the same process is not my idea of a fun time.

The second option, which ZYpp solves, is to try and find a mutually compatible version of the dependency that each plugin can use. The biggest problem with this method is that it can’t always find a solution. If there’s no compatible way of installing the libraries, it has to throw back to the user to make the decision. This isn’t a viable option, as 99.999*% worth of users wouldn’t be able to tell the difference between WP-API versions 1.9 and 2.0, and nor should they.

But there’s a third option.

Technical Debt as a Service

Code libraries are, by their nature, developer facing. A user never really needs to know that they exist, in the same way that they don’t need to know about WP_Query. In WordPress Core, we strive for (and often achieve) 100% backwards compatibility between major versions. If we were going to implement plugin dependencies, I would make it a requirement that the code libraries shoulder the same burden: don’t make a user choose between upgrades, just always keep the code backwards compatible. If you need to make architectural changes, include a backwards compatible shim to keep things working nicely.

This intentionally moves the burden of upgrading to the developer, rather than the end user.

What Version?

If we’re going to require library developers to maintain backwards compatibility, we can do away with version requirements (and thus, removing the need for a dependency solver). If a plugin needs a library, it can just specify the library slug.

Better Living Through Auto Updates

If we were to implement plugin dependencies, I think it’d be a great place to introduce auto updates being enable by default. There’s no existing architecture for us to take into account, so we can have this use the current WordPress best practices. On top of that, it’s a step towards enabling auto updates for all Core releases, and it encourages developers to create backwards compatible libraries, because their library will almost certainly be updated before a plugin using it is.

Let’s Wrap This Up

I’m still not convinced plugin dependencies is a good thing to put in Core – it introduces significant complexities to plugin updates, as well as adding another dependency on to Core. But it’s definitely a conversation worth having.

May 23, 2015

Usual Debian Server Setup

I manage a few servers for myself, friends and family as well as for the Libravatar project. Here is how I customize recent releases of Debian on those servers.

Hardware tests

apt-get install memtest86+ smartmontools e2fsprogs

Prior to spending any time configuring a new physical server, I like to ensure that the hardware is fine.

To check memory, I boot into memtest86+ from the grub menu and let it run overnight.

Then I check the hard drives using:

smartctl -t long /dev/sdX
badblocks -swo badblocks.out /dev/sdX


apt-get install etckeepr git sudo vim

To keep track of the configuration changes I make in /etc/, I use etckeeper to keep that directory in a git repository and make the following changes to the default /etc/etckeeper/etckeeper.conf:

  • turn off daily auto-commits
  • turn off auto-commits before package installs

To get more control over the various packages I install, I change the default debconf level to medium:

dpkg-reconfigure debconf

Since I use vim for all of my configuration file editing, I make it the default editor:

update-alternatives --config editor


apt-get install openssh-server mosh fail2ban

Since most of my servers are set to UTC time, I like to use my local timezone when sshing into them. Looking at file timestamps is much less confusing that way.

I also ensure that the locale I use is available on the server by adding it the list of generated locales:

dpkg-reconfigure locales

Other than that, I harden the ssh configuration and end up with the following settings in /etc/ssh/sshd_config (jessie):

HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key


UsePrivilegeSeparation sandbox

AuthenticationMethods publickey
PasswordAuthentication no
PermitRootLogin no

AcceptEnv LANG LC_* TZ
AllowGroups sshuser

or the following for wheezy servers:

HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
KexAlgorithms ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group-exchange-sha256
Ciphers aes256-ctr,aes192-ctr,aes128-ctr
MACs hmac-sha2-512,hmac-sha2-256

On those servers where I need duplicity/paramiko to work, I also add the following:

KexAlgorithms ...,diffie-hellman-group-exchange-sha1
MACs ...,hmac-sha1

Then I remove the "Accepted" filter in /etc/logcheck/ignore.d.server/ssh (first line) to get a notification whenever anybody successfully logs into my server.

I also create a new group and add the users that need ssh access to it:

addgroup sshuser
adduser francois sshuser

and add a timeout for root sessions by putting this in /root/.bash_profile:


Security checks

apt-get install logcheck logcheck-database fcheck tiger debsums corekeeper
apt-get remove john john-data rpcbind tripwire

Logcheck is the main tool I use to keep an eye on log files, which is why I add a few additional log files to the default list in /etc/logcheck/logcheck.logfiles:


while ensuring that the apache logfiles are readable by logcheck:

chmod a+rx /var/log/apache2
chmod a+r /var/log/apache2/*

and fixing the log rotation configuration by adding the following to /etc/logrotate.d/apache2:

create 644 root adm

I also modify the main logcheck configuration file (/etc/logcheck/logcheck.conf):


Other than that, I enable daily checks in /etc/default/debsums and customize a few tiger settings in /etc/tiger/tigerrc:

Tiger_Running_Procs='rsyslogd cron atd /usr/sbin/apache2 postgres'

General hardening

apt-get install harden-clients harden-environment harden-servers apparmor apparmor-profiles apparmor-profiles-extra

While the harden packages are configuration-free, AppArmor must be manually enabled:

perl -pi -e 's,GRUB_CMDLINE_LINUX="(.*)"$,GRUB_CMDLINE_LINUX="$1 apparmor=1 security=apparmor",' /etc/default/grub

Entropy and timekeeping

apt-get install haveged rng-tools ntp

To keep the system clock accurate and increase the amount of entropy available to the server, I install the above packages and add the tpm_rng module to /etc/modules.

Preventing mistakes

apt-get install molly-guard safe-rm sl

The above packages are all about catching mistakes (such as accidental deletions). However, in order to extend the molly-guard protection to mosh sessions, one needs to manually apply a patch.

Package updates

apt-get install apticron unattended-upgrades deborphan debfoster apt-listchanges update-notifier-common aptitude popularity-contest

These tools help me keep packages up to date and remove unnecessary or obsolete packages from servers. On Rackspace servers, a small configuration change is needed to automatically update the monitoring tools.

In addition to this, I use the update-notifier-common package along with the following cronjob in /etc/cron.daily/reboot-required:

cat /var/run/reboot-required 2> /dev/null || true

to send me a notification whenever a kernel update requires a reboot to take effect.

Handy utilities

apt-get install renameutils atool iotop sysstat lsof mtr-tiny

Most of these tools are configure-free, except for sysstat, which requires enabling data collection in /etc/default/sysstat to be useful.

Apache configuration

apt-get install apache2-mpm-event

While configuring apache is often specific to each server and the services that will be running on it, there are a few common changes I make.

I enable these in /etc/apache2/conf.d/security:

<Directory />
    AllowOverride None
    Order Deny,Allow
    Deny from all
ServerTokens Prod
ServerSignature Off

and remove cgi-bin directives from /etc/apache2/sites-enabled/000-default.

I also create a new /etc/apache2/conf.d/servername which contains:

ServerName machine_hostname


apt-get install postfix

Configuring mail properly is tricky but the following has worked for me.

In /etc/hostname, put the bare hostname (no domain), but in /etc/mailname put the fully qualified hostname.

Change the following in /etc/postfix/

inet_interfaces = loopback-only
myhostname = (fully qualified hostname)
smtp_tls_security_level = may
smtp_tls_protocols = !SSLv2, !SSLv3

Set the following aliases in /etc/aliases:

  • set francois as the destination of root emails
  • set an external email address for francois
  • set root as the destination for www-data emails

before running newaliases to update the aliases database.

Create a new cronjob (/etc/cron.hourly/checkmail):

ls /var/mail

to ensure that email doesn't accumulate unmonitored on this box.

Finally, set reverse DNS for the server's IPv4 and IPv6 addresses and then test the whole setup using mail root.

Network tuning

To reduce the server's contribution to bufferbloat I change the default kernel queueing discipline (jessie or later) by putting the following in /etc/sysctl.conf:


May 22, 2015

General Atomic and Molecular Electronic Structure System HPC Installation

GAMESS (General Atomic and Molecular Electronic Structure System (GAMESS)) is a general ab initio quantum chemistry package. You will need to agree to the license prior to download, which will provide a link to gamess-current.tar.gz

Download and extract, load the environment variables for atlas and gcc.

cd /usr/local/src/

tar gamess-current.tar.gz

cd gamess

module load atlas/3.10.2

module load gcc/4.9.1

read more

Craige McWhirter: How To Resolve a Volume is Busy Error on Cinder With Ceph Block Storage

When deleting a volume in OpenStack you may sometimes get an error message stating that Cinder was unable to delete the volume because the volume was busy:

2015-05-21 23:31:41.160 16911 ERROR cinder.volume.manager [req-6f77ef4d-bbff-4ff4-8a3e-4c6b264ac5ca \
04b7cb61dd3f4f2f8f80bbd9833addbd 5903e3bda1e840d492fe79fb840acacc - - -] Cannot delete volume \
f8867d43-bc82-404e-bcf5-6d345c32269e: volume is busy

There are a number of reasons why a volume may be reported by Ceph as busy, however the most common reason in my experience has been that a Cinder client connection has not yet been closed, possibly because a client crashed.

If you were to look at the volume in Cinder, that status is usually available, the record looks in order. When you check Ceph, you'll see that the volume still exists there too.

% cinder show f8867d43-bc82-404e-bcf5-6d345c32269e | grep status
|    status    |    available    |

 # rbd -p my.ceph.cinder.pool ls | grep f8867d43-bc82-404e-bcf5-6d345c32269e

Perhaps there's a lock on this volume. Let's check for locks and then remove them if we find one:

# rbd lock list my.ceph.cinder.pool/volume-f8867d43-bc82-404e-bcf5-6d345c32269e

If there are any locks on the volume, you can use lock remove using the id and locker from the previous command to delete the lock:

# rbd lock remove <image-name> <id> <locker>

What if there are no locks on the volume but you're still unable to delete it from either Cinder or Ceph? Let's check for snapshots:

# rbd -p my.ceph.cinder.pool snap ls volume-f8867d43-bc82-404e-bcf5-6d345c32269e
SNAPID NAME                                              SIZE
  2072 snapshot-33c4309a-d5f7-4ae1-946d-66ba4f5cdce3 25600 MB

When you attempt to delete that snapshot you will get the following:

# rbd snap rm my.ceph.cinder.pool/volume-f8867d43-bc82-404e-bcf5-6d345c32269e@snapshot-33c4309a-d5f7-4ae1-946d-66ba4f5cdce3
rbd: snapshot 'snapshot-33c4309a-d5f7-4ae1-946d-66ba4f5cdce3' is protected from removal.
2015-05-22 01:21:52.504966 7f864f71c880 -1 librbd: removing snapshot from header failed: (16) Device or resource busy

This reveals that it was the snapshot that was busy and locked all along.

Now we need to unprotect the snapshot:

# rbd snap unprotect my.ceph.cinder.pool/volume-f8867d43-bc82-404e-bcf5-6d345c32269e@snapshot-33c4309a-d5f7-4ae1-946d-66ba4f5cdce3

You should now be able to delete the volume and it's snapshot via Cinder.

Enjoy :-)

May 21, 2015

JAGS (Just Another Gibbs Sampler) Installation

JAGS is Just Another Gibbs Sampler. It is a program for analysis of Bayesian hierarchical models using Markov Chain Monte Carlo (MCMC) simulation not wholly unlike BUGS.

cd /usr/local/src/JAGS


tar xvf JAGS-3.4.0.tar.gz

mv JAGS-3.4.0 jags-3.4.0

cd jags-3.4.0



make check

make install

make installcheck

The config script takes the following form


install=$(basename $(pwd) | sed 's%-%/%')

read more

MuTect Installation

MuTect is a method developed at the Broad Institute for the reliable and accurate identification of somatic point mutations in next generation sequencing data of cancer genomes.

For complete details, please see the publication in Nature Biotechnology:

Cibulskis, K. et al. Sensitive detection of somatic point mutations in impure and heterogeneous cancer samples. Nat Biotechnology (2013).doi:10.1038/nbt.2514

Download after login.

read more

PROJ.4 Cartographic Projections library installation

The PROJ.4 Cartographic Projections library was originally written by Gerald Evenden then of the USGS.

Download, extract, install.

cd /usr/local/src/PROJ


tar xvf proj-4.9.1.tar.gz

cd proj-4.9.1



make check

make install

The config file is a quick executable.


./configure --prefix=/usr/local/$(basename $(pwd) | sed 's#-#/#')

read more

Geospatial Data Abstraction Library Installation

GDAL (Geospatial Data Abstraction Library) is a translator library for raster and vector geospatial data formats.

Download, extract, install.

cd /usr/local/src/GDAL


tar gdal-1.11.2.tar.gz

cd gdal-1.11.2



make install

The config file is a quick executable.


./configure --prefix=/usr/local/$(basename $(pwd) | sed 's#-#/#')

read more

Rosetta Proteins with SCons (and jam and cream)

Rosetta is a library based object-oriented software suite which provides a robust system for predicting and designing protein structures, protein folding mechanisms, and protein-protein interactions.

You'll need a license

Download, extract, load scons, and compile.

cd /usr/local/src/ROSETTA

tar xvf rosetta_src_2015.19.57819_bundle.tgz

cd rosetta_src_2015.19.57819_bundle/main/src

module load scons


read more

SCons with Modules

SCons is a software construction tool (build tool, or make tool) implemented in Python, that uses Python scripts as "configuration files" for software builds.

cd /usr/local/src/SCONS


tar xvf scons-2.3.4.tar.gz

cd scons-2.3.4

python install --prefix=/usr/local/scons/2.3.4

Change to the appropriate modules directory, check for .desc and .version and .base, create a symblink to .base

cd /usr/local/Modules/modulefiles/scons

ln -s .base 2.3.4

read more

Freesufer cluster installation

Freesurfer is a set of tools for analysis and visualization of structural and functional brain imaging data.

Check system requirements and download. Note that registration and a license key is required for functionality, but not installation.

Create a source directory, change to it, download, extract, discover that everything is bundled, create the application directory and move everything across.

read more

May 20, 2015

Movement at the Angry Beanie station

Good news everybody!

This week I've started pulling everything together to bring both For Science! and Purser Explores The World back to the internet airwaves :)

I won't reveal what the return episode of Purser Explores The World is going to be about, but suffice to say it's going to continue the same explorations and interview style that previous episodes had.

For Science! of course is going to be the return of Mel, Mags and I doing our thing about science news and getting our rant on (well Mags and Mel more than me but anyway). I'm also going to be looking at either expanding the show to include a new segment or create a smaller podcast that will be talking to researchers around the country, not more than say 15 or 20 minutes long in which we find out a bit more about the work the researcher is doing, how they got started in science and so on.

I have some other thoughts about Angry Beanie and its direction, but they are for another blog post I think.

Blog Catagories: 

APM:Plane 3.3.0 released

APM:Plane 3.3.0 released

The ardupilot development team is proud to announce the release of version 3.3.0 of APM:Plane. This is a major release with a lot of changes. Please read the release notes carefully!

The last stable release was 3 months ago, and since that time we have applied over 1200 changes to the code. It has been a period of very rapid development for ArduPilot. Explaining all of the changes that have been made would take far too long, so I've chosen some key changes to explain in detail, and listed the most important secondary changes in a short form. Please ask for details if there is a change you see listed that you want some more information on.

Arming Changes

This is the first release of APM:Plane where ARMING_CHECK and ARMING_REQUIRE both default to enabled. That means when you upgrade if you didn't previously have arming enabled you will need to learn about arming your plane.

Please see this page for more information on arming:

I know many users will be tempted to disable the arming checks, but please don't do that without careful thought. The arming checks are an important part of ensuring the aircraft is ready to fly, and a common cause of flight problems is to takeoff before ArduPilot is ready.

Re-do Accelerometer Calibration

Due to a change in the maximum accelerometer range on the Pixhawk all users must re-do their accelerometer calibration for this release. If you don't then your plane will fail to arm with a message saying that you have not calibrated the accelerometers.

Only 3D accel calibration

The old "1D" accelerometer calibration method has now been removed, so you must use the 3D accelerometer calibration method. The old method was removed because a significant number of users had poor flights due to scaling and offset errors on their accelerometers when they used the 1D method. My apologies for people with very large aircraft who find the 3D method difficult.

Note that you can do the accelerometer calibration with the autopilot outside the aircraft which can make things easier for large aircraft.


After an auto-landing the autopilot will now by default disarm after LAND_DISARMDELAY seconds (with a default of 20 seconds). This feature is to prevent the motor from spinning up unexpectedly on the ground

after a landing.

HIL_MODE parameter

It is now possible to configure your autopilot for hardware in the loop simulation without loading a special firmware. Just set the parameter HIL_MODE to 1 and this will enable HIL for any autopilot. This is designed to make it easier for users to try HIL without having to find a HIL firmware.

SITL on Windows

The SITL software in the loop simulation system has been completely rewritten for this release. A major change is to make it possible to run SITL on native windows without needing a Linux virtual machine. There should be a release of MissionPlanner for Windows soon which will make it easy to launch a SITL instance.

The SITL changes also include new backends, including the CRRCSim flight simulator. This gives us a much wider range of aircraft we can use for SITL. See for more information.

Throttle control on takeoff

A number of users had problems with pitch control on auto-takeoff, and with the aircraft exceeding its target speed during takeoff. The auto-takeoff code has now been changed to use the normal TECS throttle control which should solve this problem.

Rudder only support

There is a new RUDDER_ONLY parameter for aircraft without ailerons, where roll is controlled by the rudder. Please see the documentation for more information on flying with a rudder only aircraft: ... udder_only

APM1/APM2 Support

We have managed to keep support for the APM1 and APM2 in this release, but in order to fit it in the limited flash space we had to disable some more features when building for those boards. For this release the AP_Mount code for controlling camera mounts is disabled on APM1/APM2.

At some point soon it will become impractical to keep supporting the APM1/APM2 for planes. Please consider moving to a 32 bit autopilot soon if you are still using an APM1 or APM2.

New INS code

There have been a lot of changes to the gyro and accelerometer handling for this release. The accelerometer range on the Pixhawk has been changed to 16g from 8g to prevent clipping on high vibration aircraft, and the sampling rate on the lsm303d has been increased to 1600Hz.

An important bug has also been fixed which caused aliasing in the sampling process from the accelerometers. That bug could cause attitude errors in high vibration environments.

Numerous Landing Changes

Once again there have been a lot of improvements to the automatic landing support. Perhaps most important is the introduction of a smooth transition from landing approach to the flare, which reduces the tendency to pitch up too much on flare.

There is also a new parameter TECS_LAND_PMAX which controls the maximum pitch during landing. This defaults to 10 degrees, but for many aircraft a smaller value may be appropriate. Reduce it to 5 degrees if you find you still get too much pitch up during the flare.

Other secondary changes in this release include:

  • a new SerialManager library which gives much more flexible management of serial port assignment
  • changed the default FS_LONG_TIMEOUT to 5 seconds
  • raised default IMAX for roll/pitch to 3000
  • lowered default L1 navigation period to 20
  • new BRD_SBUS_OUT parameter to enable SBUS output on Pixhawk
  • large improvements to the internals of PX4Firmware/PX4NuttX for better performance
  • auto-formatting of microSD cards if they can't be mounted on boot (PX4/Pixhawk only)
  • a new PWM based driver for the PulsedLight Lidar to avoid issues with the I2C interface
  • fixed throttle forcing to zero when disarmed
  • only reset mission on disarm if not in AUTO mode
  • much better handling of steep landings
  • added smooth transition in landing flare
  • added HIL_MODE parameter for HIL without a special firmware
  • lowered default FS_LONG_TIMEOUT to 5 seconds
  • mark old ELEVON_MIXING mode as deprecated
  • fixed 50Hz MAVLink support
  • support DO_SET_HOME MAVLink command
  • fixed larger values of TKOFF_THR_DELAY
  • allow PulsedLight Lidar to be disabled at a given height
  • fixed bungee launch (long throttle delay)
  • fixed a bug handling entering AUTO mode before we have GPS lock
  • added CLI_ENABLED parameter
  • removed 1D accel calibration
  • added EKF_STATUS_REPORT MAVLink message
  • added INITIAL_MODE parameter
  • added TRIM_RC_AT_START parameter
  • added auto-disarm after landing (LAND_DISARMDELAY)
  • added LOCAL_POSITION_NED MAVLink message
  • avoid triggering a fence breach in final stage of landing
  • rebuild glide slope if we are above it and climbing
  • use TECS to control throttle on takeoff
  • added RUDDER_ONLY parameter to better support planes with no ailerons
  • updated Piksi RTK GPS driver
  • improved support for GPS data injection (for Piksi RTK GPS)
  • added NAV_LOITER_TO_ALT mission item
  • fixed landing approach without an airspeed sensor
  • support RTL_AUTOLAND=2 for landing without coming to home first
  • disabled camera mount support on APM1/APM2
  • added support for SToRM32 and Alexmos camera gimbals
  • added support for Jaimes mavlink enabled gimbal
  • improved EKF default tuning for planes
  • updated support for NavIO and NavIO+ boards
  • updated support for VRBrain boards
  • fixes for realtime threads on Linux
  • added simulated sensor lag for baro and mag in SITL
  • made it possible to build SITL for native Windows
  • switched to faster accel sampling on Pixhawk
  • added coning corrections on Pixhawk
  • set ARMING_CHECK to 1 by default
  • disable NMEA and SiRF GPS on APM1/APM2
  • support MPU9255 IMU on Linux
  • updates to BBBMINI port for Linux
  • added TECS_LAND_PMAX parameter
  • switched to synthetic clock in SITL
  • support CRRCSim FDM backend in SITL
  • new general purpose replay parsing code
  • switched to 16g accel range in Pixhawk
  • added FENCE_AUTOENABLE=2 for disabling just fence floor
  • added POS dataflash log message
  • changed GUIDED behaviour to match copter
  • added support for a 4th MAVLink channel
  • support setting AHRS_TRIM in preflight calibration
  • fixed a PX4 mixer out of range error

Many thanks to everyone who contributed to this release. We have a lot of new developers contributing which is really great to see! Also, apologies for those who have contributed a pull request but not yet had it incorporated (or had feedback on the change). We will be trying to get to as many PRs as we can soon.

Best wishes to all APM:Plane users from the dev team, and happy flying!

May 18, 2015

Learning to Cook

I recently noticed a significant spike in traffic to this blog and it's become pretty obvious why. The food recipes... If you're curious why they've been going up online I'm a firm believer in the following philosophy.

Only wimps use tape backup: real men just upload their important stuff on ftp, and let the rest of the world mirror it ;)

Seriously though, I have a tendency to lose things sometimes and thought that posting it here would be my best chance of never losing my them. Since it needed to be presented in public it would also mean that it would force me into writing more complete recipes rather than simply scrawling down whatever seemed pertinent at the time. (I never thought that I would be presented with opportunities through this. More on this later.)

In spite of all this, you're probably wondering why the recipes lack a bit of detail still and how I ended up with this particular style of cooking.

As you can guess from my name, I have an asian (Vietnamese to be more precise) background. Growing up I learnt that our cooking was often extremely tedious, required a lot of preparation, tasted great but often didn't fill me. Ultimately, this meant that my family wanted me to spend less time helping in the kitchen and more time tending to me studies. To a certain extent, this family policy has served us well. Many of the kids are well educated and have done well professionally.

The problem is that if you've ever worked worked a standard week over any period of time then you ultimately realise that a lot of the time you don't want to spend heaps of time cooking whether for yourself or for others (this style doesn't work long term). 

This is where I radically differ from my family. Many of them see cooking as a necessary chore (who wants to die, right? :-)) and they labour over it or else they love it with such a passion that they lose sight of the fact that there's only 24 hours in a day (there are/have been some professional chefs in the family). Ultimately, they end up wearing themselves out day after day but I've learnt to strip back recipes to their core flavours so that I can cook decent tasting food in reasonable amounts of time.

Like others, I went through multiple phases from a culinary perspective. As a child I loved to eat most things thrown at me (but my family didn't want me in the kitchen). In my teenage years, I used to enjoy and revel in fast and fatty foods but basically grew out of it as I discovered that it wasn't all that filling and could result in poor health. Just like the protaganist of 'Supersize Me' I found out that some of my bodily functions didn't work quite as well on this particular diet.

Eating out was much the same because they often added unhealthy elements to meals (high levels of MSG, sugar, salt, etc... to boost the taste). Not to mention the fact, that serving sizes could sometimes be low and prices relatively high. I basically had no choice but to learn to cook for myself. In the beginning, I began trying to reproduce restaurant meals badly. I didn't have the reportoire to be able to reproduce and balance flavours well enough to do a half decent job. Over time, I spent more time exploring cheat restaurants, diners, etc... around where I studied and/or worked. I also watched, read, and in general spent more time in the grocer just trying random sauces, spices, and so on... I developed a sense of flavour and how to achieve them from base ingredients.

This is why none of the recipes contain exact amounts of ingredients (at the moment). It's also because that was the way I learnt to cook (I was taught a bit by some of my aunts), some of the lesser talented members of the family had a tendency to fiddle constantly so listing amounts was basically useless, some people (family or not) aren't willing to share ingredients so you just have to figure it out when and if you have to, and finally I figured out that it was the easiest way for me to learn to cook. When you look at a recipe, you're often doing mental arithmetic in order to make it 'taste right'. By developing a better sense of taste I could mostly forgo this and not have to suffer the consequences of a mathematical screw up (it happened enough times in the family for me to learn to not become so reliant on it).

In general my perspective with regards to food are the following:
  • kids will eventually learn what fills them and fast food will make them feel like horrible. They will grow out of it and eat properly eventually if they are exposed to the right foods
  • rely on machinery when you can. Why waste you're time cutting food perfectly if you can get it done in a fraction of the time using the right equipment?
  • why bother with perfection if you can achieve 95% of the taste and 50% apparent effort
  • I'd much rather spend time enjoying food than cooking it
  • prior to marinating any piece of meat I create the core sauce/marinade seperately first and then add the meat. There's no chance of food posioning and I get to have an idea what it will taste like
  • balance of flavours is more important than exact amounts over and over again. You may have a different preference from time to time also. Obviously, the converse is also true. Exact amounts give you a basis from which to work from
  • don't think that more resources will make you a better chef. It's possible that the exact opposite is true at times. Think about the food of the wealthy versus that of the poor. The poor have to make the most of everything that is thrown at them, extracting every last single ounce of flavour from something small/cheap while the wealthy have the basically mix and match the very best each and every time. From a chef's perspective this means that they don't have the chance to understand flavours at a more elemental/core level
  • shop from specialist butchers, fishmongers, etc... they will often be able to get you unusual cuts/meats, have better knowledge, do extra things like cutting down large bones for soup stocks and they are also often quite a bit cheaper
  • don't freeze if you can avoid it (or at least avoid freezing some foods). Some people I know use it as a technique to save time. For some dishes this is true but for others it can alter the actual structure (and sometimes faste. Think about soups versus meats when they are dethawed correctly and incorrectly.) of the food involved leaving it a mess when you finally prepare and eat it
  • fresh means fresh. Leave fish (and some meats) in the fridge for even a day after leaving the better/stable environment at a supermarket or fishmonger and it will begin to smell and taste slightly rank. This effect increases exponentially over time
  • try everything whether that be sauces, spices, restaurants, cultures, etc... You will find cheap opportunties if you go to the right places and ultimately you will end up healther (you learn that better tasting food is often healther as well), happier (variety is the spice of life), and possibly wealthier because of it (you can save a lot by learning to cook well). The wider you're vocabulary, the better your cooking will become...
  • balance of flavours as key. Even if you stuff up a recipe you can rescue it if you know enough about this. Added too much sugar? Use sourness to balance it out, etc...
  • don't learn from a single source. If you learn purely through celebrity chefs and books you'll realise that a lot of what they do is quite gimmicky. A lot of the ingredients that they use aren't very accessible, expensive, in spite of what they say. Use your head to strip the recipes back to core flavours to save you time and money (in procuring them)
  • learning to cook well will take time. Have patience. It took me a long while before I could build a sufficient 'vocabulary' before I could build dishes that were worth staying at home for. It took me more time to learn how to reverse engineer dishes at restaurants. Use every resource at your disposal (the Internet has heaps of free information, remember?).
On a side note, based on the contents of my blog (and other places) people have semi-regularly requested to write here and for me to write for them. I'm more than happy to do this providing I have the time and the task is interesting enough... on any topic.

May 17, 2015

[debian] Fixing some issues with

I got an email last year pointing out a cosmetic issue with I think at the time of the email, the only problem was some bitrot in PHP's built-in server variables making some text appear incorrectly.

I duly added something to my TODO list to fix it, and it subsequently sat there for like 13 months. In the ensuing time, Debian changed some stuff, and my code started incorrectly handling a 302 as well, which actually broke it good and proper.

I finally got around to fixing it.

I also fixed a problem where sometimes there can be multiple entries in the Sources file for a package (switching to using would also address this), which caused sometimes caused an incorrect version of the changelog to be returned.

In the resulting tinkering, I learned about, which is totally awesome. I could stop maintaining and parsing a local copy of sid's Sources file, and just make a call to this instead.

Finally, I added linking to CVEs, because it was a quick thing to do, and adds value.

In light of, I'm very tempted to rewrite the redirector. The code is very old and hard for present-day Andrew to maintain, and I despise PHP. I'd rather write it in Python today, with some proper test coverage. I could also potentially host it on AppEngine instead of locally, just so I get some experience with AppEngine

It's also been suggested that I fold the changes into the changelog hosting on I'm hesitant to do this, as it would require changing the output from plain text to HTML, which would mess up consumers of the plain text (like the current implementation of

Twitter posts: 2015-05-11 to 2015-05-17

May 15, 2015

Lower SNR limit of Digital Voice

I’m currently working on a Digital Voice (DV) mode that will work at negative SNRs. So I started thinking about where the theoretical limits are:

  1. Lets assume we have a really good rate 0.5 FEC code that approaches the Shannon Limit of perfectly correcting random bit errors up to a channel BER of 12%
  2. A real-world code this good requires a FEC frame size of 1000′s of bits which will mean long latency (seconds). Lets assume that’s OK.
  3. A large frame size with perfect error correction means we can use a really low bit rate speech codec that can analyse seconds of speech at a time and remove all sorts of redundant information (like silence). This will allow us to code more efficiently and lower the bit rate. Also, we only want speech quality just on the limits of intelligibility. So lets assume a 300 bit/s speech codec.
  4. Using rate 0.5 FEC that’s a bit rate over the channel of 600 bit/s.
  5. Lets assume QPSK on a AWGN channel. It’s possible to make a fading channel behave like a AWGN channel if we use diversity, e.g. a long code with interleaving (time diversity), or spread spectrum (frequency diversity).
  6. QPSK at around 12% BER requires an Eb/No of -1dB or an Es/No of Eb/No + 3 = 2dB. If the bit rate is 600 bit/s the QPSK symbol rate is 300 symbols/s

So we have SNR = Es/No – 10*log10(NoiseBW/SymbolRate) = 2 – 10*log10(3000/300) = -8dB. Untrained operators find SSB very hard to use beneath 6dB, however I imagine many Ham contacts (especially brief exchanges of callsigns and signal reports) are made well beneath that. DV at -8dB would be completely noise free, but of low quality (e.g. a little robotic) and high latency.

For VHF applications C/No is a more suitable measurement, this is a C/No = SNR – 10*log10(3000) = 26.7dBHz (FM is a very scratchy readability 5 at around 43dBHz). That’s roughly a 20dB (100 x) power improvement over FM!

May 14, 2015

Leaking Information in Drupal URLs

Update: It turns out the DA was trolling. We all now know that DrupalCon North America 2017 will be in New Orleans. I've kept this post up as I believe the information about handling unpublished nodes is relevant. I have also learned that m4032404 is enabled by default in govCMS.

When a user doesn't have access to content in Drupal a 403 forbidden response is returned. This is the case out of the box for unpublished content. The problem with this is that sensitive information may be contained in the URL. A great example of this the DrupalCon site.

The way to avoid this is to use the m4032404 module which changes a 403 response to a 404. This simple module prevents your site leaking information via URLs.

DrupalCon-Philadephia.png139.21 KB


in the grit and sunbaked red

you can imagine moonscapes,

endless hot dry emptiness

but the ants commute even on this hot sand

lizards patrol their freeways with quick tongues

improbable silvergrey leaves stand isolated

sand and sticks collecting under the windward edge of any plant.

at dusk the restless kangaroos cross the landscape

in the evenings there are yowls of dingoes

dense clouds of insects orbit the lights

dawn is patterned with tracks

and if it rains

a magician's bouquet

life explodes, instant spring packetmix

just add water.

[tech] LWN Chrome extension published

I finally got around to finishing off and publishing the LWN Chrome extension that I wrote a couple of months ago.

I received one piece of feedback from someone who read my blog via Planet Debian, but didn't appear to email me from a usable email address, so I'll respond to the criticisms here.

I wrote a Chrome extension because I use Google Chrome. To the best of my knowledge, it will work with Chromium as well, but as I've never used it, I can't really say for sure. I've chosen to licence the source under the Apache Licence, and make it freely available. So the extension is available to anyone who cares to download the source and "side load" it, if they don't want to use the Chrome Web Store.

As for whether a userscript would have done the job, maybe, but I have no experience with them.

Basically, I had an itch, and I scratched it, for the browser I choose to use, and I also chose to share it freely.

May 13, 2015

Keeping Songs Fresh

If you've been keeping an eye on this blog then you would have noticed that I've been dabbling in music production of late. This is part of something that I've been working on, on and off now while I've been learning more about the craft.

This is a playlist that I've created to upload such things.

Clearly, the track sample outlined above is fairly early in it's inception but it gives you an idea of some of the stuff that I am likely to produce in future.

As to the purpose of this particular post, it's basically about how to keep a song fresh by altering various aspects of it. For instance, think about the following:
  • alter tempo (don't restrict yourself to a single tempo throughout. Listen to grid music specialists (such as 'Jeremy Ellis', and finger drummers (such as 'Mad Zach',, and you'll see that the sound is a lot more natural re is a lot to be gained by not adhering to stringently to tr
  • change key/scales (if you're aware of enough music theory you'll be aware that by altering 'modes/scales' you can change the entire feel of songs through that alone. Also remember that in the world of artificial sounds such as that produced by synthesisers scales can sometimes mean very little. Just go by ear in such cases...)
  • alter instruments for the same section (it's astonishing how much variety in software and ourboard gear you can get. Even if you just work with free stuff you'll have more than enough to build quality songs). While we're at it, give each and every instrument a chance. An example of this 'Doctor Rockit' in 'Cafe De Flore',
  • the human voice (even when re-modulated/synthesised) can completely alter the feel of a song. The timbre itself can sort of be reproduced by artificial means but not quite yet which means you lose out on a lot by rejecting it. Listen to 'Kayne West's' version of 'Harder, Better, Faster Stronger', as opposed to the original version and you'll understand what I mean,
  • if you have difficulty in finding an vocalist try specific social networks for this such as, and
  • else just become really good with instruments such as 'Chicane' in 'Offshore',
  • alter phase/time between tracks (slight changes in phase can have quite a different effect)
  • alter notes and their sequence (sounds obvious but doesn't seem to be sound obvious at times particularly when listening to heaps of club/dance music).
  • which leads us to the following point, learn to improvise and harmonise. I grew up on a lot of RnB and Hip Hop but ended up brancing out. Without this basis you'll find it very difficult to make something that doesn't sound overly repetitive. Examples of great harmony include, 'Boyz II Men' in 'End of The Road',, 'Four Seasons of Loneliness',, and 'I'll Make Love to You',
  • play around with the usual effects mid sound such as envelopes, modulation, LFO's, phasing, flanging, etc... A good example of this is with 'Flume',
  • use of polyrhythms. Can be a little bit confusing to work with but can also achieve good results,
  • use of effects such as panning, reverb, delay, EQ, etc... (be careful though. If you plan on deploying to clubs remember that their systems are often monophonic so some of your work may be for nothing. Also, a lot of people's standard stereo systems just don't have the range/ability to be able to do what you may want.)
  • use of automation in order to change relative volume of tracks/instruments in relation to one another
  • production and mixing techniques such as side-chain ducking, parallel processing, etc... Note, that sometimes you can go overboard and it can lose a lot of it's body though
  • split, explode, change sequence, ghost, reverse MIDI sections and/or audio samples
  • 'layering' sounds by having instruments play the same MIDI notes/sequences
  • think about push/pull aspects when dealing with 'fills'. Hear this in parts of Groove Aramda's, 'Lovebox',
  • add random file samples/sounds into the mixture every once in a while. A good example and common user of this technique is 'Daft Punk' in 'Around the World',
  • gradually build into sections. Keeps it sounding like a song rather than a bunch of clips that have been assembled together. Also, creates a sense of fluidity. An example of this is 'Bob Sinclair' in 'World, Hold On',
  • this takes me to my next point, take your time when it comes to building a song. I've been dealing with this problem constantly. It's not just a bunch of clips put together. It's like a story. It's composed of words, phrases, pages, and ultimately a book. Tell the story completely. An example of this is 'Tom Novy's' song ' Take It',
  • that said when pushing/pulling/building into different sections one technique you can use to add a bit of 'freshness' is just giving them a hint here and there before hitting them with the complete section
  • think about utilising the entire frequency range. I've heard heaps of songs just cramp their frequency range into too small a range and it ends up losing some expressiveness
  • think about extending notes in breakdowns. A good example of this is 'When the Light's Go Out' by 'Five,
  • good songs start with a solid base. Even if they aren't electronic they start with a solid base/beat and build there way up into something great. Listen to 'Kaskade's' song 'This Rhythm' for an example of this, as well 'Mousse T' in 'Horny',
  • use silence to your advantage. If you're just starting out you think you need to just fill every single moment in time with sound. Silence in the right places can change the entire feel of that particular section
  • don't think that pure digital or analogue is best. Fusing the two can produce wonderful results even if they are emulated via software. An example of this is using 'saturation', 'distortion', whitenoise effects to cut through the artificial/pure nature of the sounds that would otherwise be on show
  • use different sounds as well as effects during section transition. A good example of this by 'Doctor Rockit' in 'Cafe De Flore',
  • listen to heaps of different artists and read a lot. A lot of what I've learnt has actually been from 'Computer Music Magazine' (a lot of content is actually duplicated by other music magazine publishers and articles are often superficially updated by the magazine and re-published. You can save a bit of money by being watchful for these things, Don't limit yourself to keep yourself interested as well as your listeners interested

May 12, 2015

LUV Main June 2015 Meeting: Using deep mutational scanning to understand protein function / Drupal8 out of the box

Jun 2 2015 19:00
Jun 2 2015 21:00
Jun 2 2015 19:00
Jun 2 2015 21:00

200 Victoria St. Carlton VIC 3053


• Alan Rubin: Using deep mutational scanning to understand protein function

• Donna Benjamin: Drupal8 out of the box

200 Victoria St. Carlton VIC 3053 (formerly the EPA building)

Before and/or after each meeting those who are interested are welcome to join other members for dinner. We are open to suggestions for a good place to eat near our venue. Maria's on Peel Street in North Melbourne is currently the most popular place to eat after meetings.

LUV would like to acknowledge Red Hat for their help in obtaining the venue and VPAC for hosting.

Linux Users of Victoria Inc. is an incorporated association, registration number A0040056C.

June 2, 2015 - 19:00

read more

May 11, 2015

Tired but looking up

Ye gods I'm tired.

Mostly I think this is due to the prodigious amount of travel that I've been doing to Melbourne for work, which while interesting, has left me with very little time to get a handle on things that I want to do outside of work.

There is a light on the horizon though, it looks like the Melbourne sojourne is coming to an end, and that means that at the very least psychologically I can start looking at committing to doing things around Angry Beanie that I've been meaning to do for the past six months.

So here's a rundown of the tasks I've set myself.

Rebuild Angry Beanie website using Django. This is mostly an exercise in learning Django and python, both tools I've looked at before but never really had a project to get my teeth into both.

Restart production of For Science and Purser Explores The World. I've been trying to get this restarted all year, but the aforementioned Melbourne trips have really thrown a kink in things.

On the subject of PETW I'd love to hear any subjects that you'd like me to cover. I already have a list of topics I want to look at, but I'm always up for more.

Blog Catagories: 

Upcoming opportunities to talk MySQL/MariaDB in May 2015

May is quickly shaping up to be a month filled with activity in the MySQL/MariaDB space. Just a quick note to talk about where I’ll be; looking forward to meet folk to talk shop. 

  1. The London MySQL Meetup GroupMay 13 2015 – organized by former colleague & friend Ivan Zoratti, we will be doing a wrap up of recent announcements at Percona Live Santa Clara, and I’ll be showing off some of the spiffy new features we are building into MariaDB 10. 
  2. MariaDB Roadshow London – May 19 2015 – I’m going to give an overview of our roadmap, and there will be many excellent talks by colleagues there. I believe MariaDB Corporation CEO Patrik Sallner and Stu Schmidt, President at Zend will also be there. Should be a fun filled day. 
  3. Internet Society (ISOC) Hong Kong World Internet Developer Summit – May 21-22 2015 – I’ll be giving a keynote about MariaDB and how we are trying to make it important Internet infrastructure as well as making it developer friendly. 
  4. O’Reilly Velocity 2015 – May 27-29 2015 – I will in 90 minutes attempt to give a tutorial to attendees (over a 100 have already pre-registered) an overview of MySQL High Availability options and what their choices are in 2015. Expect a lot of talk on replication improvements from both MySQL & MariaDB, Galera Cluster, as well as tools around the ecosystem. 

Oh Hai 👶


Rohan Victor

Rohan Victor Pendergast, born 18:33 on 10/5/15.

Boy and Bear seemed to be what got him moving, so here they are now.

(PS: Should you ever find yourself in such a position, I highly recommend finding a hospital that gives the parents champagne for having a babby on Mother’s Day.)

May 10, 2015

Installing Ubuntu 15.04 on Acer Aspire E 11 E3-112-C6YY

Acer Aspire E 11 E3-112-C6YY is a nice 11 inch notebook which I found suitable for doing some work while commuting. And it costs only A$299 at Dick Smith (or on Ebay). Here is a step by step instruction how to setup Ubuntu 15.04 along with preinstalled Microsoft Windows 8.1 (dual boot):

  1. Before you begin it is better to install all pending updates to Windows 8.1 installed on the notebook.
  2. Download the latest desktop version of Ubuntu for amd64 architecture from, it is vivid-desktop-amd64.iso at this moment. Then create a bootable Ubuntu USB flash drive using Rufus. Choose GPT partition scheme for UEFI computer and Ubuntu ISO image just downloaded. You need an USB drive of 2GB or more. All data on this USB drive would be lost in the process.
  3. Create a recovery drive with the preinstalled Acer eRecovery Management application. Optional, but you take some risk not doing it. You would need an USB drive of 16GB or more, and all present data on that drive would be lost in the process.
  4. Shrink the main Windows partition (drive C:) by the amount you like to allocate for Ubuntu. Minimum required is just about 6.6GB, though I took the maximum possible (about 50% of 500GB drive).
  5. Turn off fast startup in Windows 8.1
  6. Insert USB drive with Ubuntu into USB port.
  7. Disable Secure Boot (select Notebook section at this link), and before exiting BIOS setup utility move the USB HDD to the top of the Boot priority list in the Boot menu. Save changes and exit.
  8. Now you should see the grub menu suggesting to select either try Ubuntu without installation, either install it. You may try it first to ensure everything works fine and then install it.
  9. When you chose to install Ubuntu the first step is to choose the language of the system. It is English by default.
  10. Next step is configuring Internet connection, usually via Wi-Fi. It is better to have it configured and run to be able to install updates during the installation.
  11. Then the installer checks requirements for successful installation: you should have enough free space; be plugged ti the power source; and be connected to the internet. You may tick boxes permitting to install updates and third-party software if you like.
  12. After that you need to select installation type. As we booting from UEFI enabled USB drive, choose the default: Install Ubuntu alongside Windows Boot Manager. Then hit Install Now and then confirm changes to be made. This is point of no return, you should finish installation passing beyond this point.
  13. In the next setup dialogues you choose your Time Zone; your keyboard layout; specify your name and the name of your laptop; choose username and password (if required) to be used.
  14. And now you need to wait until installation complete with the dialogue suggesting reboot the system to start using it. Reboot it and remove USB drive so the laptop would be booted... into Windows. That is expected.
  15. Move the mouse pointer to the bottom left corner and right click on the window icon and choose "Command Prompt (Admin)". Within the administrator's command prompt type the following command:
        bcdedit /set "{bootmgr}" path \EFI\ubuntu\grubx64.efi
  16. Shutdown Windows 8.1 again; power it on and hit F2 once Acer logo appears to enter BIOS setup utility. Move the USB HDD below HDD: xxxxxxx-xxx in the Boot priority list in the Boot menu. Save changes and exit.
  17. Now you should see GRUB menu suggesting to boot either Ubuntu, either Windows Boot Manager. Try to boot first Ubuntu; then Windows 8.1 to verify everything works fine.


Twitter posts: 2015-05-04 to 2015-05-10

UniFi systemd unit file for Ubuntu 15.04

At work we’ve started using some UniFi wireless gear and the system I’ve managed to commandeer to do the control system for it is running Kubuntu 15.04 which uses systemd. Now the UniFi Debian packages don’t supply systemd unit files so I went hunting and found a blog post by Derek Horn about getting it running on CentOS7 so I nabbed his and adapted it for Ubuntu (which wasn’t that hard).

The file lives in /etc/systemd/system/unifi.service and was enabled with systemctl enable unifi.service (from memory, there might have been another step that involved getting systemd to rescan unit files to pick up the new one, but I don’t remember for sure).

Here is the unit file:

# Systemd unit file for unifi-rapid

Description=UniFi Wireless AP Control System

#ExecStart=/usr/bin/java -Xmx1024M -jar /usr/lib/unifi/lib/ace.jar start
ExecStart=/usr/bin/jsvc -nodetach -home /usr/lib/jvm/java-7-openjdk-amd64 -cp /usr/share/java/commons-daemon.jar:/usr/lib/unifi/lib/ace.jar -pidfile /var/run/unifi/ -procname unifi -outfile SYSLOG -errfile SYSLOG -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Xmx1024M com.ubnt.ace.Launcher start
#ExecStop=/usr/bin/java -jar /usr/lib/unifi/lib/ace.jar stop
ExecStop=/usr/bin/jsvc -home /usr/lib/jvm/java-7-openjdk-amd64 -cp /usr/share/java/commons-daemon.jar:/usr/lib/unifi/lib/ace.jar -pidfile /var/run/unifi/ -procname unifi -outfile SYSLOG -errfile SYSLOG -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Xmx1024M -stop com.ubnt.ace.Launcher stop


This item originally posted here:

UniFi systemd unit file for Ubuntu 15.04

May 09, 2015

LUV Beginners May Meeting: Introduction to Ruby

May 16 2015 12:30
May 16 2015 16:30
May 16 2015 12:30
May 16 2015 16:30

RMIT Building 91, 110 Victoria Street, Carlton South

Ruby is a currently popular programming language that is powerful and easy to learn. It supports several major programming techniques: imperative, functional, and object-oriented programming. It has an active community and a large library of components that make it easy to build on the work of others and share your own work. Ruby also makes it easy to write tests to ensure your software works as intended and to include documentation along with your programs.

LUV would like to acknowledge Red Hat for their help in obtaining the Buzzard Lecture Theatre venue and VPAC for hosting.

Linux Users of Victoria Inc., is an incorporated association, registration number A0040056C.

May 16, 2015 - 12:30

read more

May 08, 2015

Linux Security Summit 2015 CFP

The CFP for the 2015 Linux Security Summit (LSS) is now open: see here.

Proposals are due by June 5th, and accepted speaker notifications will go out by June 12th.

LSS 2015 will be held over 20-21 August, in Seattle, WA, USA.

Last year’s event went really well, and we’ll follow a similar format over two days again this year.  We’re co-located again with LinuxCon, and a host of other events including Linux Plumbers, CloudOpen, KVM Forum, and ContainerCon.  We’ve been upgraded to an LF managed event this year, which means we’ll get food.

All LSS attendees, including speakers, must be registered attendees of LinuxCon.   The first round of early registration ends May 29th.

We’d like to cast our net as wide as possible in terms of presentations, so please share this info with anyone you know who’s been doing interesting Linux security development or implementation work recently.

May 07, 2015

Installing Logic 9 Under Mac OS X Under VMWare Under Windows

If you're like many others on the planet you probably can't be bothered purchasing a new piece of software in order to test an application. In this case, I've been trying to get Logic 9 (music DAW software running under VMWare under Windows).

One of the first steps is getting is getting Mac OS X installed under VMWare.

I had some issues with regards to stalling on boot until I tried the following fix inside of the relevant vmx file.

smc.present = "TRUE" -->> smc.present = "FALSE"

Then it's a case of installing the application itself. Sounds simple, but there are a few things we need to do in order to get this to work. First, is circumventing requirements checking so that it will install in spite of not being able to fulfil certain hardware requirements.

1) Mount "Logic Studio.dmg"

2) Copy "Install Logic Studio" package from mounted drive to another folder

3) Right click -> Show original

4) Right click on "Logic Studio.mpkg" -> Show Package Contents

5) Delete "\Contents\Resources\Requirements"

6) Install Logic =)

You'll also need to be able to circumvent the version checking to allow the application to run. The steps are as follows.


1. Open Terminal.
2. Type sudo nano -w /System/Library/CoreServices/SystemVersion.plist
Does the file path look familiar from my last tutorial?
3. Press Enter. You will be prompted for your password. Type it in. You won't see it on your screen. Press Enter again.

4. Use the arrow keys to navigate to the end of the 10.6.2 under ProductUserVisibleVersion and erase the version number. Enter in a version number of your choice.

5. Repeat for the version number displayed under ProductVersion.

6. Press Control key iconX (Control-X) to exit. It will ask you if you wish to "save modified buffer." Press y. It will then ask for a file name to write. Press Enter to accept the default.

7. Exit Terminal.

If you're curious I looked at trying to get Logic X running under Mac OS X but there are substantial intermiate requirements that need to be fulfilled. Moreover, I only have an old version of Mac OS X available so I only tried with Logic 9.

May 06, 2015

Ancillary Justice

ISBN: 9780356502403


I loved this book. The way the language works takes a little while to work out, but then blends into the background. The ideas here are new and interesting and I look forward to other work of Ann's. Very impressed with this book.

Tags for this post: book ann_leckie combat ai aliens

Related posts: Mona Lisa Overdrive; East of the Sun, West of the Moon; Count Zero; Emerald Sea; All The Weyrs of Pern; Against the Tide
Comment Recommend a book


an eternally urgent offer,

to match your fancy and quench your envy

setting a trap for your immediate attention.

a market to engage you

with the unlikely imagery of a commercial imagination

seeding questions and farming responses,

enhance your success, open your purse,

quench your need with our original, natural, blue pill

clinching a sale, a steal! closing a deal.

trusted partners reach into your address book,

praying for support, preying on conscience

any answer is bound to please.

just sometimes,

a paragraph of detractor text, ragged pieces

from vintage novels, snuppets of wholesome language,

a block of literary noise to confuse the filters

these hold unexpected charm.


tissue of soft dust

despite eternal sweeping

chaos replenished

a fragrant creeper

behind the red dahlia

stealing attention

bees searching pinwheels,

eyes with yellow lashes,

flowering gum tree

squeezed by distance

fence posts diminish, shadows

contrast, long and sharp

finely inked tattoos

spill piquant winding stories


cheap creamy cotton

striped with afternoon sunlight

warming my window

a dream of travel

not quenching my restlessness

just honing its point


cracked earth and dry thistles

the swamp is still thirsty

an ibis steps formally

her beak curves, deep

into the crevasses


the egret too is walking,

soft bluegrey elegance

collecting caterpillars

as they taste their green hosts

in the garden a storm

of butterflies rises,

matching her slate colours,

but no longer hungry

May 04, 2015

[life] Zoe at 5

Zoe celebrated her 5th birthday a day early with a lovely party at Sarah's house, with a bunch of her friends from Kindergarten, Prep and beyond. This birthday also means she's been living in two homes for as much of her life as she's lived in one. On that front, mercifully, she seems to be doing as well as one could possibly hope for. This is her normal, as much as it breaks my heart.

She's doing fabulously well on all fronts, really. She's grown into a lovely little girl that I always enjoy spending time with. She's finally figured out how to ride a bike, so I've bought her a bigger bike for her birthday. I believe her swimming is going really well (I haven't seen her in action for a while because she does her swim classes via after-school care, but I'm fortunate to have one of my Thermomix consultant team members be her swim teacher, so I get some feedback from time to time).

We had parent-teacher interviews at the end of last term, and from all reports there, Zoe seemed to be doing well in Prep. Her sight-words are going pretty well. She's got the hang of phonics. She can write her name. She seems to have made friends with lots of the kids in her class. We've had a few of them over for dinner. I feel very connected with the school community.

I'm really grateful that I got about 5 weeks at the start of Prep before I returned to work. I got to be really involved with school for a little bit. I helped out with her school swim classes. I helped out with a literacy group. I did Tuckshop a couple of times. It was lovely. I wish I could be a stay at home parent so I could do that sort of thing all the time, but that's just not possible (at the moment, anyway). The school clearly relies quite heavily on parent helpers.

Five (and the lead up to it) seems to be a pretty fantastic age. I'm loving being her Dad now just as much as any other time.


we walk to the river to collect buckets of leaf litter for the garden.

a gradual taming of the red earth for zucchinis and beans.

she keeps close to my side

a honey coloured dark eyed shadow

we share the quiet society of insects, lizards and garden life

until evening when he returns.

catching the sun

on the edge of the porch they stand

close, like people huddled on a train platform

hailing the morning sun with relief and green vitality

a small distance above the chill

night air still lurks in the shadows

finding the lowest ground with the lowest temperature

not quite frozen this time.

On Super Hero Movies and worthiness

This was originally going to be a comment on this post over at the Guardian

Yeah so here's the thing. Hollywood isn't about worthy (in the academic sense), has never been about worthy as a business model.

Rather it's about bums on seats. From the time before they invented talkies the whole business is about making sure that people spend their hard earned cash watching the studios movie rather than someone elses.

I also think that Wilson is conflating two different phenomena. He blames the lack of "worthiness" for the fact that Directors and "auteurs" are moving away from film and towards the internet as a distribution model. This is what's known as bollocks.  This has nothing to do with Super Hero Films and everything to do with the fact that the market for entertainment itself is breaking up, meaning that people who fill a niche can now better connect with their target market.

Hollywood in twenty years time will be a completely different beast than it is today. And it's not because Super Hero Films with eventually die out (and they will). It's because the world has changed, and honestly, I think you'd be hard pressed to find anyone who will mourne the passing.

As I mentioned earlier, indie film isn't dying because of the rash of Super Hero Films, it's changing its focus. If you're an indie film maker, why would you ignore the worlds biggest market place for entertainment? if you've got a choice between an extremely limited run in an extremely limited number of cinemas with zero marketing budget, and the internet, with services like iTunes, Netflix or Google Play, I think the choice is obvious.

Honestly, there's a level of misguided snobbery around the original post. "Why does Hollywood make films about men in tights?" seems to be tone, when in fact it's not Hollywood that is "threatening" the authors idea of what is good, but the internet.

Blog Catagories: 

May 03, 2015

Twitter posts: 2015-04-27 to 2015-05-03

May 02, 2015

SM1000 Part 13 – Shipping!

The enclosure has arrived from the new manufacturer! Edwin and team at Dragino are now assembling, testing, and shipping the first batch of 100 SM1000s. We plan to ship all Aliexpress pre-orders in week starting 3 May, Australian orders the week starting 10 May.

We have sold almost all of the first batch just in pre-orders! Rick and Edwin have already started work on the next batch of 100, making some small changes to help production.

It is remarkable just how long the “little details” take to work out when putting a product into production. I had the prototype SM1000 working in September, and the first revision of the case was ready before Christmas. Things always take longer than you expect. Oh well, we have made it in the end. We are shipping about 14 months after Rick and I started work on the project, which is not bad for any product I guess. Thanks so much Rick and Edwin!

SM1000 Support

Here is the SM1000 user Guide.

For SM1000 support please post to the Codec 2 mailing list, that way we can all share the information. We’ll publish some SM1000 user guide information over the next few weeks. Maybe a wiki, so you can all join in. I really want this to be a community project.

FreeDV News

In other FreeDV news I’ve been working hard on a new “negative SNR” FreeDV mode that will find it’s way into the SM1000 and other FreeDV platforms later this year. So far I’ve developed a prototype 650 bit/s version of Codec 2 and Octave/C versions of a new coherent PSK HF modem with frequency diversity which greatly helps HF fading channel performance. I am currently being frustrated by HF modem frequency offset estimation (yet again!) but I’ll get there eventually. Other parts of the new coherent PSK HF modem are working really well.

In the VHF space, Brady KC9TPA, has been working hard on a design and PCB layout for a prototype VHF radio that can run FreeDV and demonstrate our advanced new ideas for VHF Digital Voice. Wish I was building radios too but I’m knee deep in DSP code!

Dayton 2015

Rick will be attending the Dayton Hamfest and presenting a talk on the SM1000, and will have a bunch of SM1000s for you to play with. Mel, Bruce and team will have a booth at Dayton with FreeDV and the SM1000 on display – thanks guys for all your efforts and kind support.


Under a week to go – closes Friday 8th May

With just under a week to go until the PyCon Australia 2015 Call for Proposals closes, we thought it would be a good idea to give everyone an update and a reminder. We’re very happy with the proposals we’ve already received, but we’re eager to receive more! We hope our proposal writing working bees in Brisbane have been of help, and hope to roll them out to more cities next year. If you’ve got any questions please get in touch (numerous contact details are up on We would like to give a special shout out for the Education MiniConf, which is new this year: if you know people teaching and using computing in the education realm, please forward this CFP on.

The deadline for proposal submission is Friday 8th May, 2015.


The conference this year will be held on Saturday 1st and Sunday 2nd August 2015 in Brisbane. PyCon Australia attracts professional developers from all walks of life, including industry, government, and science, as well as enthusiast and student developers. We’re looking for proposals for presentations and tutorials on any aspect of Python programming, at all skill levels from novice to advanced.

Presentation subjects may range from reports on open source, academic or commercial projects; or even tutorials and case studies. If a presentation is interesting and useful to the Python community, it will be considered for inclusion in the program.

We’re especially interested in short presentations that will teach conference-goers something new and useful. Can you show attendees how to use a module? Explore a Python language feature? Package an application?


Four Miniconfs will be held on Friday 31st July, as a prelude to the main conference. Miniconfs are run by community members and are separate to the main conference. If you are a first time speaker, or your talk is targeted to a particular field, the Miniconfs might be a better fit than the main part of the conference. If your proposal is not selected for the main part of the conference, it may be selected for one of our Miniconfs:

DjangoCon AU is the annual conference of Django users in the Southern Hemisphere. It covers all aspects of web software development, from design to deployment – and, of course, the use of the Django framework itself. It provides an excellent opportunity to discuss the state of the art of web software development with other developers and designers.

The Python in Education Miniconf aims to bring together community workshop organisers, professional Python instructors and professional educators across primary, secondary and tertiary levels to share their experiences and requirements, and identify areas of potential collaboration with each other and also with the broader Python community.

The Science and Data Miniconf is a forum for people using Python to tackle problems in science and data analysis. It aims to cover commercial and research interests in applications of science, engineering, mathematics, finance, and data analysis using Python, including AI and ‘big data’ topics.

The OpenStack Miniconf is dedicated to talks related to the OpenStack project and we welcome proposals of all kinds: technical, community, infrastructure or code talks/discussions; academic or commercial applications; or even tutorials and case studies. If a presentation is interesting and useful to the OpenStack community, it will be considered for inclusion. We also welcome talks that have been given previously in different events.

First Time Speakers

We welcome first-time speakers; we are a community conference and we are eager to hear about your experience. If you have friends or colleagues who have something valuable to contribute, twist their arms to tell us about it! Please also forward this Call for Proposals to anyone that you feel may be interested.

The most recent call for proposals information can always be found at:

See you in Brisbane in July!

Important Dates

Call for Proposals opens: Friday 27th March, 2015

Proposal submission deadline: Friday 8th May, 2015

Proposal acceptance: Monday 25 May, 2015

Filed under: Uncategorized

Fitbit and Android Lollipop

If you use FitBit tracker with a smartphone which has been recently undated to Android 5.0 Lollipop, you might faced the same problem as me - the Fitbit app has stopped synching with the tracker.

Surprisingly there are a lot of people complaining online or even raging in putting one star rating to the app in Google Play. However, the solution which helped me: uninstall and reinstall the app, is mentioned on the FitBit Help web-site, and it takes around a minute to remove and reinstall the app.

April 30, 2015

Some bitcoin mempool data: first look

Previously I discussed the use of IBLTs (on the pettycoin blog).  Kalle and I got some interesting, but slightly different results; before I revisited them I wanted some real data to play with.

Finally, a few weeks ago I ran 4 nodes for a week, logging incoming transactions and the contents of the mempools when we saw a block.  This gives us some data to chew on when tuning any fast block sync mechanism; here’s my first impressions looking a the data (which is available on github).

These graphs are my first look; in blue is the number of txs in the block, and in purple stacked on top is the number of txs which were left in the mempool after we took those away.

The good news is that all four sites are very similar; there’s small variance across these nodes (three are in Digital Ocean data centres and one is behind two NATs and a wireless network at my local coworking space).

The bad news is that there are spikes of very large mempools around block 352,800; a series of 731kb blocks which I’m guessing is some kind of soft limit for some mining software [EDIT: 750k is the default soft block limit; reported in 1024-byte quantities as does, this is 732k.  Thanks sipa!].  Our ability to handle this case will depend very much on heuristics for guessing which transactions are likely candidates to be in the block at all (I’m hoping it’s as simple as first-seen transactions are most likely, but I haven’t tested yet).

Transactions in Mempool and in Blocks: Australia (poor connection)

Transactions in Mempool and in Blocks: Singapore

Transactions in Mempool and in Blocks: San Francisco

Transactions in Mempool and in Blocks: San Francisco (using Relay Network)

Coding club day one: a simple number guessing game in python

I've recently become involved in a new computer programming club at my kids' school. The club runs on Friday afternoons after school and is still very new so we're still working through exactly what it will look like long term. These are my thoughts on the content from this first session. The point of this first lesson was to approach a programming problem where every child stood a reasonable chance of finishing in the allotted 90 minutes. Many of the children had never programmed before, so the program had to be kept deliberately small. Additionally, this was a chance to demonstrate how literal computers are about the instructions they're given -- there is no room for intuition on the part of the machine here, it does exactly what you ask of it.

The task: write a python program which picks a random number between zero and ten. Ask the user to guess the number the program has picked, with the program telling the user if they are high, low, or right.

We then brainstormed the things we'd need to know how to do to make this program work. We came up with:
  • How do we get a random number?
  • What is a variable?
  • What are data types?
  • What is an integer? Why does that matter?
  • How do we get user input?
  • How do we do comparisons? What is a conditional?
  • What are the possible states for the game?
  • What is an exception? Why did I get one? How do I read it?

With that done, we were ready to start programming. This was done with a series of steps that we walked through as a group -- let's all print hello work. Now let's generate a random number and print it. Ok, cool, now let's do input from a user. Now how do we compare that with the random number? Finally, how do we do a loop which keeps prompting until the user guesses the random number?

For each of these a code snippet was written on the whiteboard and explained. It was up to the students to put them together into a program which actually works.

Due to limitations in the school's operating environment (no local python installation and not working due to firewalling) we used for this exercise. The code that the kids ended up with looks like this:

    import random
    # Pick a random number
    number = random.randint(0, 10)
    # Now ask for guesses until the correct guess is made
    done = False
    while not done:
        guess = int(raw_input('What is your guess?'))
        print 'You guessed: %d' % guess
        if guess < number:
            print 'Higher!'
        elif guess > number:
            print 'Lower!'
            print 'Right!'
            done = True

The plan for next session (tomorrow, in the first week of term two) is to recap what we did at the end of last term and explore this program to make sure everyone understands how it works.

Tags for this post: coding_club teaching coding

Related posts: I'm glad I've turned on comments here; Huffman coding


April 29, 2015

Constructive Conflict Resolution

I'm speaking at DrupalCon Los Angeles. 5pm, Tuesday 12 May in the 518 - Trellon room.

I first spoke about Constructive Conflict Resolution in Amsterdam at DrupalCon last year. I posted the slides, recording and speakers notes from that talk to the PreviousNext blog.

I'm reprising that talk in Los Angeles because someone else is now unable to make it, and I was asked if I could fill in. When I originally proposed the talk for LA I had planned to rework the slide and narrative - but unfortunately won't have much time to do that before the conference. However this is a conversation starter, and we'll have an opportunity in the room to discuss how we might embrace conflict as a force for good, as a force for progress. How to harness it, how to minimise it's potential for harm.

I hope to see you there!

Constructive Conflict Resolution will be in the core conversations track at DrupalCon Los Angeles.

Craige McWhirter: Rebuilding An OpenStack Instance and Keeping the Same Fixed IP

OpenStack and in particular the compute service, Nova, has a useful rebuild function that allows you to rebuild an instance from a fresh image while maintaining the same fixed and floating IP addresses, amongst other metadata.

However if you have a shared storage back end, such as Ceph, you're out of luck as this function is not for you.

Fortunately, there is another way.

Prepare for the Rebuild:

Note the fixed IP address of the instance that you wish to rebuild and the network ID:

$ nova show demoinstance0 | grep network
| DemoTutorial network                       |,                     |
$ export FIXED_IP=
$ neutron floatingip-list | grep
| ee7ecd21-bd93-4f89-a220-b00b04ef6753 |                  |      |
$ export FLOATIP_ID=ee7ecd21-bd93-4f89-a220-b00b04ef6753
$ neutron net-show DemoTutorial | grep " id "
| id              | 9068dff2-9f7e-4a72-9607-0e1421a78d0d |
$ export OS_NET=9068dff2-9f7e-4a72-9607-0e1421a78d0d

You now need to delete the instance that you wish to rebuild:

$ nova delete demoinstance0
Request to delete server demoinstance0 has been accepted.

Manually Prepare the Networking:

Now you need to re-create the port and re-assign the floating IP, if it had one:

$ neutron port-create --name demoinstance0 --fixed-ip ip_address=$FIXED_IP $OS_NET
Created a new port:
| Field                 | Value                                                                                 |
| admin_state_up        | True                                                                                  |
| allowed_address_pairs |                                                                                       |
| binding:vnic_type     | normal                                                                                |
| device_id             |                                                                                       |
| device_owner          |                                                                                       |
| fixed_ips             | {"subnet_id": "eb5db27f-edad-480e-92cb-1f8fec8848a8", "ip_address": ""}  |
| id                    | c1927578-451b-4682-8888-55c7163898a4                                                  |
| mac_address           | fa:16:3e:5a:39:67                                                                     |
| name                  | demoinstance0                                                                         |
| network_id            | 9068dff2-9f7e-4a72-9607-0e1421a78d0d                                                  |
| security_groups       | 5898c15a-4670-429b-a414-9f59671c4d8b                                                  |
| status                | DOWN                                                                                  |
| tenant_id             | gsu7j52c50804cf3aad71b92e6ced65e                                                      |
$ export OS_PORT=c1927578-451b-4682-8888-55c7163898a4
$ neutron floatingip-associate $FLOATIP_ID $OS_PORT
Associated floating IP ee7ecd21-bd93-4f89-a220-b00b04ef6753
$ neutron floatingip-list | grep $FIXED_IP
| ee7ecd21-bd93-4f89-a220-b00b04ef6753 |   |     | c1927578-451b-4682-8888-55c7163898a4 |


Now you need to boot the instance again and specify port you created:

$ nova boot --flavor=m1.tiny --image=MyImage --nic port-id=$OS_PORT demoinstance0
$ nova show demoinstance0 | grep network
| DemoTutorial network                       |,                     |

Now your rebuild has been completed, you've got your old IPs back and you're done. Enjoy :-)

SPARC Processor Documentation Online

For folks who don’t follow my twitter or plus accounts, there’s a bunch of SPARC processor documentation here:

This is up to T4 & M5 and also now includes legacy systems back to Ultra-SPARC I.  Thanks to all who worked on getting these published.

OpenStack Hint of the Day: Wed Apr 29

When running tox and you get something like this:

mrda@garner:~/src/python-ironicclient (review/michael_davies/file-caching)$ tox -e py34

py34 runtests: PYTHONHASHSEED='3098345924'

py34 runtests: commands[0] | python testr --slowest --testr-args=

running testr

running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} ${PYTHON:-python} -m discover -t ./ ${OS_TEST_PATH:-./ironicclient/tests/unit}  --list 

db type could not be determined

error: testr failed (3)

ERROR: InvocationError: '/home/mrda/src/python-ironicclient/.tox/py34/bin/python testr --slowest --testr-args='

________________________________________________________________________________________________ summary _________________________________________________________________________________________________

ERROR:   py34: commands failed

The solution is to "rm -rf .testrepository/" and try again.

(Thanks to this little reference hidden away

April 28, 2015

Going beyond 1.3 MILLION SQL Queries/second

So, on a large IBM POWER8 system I was recently running the newly coined “yesmark” benchmark, which is best translated as this:

Benchmark (N for concurrency): for i in {1..N}; do yes "DO 0;" | mysql > /dev/null & done
Live results: mysqladmin -ri 1 extended-status | grep Questions

Which sounds all fun until you realize that it’s *amazingly* close in results to a sysbench point select benchmark these days (well, with MySQL 5.7.7).

Since yesmark doesn’t use InnoDB though, MariaDB is back in the game.

I don’t think it matters between MariaDB and MySQL at this point for yesbench. With MySQL in a KVM guest on a shared 2 socket POWER8 I could get 754kQPS and on a larger system, I could get 1.3 million / sec.

1.3 Million queries / sec is probably the highest number anybody has ever seen out of MySQL or MariaDB, so that’s fairly impressive in itself.

What’s also impressive is that on this workload, mysqld was still only using 50% of CPU in the system. The mysql command line client was really heavy user.

Other users are: 8% completely idle, another 12% in linux scheduler (alarmingly high really). So out of all execution time, only about 44% spent in mysqld, 29% in mysql client.

It seems that the current issues scaling to two socked POWER8 machines are the same as with scaling to other large systems, when we go beyond about 20 POWER8 cores (SMT8), we start to find new and interesting challenges.

April 27, 2015

New gst-rpicamsrc features

I’ve pushed some new changes to my Raspberry Pi camera GStreamer wrapper, at

These bring the GStreamer element up to date with new features added to raspivid since I first started the project, such as adding text annotations to the video, support for the 2nd camera on the compute module, intra-refresh and others.

Where possible, you can now dynamically update any of the properties – where the firmware supports it. So you can implement digital zoom by adjusting the region-of-interest (roi) properties on the fly, or update the annotation or change video effects and colour balance, for example.

The timestamps produced are now based on the internal STC of the Raspberry Pi, so the audio video sync is tighter. Although it was never terrible, it’s now more correct and slightly less jittery.

The one major feature I haven’t enabled as yet is stereoscopic handling. Stereoscopic capture requires 2 cameras attached to a Raspberry Pi Compute Module, so at the moment I have no way to test it works.

I’m also working on GStreamer stereoscopic handling in general (which is coming along). I look forward to releasing some of that code soon.


Using an i2c RTC with the Carambola2 (or any OpenWRT modified router)

Using i2c with a modded router is simple enough, if you have two spare GPIO then the module package kmod-i2c-gpio-custom allows selected GPIO pins to be bound to SCL and SDA respectively when the module is loaded.

However for inexplicable reasons the ability to bind an i2c RTC module to the Linux hardware clock is disabled by default by the OpenWRT configuration mechanism for ar71xx and other consumer router architectures, and there is no way to turn it on without patching!

Regardless, here is how to use an i2c RTC with the Carambola2 or any other ar71xx architecture router (e.g. WRTnode, etc.)

  • Patch the file target/linux/ar71xx/generic/ as follows:

    -FEATURES += squashfs
    +FEATURES += squashfs +rtc
  • Patch the kernel configuration target/linux/ar71xx/config-3.xx where (xx depends on your version of OpenWRT) as follows:

  • Note: the kernel configuration can be modified via the kernel build system using the command make kernel_menuconfig
  • Note: add other kernel i2c RTC modules as required
  • Add the module to your image:

  • If you have previously built OpenWRT then remove the tmp/ directory, or the change ‘+rtc’ will be ignored and the DS1307 module will not be included in your image
  • Run make defconfig
  • Build your image: make -j2

If everything worked, then the the file /lib/modules/3.xx…/rtc-ds1307.ko should be in the resulting image

Following is an aggregation of information I was already able to find elsewhere on the net.

  • Ensure that i2c-tools package is installed as well. This may require the ‘oldpackages’ feed.
  • Configure the module as follows by creating a file /etc/modules.d/99gpio-i2c-rtc
  • You can also put this file into files/etc/modules.d/99gpio-i2c-rtc for it to be automatically added to your image
  • Create the following content, where in this example 18 == SDA pin id and 19 == SCL pin id

    i2c-gpio-custom bus0=0,18,19
  • There are additional arguments controlling delays, etc.; refer to package/kernel/i2c-gpio-custom/src/i2c-gpio-custom.c
  • Create a script, /etc/init.d/rtc-driver to load the device driver and set the time.

    #!/bin/sh /etc/rc.common
    logger "Setup i2c RTC"
    echo ds1307 0x68 > '/sys/class/i2c-dev/i2c-0/device/new_device'
    if hwclock | grep 'Jan' | grep -q 2000 ; then
      logger "RTC appears to have a flat battery..."
      logger "RTC set hwclock"
      hwclock -s
  • Create a symlink…

    ln -s /etc/init.d/rtc-driver /etc/rc.d/S11rtc-driver
  • Note, if you are running ntp that will take over anyway, but for system with an intermittent or no network connection, or if the network is down on boot, the RTC will provide a better time than 1 Jan 2012 or whatever…

You can test the above out before scripting it by booting the system and manually stepping through:

modprobe i2c-gpio-custom bus0=0,18,19
i2cdetect -l
modprobe rtc-ds1307
echo ds1307 0x68 > '/sys/class/i2c-dev/i2c-0/device/new_device'


PS Dont forget pullup presistors, and take care interfacing between 5V and 3.3V systems and peripherals…

Unbrick the NUC

It seems there are many folks with the suspend of death on the NUC. When you suspend to RAM you can't get back. When you disconnect power for a while you can't turn it on again. Welcome to brickland, population: you. I found that following the advice on the forums if I disconnect the CMOS battery for a bit then I could turn on the NUC again.

The downside is that the CMOS battery is installed under the motherboard, so you have to remove the motherboard which is no easy task the first time. Then each subsequent time that the NUC bricks you have to take it apart again to such a great extent.

Luckily I found these extension leads which let me bring out the battery from the case. So hopefully now a debrick isn't going to involve a system teardown anymore.

LUV Main May 2015 Meeting: Performance Co-Pilot / Android Privacy 101

May 5 2015 19:00
May 5 2015 21:00
May 5 2015 19:00
May 5 2015 21:00

The Buzzard Lecture Theatre. Evan Burge Building, Trinity College, Melbourne University Main Campus, Parkville.


• Nathan Scott: Performance Co-Pilot

• Paul Fenwick: Android Privacy 101

The Buzzard Lecture Theatre, Evan Burge Building, Trinity College Main Campus Parkville Melways Map: 2B C5

Notes: Trinity College's Main Campus is located off Royal Parade. The Evan Burge Building is located near the Tennis Courts. See our Map of Trinity College. Additional maps of Trinity and the surrounding area (including its relation to the city) can be found at

Parking can be found along or near Royal Parade, Grattan Street, Swanston Street and College Crescent. Parking within Trinity College is unfortunately only available to staff.

For those coming via Public Transport, the number 19 tram (North Coburg - City) passes by the main entrance of Trinity College (Get off at Morrah St, Stop 12). This tram departs from the Elizabeth Street tram terminus (Flinders Street end) and goes past Melbourne Central Timetables can be found on-line at:

Before and/or after each meeting those who are interested are welcome to join other members for dinner. We are open to suggestions for a good place to eat near our venue. Maria's on Peel Street in North Melbourne is currently the most popular place to eat after meetings.

LUV would like to acknowledge Red Hat for their help in obtaining the Buzzard Lecture Theatre venue and VPAC for hosting.

Linux Users of Victoria Inc. is an incorporated association, registration number A0040056C.

May 5, 2015 - 19:00

read more

April 26, 2015

Twitter posts: 2015-04-20 to 2015-04-26

Anti-Systemd People

For the Technical People

This post isn’t really about technology, I’ll cover the technology briefly skip to the next section if you aren’t interested in Linux programming or system administration.

I’ve been using the Systemd init system for a long time, I first tested it in 2010 [1]. I use Systemd on most of my systems that run Debian/Wheezy (which means most of the Linux systems I run which aren’t embedded systems). Currently the only systems where I’m not running Systemd are some systems on which I don’t have console access, while Systemd works reasonably well it wasn’t a standard init system for Debian/Wheezy so I don’t run it everywhere. That said I haven’t had any problems with Systemd in Wheezy, so I might have been too paranoid.

I recently wrote a blog post about systemd, just some basic information on how to use it and why it’s not a big deal [2]. I’ve been playing with Systemd for almost 5 years and using it in production for almost 2 years and it’s performed well. The most serious bug I’ve found in systemd is Bug #774153 which causes a Wheezy->Jessie upgrade to hang until you run “systemctl daemon-reexec” [3].

I know that some people have had problems with systemd, but any piece of significant software will cause problems for some people, there are bugs in all software that is complex enough to be useful. However the fact that it has worked so well for me on so many systems suggests that it’s not going to cause huge problems, it should be covered in the routine testing that is needed for a significant deployment of any new version of a distribution.

I’ve been using Debian for a long time. The transitions from libc4 to libc5 and then libc6 were complex but didn’t break much. The use of devfs in Debian caused some issues and then the removal of devfs caused other issues. The introduction of udev probably caused problems for some people too. Doing major updates to Debian systems isn’t something that is new or which will necessarily cause significant problems, I don’t think that the change to systemd by default compares to changing from a.out binaries to ELF binaries (which required replacing all shared objects and executables).

The Social Issue of the Default Init

Recently the Debian technical committee determined that Systemd was the best choice for the default init system in Debian/Jessie (the next release of Debian which will come out soon). Decisions about which programs should be in the default install are made periodically and it’s usually not a big deal. Even when the choice is between options that directly involve the user (such as the KDE and GNOME desktop environments) it’s not really a big deal because you can just install a non-default option.

One of the strengths of Debian has always been the fact that any Debian Developer (DD) can just add any new package to the archive if they maintain it to a suitable technical standard and if copyright and all other relevant laws are respected. Any DD who doesn’t like any of the current init systems can just package a new one and upload it. Obviously the default option will get more testing, so the non-default options will need more testing by the maintainer. This is particularly difficult for programs that have significant interaction with other parts of the system, I’ve had difficulties with this over the course of 14 years of SE Linux development but I’ve also found that it’s not an impossible problem to solve.

It’s generally accepted that making demands of other people’s volunteer work is a bad thing, which to some extent is a reasonable position. There is a problem when this is taken to extremes, Debian has over 1000 developers who have to work together so sometimes it’s a question of who gets to do the extra work to make the parts of the distribution fit together. The issue of who gets to do the work is often based on what parts are the defaults or most commonly used options. For my work on SE Linux I often have to do a lot of extra work because it’s not part of the default install and I have to make my requests for changes to other packages be as small and simple as possible.

So part of the decision to make Systemd be the default init is essentially a decision to impose slightly more development effort on the people who maintain SysVInit if they are to provide the same level of support – of course given the lack of overall development on SysVInit the level of support provided may decrease. It also means slightly less development effort for the people who maintain Systemd as developers of daemon packages MUST make them work with it. Another part of this issue is the fact that DDs who maintain daemon packages need to maintain init.d scripts (for SysVInit) and systemd scripts, presumably most DDs will have a preference for one init system and do less testing for the other one. Therefore the choice of systemd as the default means that slightly less developer effort will go into init.d scripts. On average this will slightly increase the amount of sysadmin effort that will be required to run systems with SysVInit as the scripts will on average be less well tested. This isn’t going to be a problem in the short term as the current scripts are working reasonably well, but over the course of years bugs may creep in and a proposed solution to this is to have SysVInit scripts generated from systemd config files.

We did have a long debate within Debian about the issue of default init systems and many Debian Developers disagree about this. But there is a big difference between volunteers debating about their work and external people who don’t contribute but believe that they are entitled to tell us what to do. Especially when the non-contributors abuse the people who do the work.

The Crowd Reaction

In a world filled with reasonable people who aren’t assholes there wouldn’t be any more reaction to this than there has been to decisions such as which desktop environment should be the default (which has caused some debate but nothing serious). The issue of which desktop environment (or which version of a desktop environment) to support has a significant affect on users that can’t be avoided, I could understand people being a little upset about that. But the init system isn’t something that most users will notice – apart from the boot time.

For some reason the men in the Linux community who hate women the most seem to have taken a dislike to systemd. I understand that being “conservative” might mean not wanting changes to software as well as not wanting changes to inequality in society but even so this surprised me. My last blog post about systemd has probably set a personal record for the amount of misogynistic and homophobic abuse I received in the comments. More gender and sexuality related abuse than I usually receive when posting about the issues of gender and sexuality in the context of the FOSS community! For the record this doesn’t bother me, when I get such abuse I’m just going to write more about the topic in question.

While the issue of which init system to use by default in Debian was being discussed we had a lot of hostility from unimportant people who for some reason thought that they might get their way by being abusive and threatening people. As expected that didn’t give the result they desired, but it did result in a small trend towards people who are less concerned about the reactions of users taking on development work related to init systems.

The next thing that they did was to announce a “fork” of Debian. Forking software means maintaining a separate version due to a serious disagreement about how it should be maintained. Doing that requires a significant amount of work in compiling all the source code and testing the results. The sensible option would be to just maintain a separate repository of modified packages as has been done many times before. One of the most well known repositories was the Debian Multimedia repository, it was controversial due to flouting legal issues (the developer produced code that was legal where they lived) and due to confusion among users. But it demonstrated that you can make a repository containing many modified packages. In my work on SE Linux I’ve always had a repository of packages containing changes that haven’t been accepted into Debian, which included changes to SysVInit in about 2001.

The latest news on the fork-Debian front seems to be the call for donations [4]. Apparently most of the money that was spent went to accounting fees and buying a laptop for a developer. The amount of money involved is fairly small, Forbes has an article about how awful people can use “controversy” to get crowd-funding windfalls [5].

MikeeUSA is an evil person who hates systemd [6]. This isn’t any sort of evidence that systemd is great (I’m sure that evil people make reasonable choices about software on occasion). But it is a significant factor in support for non-systemd variants of Debian (and other Linux distributions). Decent people don’t want to be associated with people like MikeeUSA, the fact that the anti-systemd people seem happy to associate with him isn’t going to help their cause.


Forking Debian is not the correct technical solution to any problem you might have with a few packages. Filing bug reports and possibly forking those packages in an external repository is the right thing to do.

Sending homophobic and sexist abuse is going to make you as popular as the GamerGate and people. It’s not going to convince anyone to change their mind about technical decisions.

Abusing volunteers who might consider donating some of their time to projects that you like is generally a bad idea. If you abuse them enough you might get them to volunteer less of their time, but the most likely result is that they just don’t volunteer on anything associated with you.

Abusing people who write technical blog posts isn’t going to convince them that they made an error. Abuse is evidence of the absence of technical errors.

April 25, 2015

The True Meaning of Myki

Those around Victoria will be familiar with our public transport payment system called “Myki” which has had, shall we say, some teething troubles. It appears this was well known to the Vikings over 1,000 years ago as this list of Old Norse words that made it into English has:

muck – myki (cow dung)

So there you go, Myki is actually Old Norse for bullshit. :-)

This item originally posted here:

The True Meaning of Myki

Tuggeranong Trig (again)

The cubs at my local scout group are interested in walking to a trig, but have some interesting constraints around mobility for a couple of their members. I therefore offered to re-walk Tuggeranong Trig in Oxley with an eye out for terrain. I think this walk would be very doable for cubs -- its 650 meters with only about 25 meters of vertical change. The path is also ok for a wheelchair I think.


Interactive map for this route.

Tags for this post: blog pictures 20150415-tuggeranong_trig photo canberra bushwalk trig_point

Related posts: Goodwin trig; Big Monks; Narrabundah trig and 16 geocaches; Cooleman and Arawang Trigs; One Tree and Painter; A walk around Mount Stranger


April 24, 2015

Peace and Freedom

Over 1000 women gathered in the Hague in 1915

It's ANZAC day.

It's the 100 year anniversary of a particularly bad battle in Turkey, that has somehow come to represent the apex of Australian and NewZealand glorification of war. Sure, we say it's not glorifying war - but seriously how is this wall to wall coverage not glorification? The coverage in all media over the past week has numbed my senses. Not made me reflect on sacrifice.

All our focus on this one stupid battle? I'd like to put some focus on those efforts to stop the slaughter.

Gallipolli was ultimately a battle lost for the ANZACs.

So too was the attempt by over 1000 women who came together in 1915 to try to stop war. To call for resolutions for peace. To identify and disarm the causes of conflict. If only we could reflect more on that effort.

The Women's International League for Peace and Freedom -

Image: Screengrab from

Text in the image says:

As the British army, including Anzacs, is invading Turkey more than 1000 women from both warring and neutral nations meet in The Hague for the International Congress of Women. They set out resolutions for ending all war and resolve to take them immediately to all heads of state in Europe and the USA. They name themselves the International Committee of Women for Permanent Peace.

"I know that the idea that lasting peace can be gained through war is nonsense" - Eleanor Moore

April 23, 2015

Why is the Arduino IDE so stupid?

If I perform the following actions:

  • File, New

    Opens a new editor window. Reasonable enough, although I would have preferred a default single-window GUI model like QtCreator or even gedit.
  • File, Save

    Opens a save as dialog. In spite of the Arduino ‘sketchbook’ directory, it opens in my home directory.
  • New Folder

    Creates a directory New Folder, but doesn’t shift the focus to it, leaving you confused when this is done in a directory with a lot of files…
  • Click on ‘New Folder’ and rename it, say, Test123
  • Navigate into Test123/
  • Type in a filename for the project, say, TestTest1
  • Hit save.

    So now Arduino IDE dutifully ignores what I typed and proceeds to create a tab called ‘Test123′.

    It will even do this if ‘Test123/’ already existed.

  • File, Save As.

    It forgets where you where in the hierarchy and starts in the home directory again(!)
  • Navigate to Test123/ intending to use it as a container for multiple projects
  • Type in a filename, say Hello, then hit Save
  • The sketch is _still_ called Test123.


So insanely enough, it seems you essentially create a director and thats where the sketch gets its name.  Within that directory it creates a file with the same name with the extension ‘.ino’

Lets try something else:

  • From the shell, create a directory, Test456 and create a readme.txt file, and a directory Test456a and a file Test456a/readme2.txt
  • File New
  • File save
  • Navigate into Test456
  • Type in helloworld for the name
  • Again, the project gets called Test456
  • But take a look in the directory Test456: the contents are now gone (all, including the sub directory Test456a) and replaced with Test123.ino


Luckily I discovered this in a directory in a git working copy with no modifications so I didn’t lose anything important.

Testing done using Arduino1.5.8 amd64 for Digispark. So its a little out of date but not exactly the oldest either.

I have used Arduino before and to be honest I don’t recall it being this stupid, but maybe I just got lucky.

One difference is this time I got sick of the massive latency opening the windows and tried a few different Java JRE (openjdk6, openjdk7, gcc4.7-jre) before discovering that with gcc4.7-jre the menus are as snappy as the openbox right click menu, or even a (*sharp intake of breath*) brand new Windows 7 corporate desktop… maybe there is some API implementation difference between the JRE’s that affects the file save dialog functionality.

I don’t seem to have any issues opening projects.

So my workflow for creating a new project now consists of:

  • From the shell, create a directory in the relevant part of the git working copy I am using
  • Create a new empty .ino text file with the same name as the directory (or copy a template I made)
  • Open it with the IDE and start working


April 19, 2015

Twitter posts: 2015-04-13 to 2015-04-19

April 18, 2015

A letter sent, a disappointment received.

While I was distracted from the whole blogging thing, something did actually get me hacking at the keyboard on something that wasn't code. That was the metadata laws and the actions of the Labor party in allowing them to pass through with a few amendments that in the long run are going to be meaningless.

So I hacked out an email to my local federal member Stephen Jones (which I've included below).

I didn't actually receive a response from Stephen Jones until after the legislation passed through the Senate, and I have to say that I was seriously disappointed. I don't expect a lot from my representatives, but what I would like is something that actually addresses the points that I set out in the original email. What I got was the stock standard "we need to do this because [INSERT SOMETHING ABOUT TERRISMS HERE]".


Dear Stephen Jones,

I've always found you to be a decent person and someone who cares for his electorate. However I am deeply concerned at the fact that you and Labor seem to have allowed the governments Data Retention Legislation to pass without either looking at the amendments or in fact seriously considering whether it is needed at all.

Leaving aside the near Orwellian prospect of the entire nations communications being tracked for a rolling period of two years. There are a huge number of problems that seem to have been overlooked in the name of "national security".

- No warrants. There is no judicial oversight of the access to this data. I cannot believe that this is a thing that is supported. Why do we now think that it's not possible for police and other services to misuse their powers? Checks and balances exist for a reason and any move to water them down is dangerous.

- The actual data to be retained still has not been defined. In fact the legislation with amendments requires the ISP's to determine what "type" of communication it is, which means that the ISP will need to look at the content of the packet. This is not just "envelope" stuff, this is looking inside the envelope and working out what the letter you're mailing is about. This is bad.

- There doesn't appear to actually be a need for it. What problem does it solve that hasn't already been solved? The police and intelligence services seem to be operating quite well already, arresting those who would do us harm and relying on targetted communicates intercepts.

- The possibilities for abuse are through the roof. Not only for official abuse but you've just created a massive honeypot for every script kiddie and cracker around.

- What safeguards are there against using this information retrospectively? Say a new government comes into power and decides that something should be illegal and it should be illegal retrospectively? What's to stop them using this great store of data to start prosecuting people?

Labor and the government have both just told the Australian public that they are now suspect. That their every action needs to be tracked, just in case they may do something wrong. This is not something I am comfortable with, and frankly neither should you be.


Blog Catagories: 

April 16, 2015

Dare Devil

So we've been watching Dare Devil over the last couple of nights, we're up to episode 3 and I have to say I'm really impressed.

I've never been a big fan of Dare Devil the character, and dear god the movie was complete shite (up there with the first Hulk movie featuring Eric Bana for badness), but this series has really sucked me in.

For episode 3 what really brought it home for me was Ben Urich. A journalist for the Daily Bugle in the comics, Urich represents the every man and is often used to tell the story of the normal people caught up in the semi regular destruction rained down upon New York (which has included the Hulk taking over, everyone in Manhatten being turned into Spider creatures, the almost annual flooding by Namor and of course an alien invasion or two).

I'm really liking the shorter series formats for the Marvel shows as well (well leaving aside Agents of Shield). They carry the comic book story arc feel much better than trying to drag things out for 23 episodes. Agent Carter proved that and now so is Dare Devil.

All in I'm pretty happy with the state of Marvels Cinematic Universe, and am looking forward to seeing the next tranche.

Blog Catagories: 

April 14, 2015

Imagine a roadblock which is a wall of perfectly transparent AeroGel.

Here you are, barrelling down a highway at the speed limit, when suddenly you realise that you have come to a halt, so gently that you weren’t aware of as much as having slowed down.

Viola! You now have some idea of what is like to have been Gaslighted for over 5000 days by a person who is an emotional vampire: their goal is not to kill you, it’s to keep sucking away your self in order to present a façade of having a self themselves.

If you have been “told,” tens of thousands of times in indirect ways (never directly: you only become aware of an increasing number of knives accumulating in your back over a span of time), that you cannot succeed, that establishes just such an emotional roadblock.

Right now, teaching a Raspberry Pi to sing is not happening. I know what needs to be done. The resources to discover exactly how to do it are freely available. It simply does not happen. Welcome to the AeroGel roadblock.

The self-righteous Psychopath who spent so much time installing this roadblock in my mind can do no wrong in their own eyes. To actually imply that their integrity is less than complete inspires a rage attack (which is not the same as anger: there is no control at all). Deprogramming each of these blocks will not take place instantly.

April 13, 2015

Get off my Lawn

When I was about the right age to first think that taking compromising photos of myself might be good for a lark, technology was a little different. Mobile phones that weren’t actually bricks anymore could show maybe two lines of pixelated text on an unpleasantly glowing background, terrible quality digital cameras were barely affordable, and connecting to the internet actually had a sound – kind of like KSShhh-aaa-KWEO-pung-pung-drhdrhdrhd-KHH, but it went for longer than that. Or maybe it was: mobile phones only existed in gangster movies where they were installed as part of a car, digital cameras didn’t exist, and I only had access to a few local BBSes. I forget the specifics, but that’s not the point – the point is that when I was in my teens, technology was shit, and nobody had any of it. Now, technology is excellent, everybody has all of it, it’s really easy to use, and the ways in which we interact with our technology shape the ways we expect our technology to work.

If I write an email to someone, I’m thinking “I will type my message in this box here, hit SEND, and then they will receive the message and read it”. I am not thinking “I will type my message in here, hit SEND, then it will be transmitted in plain text across a vast network of computer systems, through a number of mail servers, possibly be recorded by several government agencies in case I’m a terrorist, be stored for a little while in a mail spool and possibly backed up by some ISP, before eventually being downloaded and read by the intended recipient”.

Same with photos: “I will take a picture and share it with my wife” is a distinctly personal experience (regardless of what it’s a photo of), and that’s what I’m thinking at the time. I am not thinking “I will take a picture with my phone which will then be uploaded across that same vast network to a cloud system somewhere and stored for Eris-only-knows how long in some other jurisdiction which can probably be hacked by script kiddies”.

Technology now is all about communicating with people, and about sharing our experiences, and that we can do this without having any idea what’s actually going on is fantastic. The price though is that with each service we use, we give up a certain amount of privacy, and what privacy we give up is not necessarily obvious.

To go back to the compromising photo example: When all I had was a little film camera, nobody I knew ever took photos they wouldn’t be happy for random strangers to see, because we all knew that we had to take the film to get processed – the mechanics of how the technology worked were at least somewhat obvious to the people using the technology. As far as I am aware, there are no nude photos in existence of my teenage self and partners, because we didn’t want those perverts in the photo shop to see them.

I want a world where user experience accurately reflects potential privacy – not “sharing to circles”, or allegedly private “private messages”, but where any share that could conceivably result in non-private communication is preceded by a dialog that states “I hope you know that this will go down on your permanent record”. Because privacy is important – as Bruce Schneier said: “Privacy is not about something to hide. Privacy is about human dignity. Privacy is about individuality. Privacy is about being able to decide when and how we show ourselves to other people.”

Managing Variables in Drupal 7

A couple of times recently the issue of managing variables in Drupal 7 has come up in conversation with other developers. This post outlines the various ways of managing variables in Drupal sites. The three things this guide ensures:

  • Sensitive data is kept secure
  • Variables are correct in each environment
  • You are able to track your variables (and when they changed)

The Variables Table

The most common place you'll find configuration variables is in Drupal's variable table (aka {variable}). The values in this table are often managed via admin forms that use system_settings_form(). Users enter the values click "Save configuration" and the data is stored in the database.

If you prefer to manage your configuration via the command line and know the variable you wish to set you can use drush vset. This does exactly the same thing as admin form, without needing to click on a mouse.

$conf Array

While the variables table is great at storing our variables, there are times when you want to enforce a setting. This might be because you want to prevent users from changing it (accidentally or otherwise) or because you need it to be different in each environment. The $conf array in settings.php always overrides any values in the variable table.

Acquia, Pantheon and all provide environment variables so you can use different values in your $conf array depending on the environment.

Exporting Variables

In Drupal 7, the common way to export your variables is by using Strongarm with Features. I'm not going to cover how to do this as there is loads of documentation already available on this topic.

If your variable changes on a per environment basis or if it calculated on the fly, then you won't want to use strongarm+features as the exported values are static. You will need to put them in settings.php.

Note to self: I should debug and reroll my patch for adding support in alter hooks strongarm.

My settings.php is Out of Control!

This is a common problem, especially on more complex sites. To avoid this I recommend creating sites/default/settings/settings.[env].php files. Your settings.php file should check for the environment in an environment variable and then include the appropriate settings.[env].php file.

What About Sensitive Data?

You can encrypt variables on a case by case basis using the encrypt module and some custom code similar to what I recently implemented in the Acquia SDK module (see on store and on read examples). Warning: This doesn't encrypt the data if you're using drush vset.

If you are storing sensitive data in your variables table I would recommend you implement hook_sql_sync_sanitize() which will delete the sensitive data from your db when drush sql-sanitize or drush sql-sync --sanitize are run.

How to Decide?

This little code snippet should help you decide.


// Don't try using this code in your Drupal site.

if (!using_version_control()) {
  // Seriously there is no point in doing this without version control.

if (is_data_sensitive($var)) {
  $var = encrypt_var($var);
  if (!we_use_drush_based_workflows()) {
    // I'm serious!

if (is_unique_per_environment($var)) {
else {
  if (!we_use_features_based_workflow()) {
    // I'm serious!
Would a book entitled “I married a Psychopath” or the like sell well?

One of the risks here for even a strong Empath is that there are no “red flags” in the differences between feelings and expression of them (body-language etc), for the very simple reason that there are no feelings, so there are no differences to sense.

It must be a lonely, empty life for someone who consists only of an empty bubble of Ego. Yet they are the only person who could change that. It begins with genuine humility (which has nothing to do with acting humble). They need to think nothing of themselves.

This may not sound so difficult until you understand that they think everything of themselves, full time. Religion (including Atheism) is not possible for them, as the only person they worship is themselves.

April 12, 2015

Twitter posts: 2015-04-06 to 2015-04-12

Pigs and Bread

In farming related news, we have pigs again, and I’ve finally written up my bread recipe on our new blog at My random commentary about food and farming related matters will henceforth be posted there, while everything else I usually rattle on about at length will remain here.

Enjoy :-)

One Tree and Painter

Paul and I set off to see two trigs today. One Tree is on the ACT border and is part of the centenary trail. Painter is a suburban trig in Belconnen. Much fun was had, I hope I didn't make Paul too late for the wedding he had to go to.


Interactive map for this route.

Interactive map for this route.

Tags for this post: blog pictures 20150412-one_tree_painter photo canberra bushwalk trig_point

Related posts: Goodwin trig; Big Monks; Narrabundah trig and 16 geocaches; Cooleman and Arawang Trigs; A walk around Mount Stranger; Forster trig


April 11, 2015

So, it's been a while

Well as you can see it's been a while since I last posted here, just over a year in fact, so it's time for a bit of a clean up.

As you can see I've started redesigning things, updated the theme so that it's a bit more mobile friendly (as in will be viewable on mobile), added in the feeds from Angry Beanie and I'll be doing more work around including information about the projects I'm working on such as Govchecker and Zooborns for Android

I'm also going to try and do more writing here. I think I've fallen into the trap of not writing because I use twitter or facebook instead. Blogging though helps me to focus my thoughts a bit more so we'll see how that goes.

Anyway, this blog as ever is a work in progress, so we'll see what comes.

Oh and one more thing, you'll see that I've replaced the drupal comments system with disqus instead. This way we can hopefully avoid the comment spam problem I was getting before.

Blog Catagories: 

April 10, 2015

Tiny Tim improves and gets Smaller

I finally switched Tiny Tim over to a lipo battery. Almost everything worked when I tested the new battery, the only thing that failed in a major way were the two 2812 LEDs which, either didn't come on or came on for a very quick moment and went dark. So Tim is now smaller again without the "huge" AA battery pack at it's tail.

The 2812 story was interesting. It wasn't going to be happy jumping to the 7.6v of the 2S lipo. So I tried various voltage divider setups which didn't work either. I ended up using a common 5v regulator and the lights work fine again. I think I was maybe using too high resistor values in the divider and the 2812s didn't like it. At any rate, they apparently want a good regulated power source, and I wasn't giving it one before I switched over to using the regulator.

On the whole, going from 5-6v of the AA pack to 7.6v has made it a snappier mover. I tried it initially with the battery on the bench and found it would lift the back off the desk under hard break.

Next up is probably attaching a claw or drop mechanism and ultrasound sensor and then take on the Sparkfun autonomous ping-pong ball into cup challenge. I'll probably control it via wireless from a second on board micro-controller. The drop, ultrasound, and autonomous navigation micro (and additional battery) can all be put into a single "module" that I can then bolt to Tim. All the navigation micro needs to do is control the differential drive like a remote control would. This way, the existing micro etc on Tim doesn't change at all in order for the challenge to be accepted.


Writing your first conference proposal can be difficult, so we’re running a working bee at UQ on Saturday 11th (in conjunction with Humbug). If you’ve never written a conference proposal before, or you’d like yours given the once over, please come along, register over at meetup.

Filed under: Uncategorized

Towards (and beyond) ONE MILLION queries per second

At Percona Live MySQL Conference 2015 next week I’ll be presenting on “Towards One MILLION queries per second” on 14th April at 4:50pm in Ballroom A.

This is the story of work I’ve been doing to get MySQL executing ONE MILLION SQL queries per second. It involves tales of MySQL, tales of the POWER8 Processor and a general amount of fun in extracting huge amounts of performance.

As I speak, I’m working on some even more impressive benchmark results! New hardware, new MySQL versions and really breaking news on MySQL scalability.

April 09, 2015

Thinking time

I've had a lot of things to think about this week, so I've gone on a few walks. I found some geocaches along the way, but even better I think my head is a bit more sorted out now.

Interactive map for this route.

Interactive map for this route.

Interactive map for this route.

Tags for this post: blog canberra bushwalk

Related posts: Goodwin trig; Big Monks; Geocaching; Confessions of a middle aged orienteering marker; A quick walk through Curtin; Narrabundah trig and 16 geocaches


April 08, 2015


The WSJ has an interesting article about an investor who is funding claims to invalidate patents. The logic is that he shorts the stock. When the patent is invalidated, the stock plummets. He sells the stock – profit.  Hat tip: Andrew Wilson

Lightning Networks Part IV: Summary

This is the fourth part of my series of posts explaining the bitcoin Lightning Networks 0.5 draft paper.  See Part I, Part II and Part III.

The key revelation of the paper is that we can have a network of arbitrarily complicated transactions, such that they aren’t on the blockchain (and thus are fast, cheap and extremely scalable), but at every point are ready to be dropped onto the blockchain for resolution if there’s a problem.  This is genuinely revolutionary.

It also vindicates Satoshi’s insistence on the generality of the Bitcoin scripting system.  And though it’s long been suggested that bitcoin would become a clearing system on which genuine microtransactions would be layered, it was unclear that we were so close to having such a system in bitcoin already.

Note that the scheme requires some solution to malleability to allow chains of transactions to be built (this is a common theme, so likely to be mitigated in a future soft fork), but Gregory Maxwell points out that it also wants selective malleability, so transactions can be replaced without invalidating the HTLCs which are spending their outputs.  Thus it proposes new signature flags, which will require active debate, analysis and another soft fork.

There is much more to discover in the paper itself: recommendations for lightning network routing, the node charging model, a risk summary, the specifics of the softfork changes, and more.

I’ll leave you with a brief list of requirements to make Lightning Networks a reality:

  1. A soft-fork is required, to protect against malleability and to allow new signature modes.
  2. A new peer-to-peer protocol needs to be designed for the lightning network, including routing.
  3. Blame and rating systems are needed for lightning network nodes.  You don’t have to trust them, but it sucks if they go down as your money is probably stuck until the timeout.
  4. More refinements (eg. relative OP_CHECKLOCKTIMEVERIFY) to simplify and tighten timeout times.
  5. Wallets need to learn to use this, with UI handling of things like timeouts and fallbacks to the bitcoin network (sorry, your transaction failed, you’ll get your money back in N days).
  6. You need to be online every 40 days to check that an old HTLC hasn’t leaked, which will require some alternate solution for occasional users (shut down channel, have some third party, etc).
  7. A server implementation needs to be written.

That’s a lot of work!  But it’s all simply engineering from here, just as bitcoin was once the paper was released.  I look forward to seeing it happen (and I’m confident it will).

Reading the Lord of the Rings aloud

The reading project that I am working on is a re-read of the Lord of the Rings. I’ve read the book/trilogy around a The_Lord_of_the_Rings_Trilogydozen times over the years but the two main differences this time are that I am reading it aloud and that I am consulting a couple of commentaries as I go. The references works I am using are The Lord of the Rings: A Reader’s Companion and the The Lord of the Rings Reread series by Kate Nepveu. The Companion is a fairly large book (860 pages) that follows the text page by page and gives explanations for words, characters and the history/development of the text. These can range from a simple definition to a couple of pages on a specific topic or character. The reread has a quick synopsis at the start of the article for each chapter and then some commentary by Kate followed by some comments from her readers (which I usually only quickly skim).

I started my read-aloud on February 15th 2015 and I am now ( April 7th ) just past the half-way point ( I completed The Fellowship of the Ring on March 27th) . My process is to read the text for 30-60 minutes ( I’m reading the three-book 1979 3rd edition paperback edition, which amusingly has various errors that the Reader’s Companion points out as I go) which gets me though 5-10 pages. I read aloud everything on the page including chapter titles, songs, non-English words and footnotes. A few times I have checked the correct pronunciation of words ( Eomer is one ) but otherwise I try not to get distracted. Once I finish for the session I open the Reader’s Companion and check the entries for the pages I have just read and at the end of each chapter ( chapters are usually around 20-30 pages) I have a look at Kate’s blog entry. I try an read most days and sometimes do extras on weekends.

One thing I really need to say is that I really am enjoying the whole thing. I love the book (like I said I’ve read it over a dozen times) and reading it aloud makes the experience even better. The main difference is that I do not skip over words/sentences/paragraphs which tends to happen when I read normally. So I don’t miss phrases like the description of Lake Hithoel:

The sun, already long fallen from the noon, was shining in a windy sky. The pent waters spread out into a long oval lake, pale Nen Hithoel, fenced by steep grey hills whose sides were clad with trees. At the far southern end rose three peaks. The midmost stood somewhat forward from the others and sundered from them, an island in the waters, about which the flowing River flung pale shimmering arms. Distant but deep there came up on the wind a roaring sound like the roll of thunder heard far away.


Nor do I skip the other little details that are easy to miss, like Grishnakh and his Mordor Orcs leaving the rest of the group for a couple of days on the plains of Rohan or the description of country leading up to the west gate of Moria. Although I do wish I’d seen the link to the map of Helm’s Deep halfway down this page before I’d read the chapter as it would have made things clearer. The Companion is also good at pointing out how things fit in the chronology, so when somebody gazes at the horizon and sees a cloud of smoke it will say what event elsewhere in the book (or other writing) that is from. You also get a great feel for Tolkien’s language and words and his vivid descriptions of scenes and landscape (often up to a page long) such the example above. Although I do find he uses “suddenly” an awful lot when he has new events/people break into the narrative.

The readers companion is a great resource, written by two serious Tolkien scholars but intended for general readers rather than academics. Kate Nepveu’s articles are also very useful in giving a more opinionated and subjective commentary. I would definitely recommend the experience to others who are fans of the Lord of the Rings. I’m not sure how well it would work with other books but certainly it enhances a work I already know well and love.

At the current rate I am expecting to finish some time in June or July. The next project I’m planning is Shakespeare’s plays. I am planning on reading each one (multiple times including possibly at least once aloud) and watching the BBC Television Shakespeare and other adaptations and commentaries. My plan is that I’ll cover the majority of them  but I’ll see how I go, However I’d like to at least get though the major ones.


April 06, 2015

Lightning Networks Part III: Channeling Contracts

This is the third part of my series of posts explaining the bitcoin Lightning Networks 0.5 draft paper.

In Part I I described how a Poon-Dryja channel uses a single in-blockchain transaction to create off-blockchain transactions which can be safely updated by either party (as long as both agree), with fallback to publishing the latest versions to the blockchain if something goes wrong.

In Part II I described how Hashed Timelocked Contracts allow you to safely make one payment conditional upon another, so payments can be routed across untrusted parties using a series of transactions with decrementing timeout values.

Now we’ll join the two together: encapsulate Hashed Timelocked Contracts inside a channel, so they don’t have to be placed in the blockchain (unless something goes wrong).

Revision: Why Poon-Dryja Channels Work

Here’s half of a channel setup between me and you where I’m paying you 1c: (there’s always a mirror setup between you and me, so it’s symmetrical)

Half a channel: we will invalidate transaction 1 (in favour of a new transaction 2) to send funds.

The system works because after we agree on a new transaction (eg. to pay you another 1c), you revoke this by handing me your private keys to unlock that 1c output.  Now if you ever released Transaction 1, I can spend both the outputs.  If we want to add a new output to Transaction 1, we need to be able to make it similarly stealable.

Adding a 1c HTLC Output To Transaction 1 In The Channel

I’m going to send you 1c now via a HTLC (which means you’ll only get it if the riddle is answered; if it times out, I get the 1c back).  So we replace transaction 1 with transaction 2, which has three outputs: $9.98 to me, 1c to you, and 1c to the HTLC: (once we agree on the new transactions, we invalidate transaction 1 as detailed in Part I)

Our Channel With an Output for an HTLC

Note that you supply another separate signature (sig3) for this output, so you can reveal that private key later without giving away any other output.

We modify our previous HTLC design so you revealing the sig3 would allow me to steal this output. We do this the same way we did for that 1c going to you: send the output via a timelocked mutually signed transaction.  But there are two transaction paths in an HTLC: the got-the-riddle path and the timeout path, so we need to insert those timelocked mutually signed transactions in both of them.  First let’s append a 1 day delay to the timeout path:

Timeout path of HTLC, with locktime so it can be stolen once you give me your sig3.

Similarly, we need to append a timelocked transaction on the “got the riddle solution” path, which now needs my signature as well (otherwise you could create a replacement transaction and bypass the timelocked transaction):

Full HTLC: If you reveal Transaction 2 after we agree it’s been revoked, and I have your sig3 private key, I can spend that output before you can, down either the settlement or timeout paths.

Remember The Other Side?

Poon-Dryja channels are symmetrical, so the full version has a matching HTLC on the other side (except with my temporary keys, so you can catch me out if I use a revoked transaction).  Here’s the full diagram, just to be complete:

A complete lightning network channel with an HTLC, containing a glorious 13 transactions.

Closing The HTLC

When an HTLC is completed, we just update transaction 2, and don’t include the HTLC output.  The funds either get added to your output (R value revealed before timeout) or my output (timeout).

Note that we can have an arbitrary number of independent HTLCs in progress at once, and open and/or close as many in each transaction update as both parties agree to.

Keys, Keys Everywhere!

Each output for a revocable transaction needs to use a separate address, so we can hand the private key to the other party.  We use two disposable keys for each HTLC[1], and every new HTLC will change one of the other outputs (either mine, if I’m paying you, or yours if you’re paying me), so that needs a new key too.  That’s 3 keys, doubled for the symmetry, to give 6 keys per HTLC.

Adam Back pointed out that we can actually implement this scheme without the private key handover, and instead sign a transaction for the other side which gives them the money immediately.  This would permit more key reuse, but means we’d have to store these transactions somewhere on the off chance we needed them.

Storing just the keys is smaller, but more importantly, Section 6.2 of the paper describes using BIP 32 key hierarchies so the disposable keys are derived: after a while, you only need to store one key for all the keys the other side has given you.  This is vastly more efficient than storing a transaction for every HTLC, and indicates the scale (thousands of HTLCs per second) that the authors are thinking.

Next: Conclusion

My next post will be a TL;DR summary, and some more references to the implementation details and possibilities provided by the paper.


[1] The new sighash types are fairly loose, and thus allow you to attach a transaction to a different parent if it uses the same output addresses.  I think we could re-use the same keys in both paths if we ensure that the order of keys required is reversed for one, but we’d still need 4 keys, so it seems a bit too tricky.

April 05, 2015

Twitter posts: 2015-03-30 to 2015-04-05

April 04, 2015

Bendora Arboretum and Bulls Head trig

Prompted largely by a not very detailed entry in a book, a bunch of friends and I went to explore Bendora Arboretum. The arboretum was planted in the 1940's as scientific experiments exploring what soft woods would grow well in our climate -- this was prompted by the large amount of wood Australia was importing at the time. There were 34 Arboreta originally, but only this one remains. The last three other than this one were destroyed in the 2003 bush fires.

This walk appears in Best Bush, Town and Village Walks in and around the ACT by Marion Stuart, which was the inspiration for this outing. The only thing to note with her description is that the walk is a fair bit longer than she describes -- its 2km from the locked gate to the hut, which means a 4km return walk before you explore the arboretum at all. The arboretum has received some attention from the ACT government recently, with new signage and a fresh gravel pass. Also please note this area might only be accessible by four wheel drive in winter, which is not mentioned in the book.

We also did a side trip to Bulls Head trig, which was interesting as its not the traditional shape.


See more thumbnails

Interactive map for this route.

Interactive map for this route.

Tags for this post: blog pictures 20150404-bendora_bulls_head photo canberra bushwalk trig_point

Related posts: Goodwin trig; Big Monks; Narrabundah trig and 16 geocaches; Cooleman and Arawang Trigs; One Tree and Painter; A walk around Mount Stranger


April 03, 2015

Using OpenVPN on Android Lollipop

I use my Linode VPS as a VPN endpoint for my laptop when I'm using untrusted networks and I wanted to do the same on my Android 5 (Lollipop) phone.

It turns out that it's quite easy to do (doesn't require rooting your phone) and that it works very well.

Install OpenVPN

Once you have installed and configured OpenVPN on the server, you need to install the OpenVPN app for Android (available both on F-Droid and Google Play).

From the easy-rsa directory you created while generating the server keys, create a new keypair for your phone:

./build-key nexus6        # "nexus6" as Name, no password

and then copy the following files onto your phone:

  • ca.crt
  • nexus6.crt
  • nexus6.key
  • ta.key

Create a new VPN config

If you configured your server as per my instructions, these are the settings you'll need to use on your phone:


  • LZO Compression: YES
  • Type: Certificates
  • CA Certificate: ca.crt
  • Client Certificate: nexus6.crt
  • Client Certificate Key: nexus6.key

Server list:

  • Server address:
  • Port: 1194
  • Protocol: UDP
  • Custom Options: NO


  • Expect TLS server certificate: YES
  • Certificate hostname check: YES
  • Remote certificate subject: server
  • Use TLS Authentication: YES
  • TLS Auth File: ta.key
  • TLS Direction: 1
  • Encryption cipher: AES-256-CBC
  • Packet authentication: SHA384 (not SHA-384)

That's it. Everything else should work with the defaults.

April 01, 2015

Lightning Networks Part II: Hashed Timelock Contracts (HTLCs)

In Part I, we demonstrated Poon-Dryja channels; a generalized channel structure which used revocable transactions to ensure that old transactions wouldn’t be reused.

A channel from me<->you would allow me to efficiently send you 1c, but that doesn’t scale since it takes at least one on-blockchain transaction to set up each channel. The solution to this is to route funds via intermediaries;  in this example we’ll use the fictitious “MtBox”.

If I already have a channel with MtBox’s Payment Node, and so do you, that lets me reliably send 1c to MtBox without (usually) needing the blockchain, and it lets MtBox send you 1c with similar efficiency.

But it doesn’t give me a way to force them to send it to you; I have to trust them.  We can do better.

Bonding Unrelated Transactions using Riddles

For simplicity, let’s ignore channels for the moment.  Here’s the “trust MtBox” solution:

I send you 1c via MtBox; simplest possible version, using two independent transactions. I trust MtBox to generate its transaction after I send it mine.

What if we could bond these transactions together somehow, so that when you spend the output from the MtBox transaction, that automatically allows MtBox to spend the output from my transaction?

Here’s one way. You send me a riddle question to which nobody else knows the answer: eg. “What’s brown and sticky?”.  I then promise MtBox the 1c if they answer that riddle correctly, and tell MtBox that you know.

MtBox doesn’t know the answer, so it turns around and promises to pay you 1c if you answer “What’s brown and sticky?”. When you answer “A stick”, MtBox can pay you 1c knowing that it can collect the 1c off me.

The bitcoin blockchain is really good at riddles; in particular “what value hashes to this one?” is easy to express in the scripting language. So you pick a random secret value R, then hash it to get H, then send me H.  My transaction’s 1c output requires MtBox’s signature, and a value which hashes to H (ie. R).  MtBox adds the same requirement to its transaction output, so if you spend it, it can get its money back from me:

Two Independent Transactions, Connected by A Hash Riddle.

Handling Failure Using Timeouts

This example is too simplistic; when MtBox’s PHP script stops processing transactions, I won’t be able to get my 1c back if I’ve already published my transaction.  So we use a familiar trick from Part I, a timeout transaction which after (say) 2 days, returns the funds to me.  This output needs both my and MtBox’s signatures, and MtBox supplies me with the refund transaction containing the timeout:

Hash Riddle Transaction, With Timeout

MtBox similarly needs a timeout in case you disappear.  And it needs to make sure it gets the answer to the riddle from you within that 2 days, otherwise I might use my timeout transaction and it can’t get its money back.  To give plenty of margin, it uses a 1 day timeout:

MtBox Needs Your Riddle Answer Before It Can Answer Mine

Chaining Together

It’s fairly clear to see that longer paths are possible, using the same “timelocked” transactions.  The paper uses 1 day per hop, so if you were 5 hops away (say, me <-> MtBox <-> Carol <-> David <-> Evie <-> you) I would use a 5 day timeout to MtBox, MtBox a 4 day to Carol, etc.  A routing protocol is required, but if some routing doesn’t work two nodes can always cancel by mutual agreement (by creating timeout transaction with no locktime).

The paper refers to each set of transactions as contracts, with the following terms:

  • If you can produce to MtBox an unknown 20-byte random input data R from a known H, within two days, then MtBox will settle the contract by paying you 1c.
  • If two days have elapsed, then the above clause is null and void and the clearing process is invalidated.
  • Either party may (and should) pay out according to the terms of this contract in any method of the participants choosing and close out this contract early so long as both participants in this contract agree.

The hashing and timelock properties of the transactions are what allow them to be chained across a network, hence the term Hashed Timelock Contracts.

Next: Using Channels With Hashed Timelock Contracts.

The hashed riddle construct is cute, but as detailed above every transaction would need to be published on the blockchain, which makes it pretty pointless.  So the next step is to embed them into a Poon-Dryja channel, so that (in the normal, cooperative case) they don’t need to reach the blockchain at all.

March 31, 2015

PyCon Australia 2015 Call for Proposals is Open!

Closes Friday 8th May

PyCon Australia 2015 is pleased to announce that its Call for Proposals is now open!

The conference this year will be held on Saturday 1st and Sunday 2nd August 2015 in Brisbane. We'll also be featuring a day of Miniconfs on Friday 31st July.

The deadline for proposal submission is Friday 8th May, 2015.

PyCon Australia attracts professional developers from all walks of life, including industry, government, and science, as well as enthusiast and student developers. We’re looking for proposals for presentations and tutorials on any aspect of Python programming, at all skill levels from novice to advanced.

Presentation subjects may range from reports on open source, academic or commercial projects; or even tutorials and case studies. If a presentation is interesting and useful to the Python community, it will be considered for inclusion in the program.

We're especially interested in short presentations that will teach conference-goers something new and useful. Can you show attendees how to use a module? Explore a Python language feature? Package an application?


Four Miniconfs will be held on Friday 31st July, as a prelude to the main conference. Miniconfs are run by community members and are separate to the main conference. If you are a first time speaker, or your talk is targeted to a particular field, the Miniconfs might be a better fit than the main part of the conference. If your proposal is not selected for the main part of the conference, it may be selected for one of our Miniconfs:

DjangoCon AU is the annual conference of Django users in the Southern Hemisphere. It covers all aspects of web software development, from design to deployment - and, of course, the use of the Django framework itself. It provides an excellent opportunity to discuss the state of the art of web software development with other developers and designers.

The Python in Education Miniconf aims to bring together community workshop organisers, professional Python instructors and professional educators across primary, secondary and tertiary levels to share their experiences and requirements, and identify areas of potential collaboration with each other and also with the broader Python community.

The Science and Data Miniconf is a forum for people using Python to tackle problems in science and data analysis. It aims to cover commercial and research interests in applications of science, engineering, mathematics, finance, and data analysis using Python, including AI and 'big data' topics.

The OpenStack Miniconf is dedicated to talks related to the OpenStack project and we welcome proposals of all kinds: technical, community, infrastructure or code talks/discussions; academic or commercial applications; or even tutorials and case studies. If a presentation is interesting and useful to the OpenStack community, it will be considered for inclusion. We also welcome talks that have been given previously in different events.

First Time Speakers

We welcome first-time speakers; we are a community conference and we are eager to hear about your experience. If you have friends or colleagues who have something valuable to contribute, twist their arms to tell us about it! Please also forward this Call for Proposals to anyone that you feel may be interested.

The most recent call for proposals information can always be found at:

See you in Brisbane in July!

Important Dates

  1. Call for Proposals opens: Friday 27th March, 2015
  2. Proposal submission deadline: Friday 8th May, 2015
  3. Proposal acceptance: Monday 25 May, 2015

LUV Main April 2015 Meeting: Storytelling for Digital Media / Deploying Microservices Effectively

Apr 7 2015 19:00
Apr 7 2015 21:00
Apr 7 2015 19:00
Apr 7 2015 21:00

The Buzzard Lecture Theatre. Evan Burge Building, Trinity College, Melbourne University Main Campus, Parkville.


• Katherine Phelps: Storytelling for Digital Media

• Daniel Hall: Deploying Microservices Effectively

The Buzzard Lecture Theatre, Evan Burge Building, Trinity College Main Campus Parkville Melways Map: 2B C5

Notes: Trinity College's Main Campus is located off Royal Parade. The Evan Burge Building is located near the Tennis Courts. See our Map of Trinity College. Additional maps of Trinity and the surrounding area (including its relation to the city) can be found at

Parking can be found along or near Royal Parade, Grattan Street, Swanston Street and College Crescent. Parking within Trinity College is unfortunately only available to staff.

For those coming via Public Transport, the number 19 tram (North Coburg - City) passes by the main entrance of Trinity College (Get off at Morrah St, Stop 12). This tram departs from the Elizabeth Street tram terminus (Flinders Street end) and goes past Melbourne Central Timetables can be found on-line at:

Before and/or after each meeting those who are interested are welcome to join other members for dinner. We are open to suggestions for a good place to eat near our venue. Maria's on Peel Street in North Melbourne is currently the most popular place to eat after meetings.

LUV would like to acknowledge Red Hat for their help in obtaining the Buzzard Lecture Theatre venue and VPAC for hosting.

Linux Users of Victoria Inc. is an incorporated association, registration number A0040056C.

April 7, 2015 - 19:00

read more

Lightning Networks Part I: Revocable Transactions

I finally took a second swing at understanding the Lightning Network paper.  The promise of this work is exceptional: instant reliable transactions across the bitcoin network. But the implementation is complex and the draft paper reads like a grab bag of ideas; but it truly rewards close reading!  It doesn’t involve novel crypto, nor fancy bitcoin scripting tricks.

There are several techniques which are used in the paper, so I plan to concentrate on one per post and wrap up at the end.

Revision: Payment Channels

I open a payment channel to you for up to $10

A Payment Channel is a method for sending microtransactions to a single recipient, such as me paying you 1c a minute for internet access.  I create an opening transaction which has a $10 output, which can only be redeemed by a transaction input signed by you and me (or me alone, after a timeout, just in case you vanish).  That opening transaction goes into the blockchain, and we’re sure it’s bedded down.

I pay you 1c in the payment channel. Claim it any time!

Then I send you a signed transaction which spends that opening transaction output, and has two outputs: one for $9.99 to me, and one for 1c to you.  If you want, you could sign that transaction too, and publish it immediately to get your 1c.

Update: now I pay you 2c via the payment channel.

Then a minute later, I send you a signed transaction which spends that same opening transaction output, and has a $9.98 output for me, and a 2c output for you. Each minute, I send you another transaction, increasing the amount you get every time.

This works because:

  1.  Each transaction I send spends the same output; so only one of them can ever be included in the blockchain.
  2. I can’t publish them, since they need your signature and I don’t have it.
  3. At the end, you will presumably publish the last one, which is best for you.  You could publish an earlier one, and cheat yourself of money, but that’s not my problem.

Undoing A Promise: Revoking Transactions?

In the simple channel case above, we don’t have to revoke or cancel old transactions, as the only person who can spend them is the person who would be cheated.  This makes the payment channel one way: if the amount I was paying you ever went down, you could simply broadcast one of the older, more profitable transactions.

So if we wanted to revoke an old transaction, how would we do it?

There’s no native way in bitcoin to have a transaction which expires.  You can have a transaction which is valid after 5 days (using locktime), but you can’t have one which is valid until 5 days has passed.

So the only way to invalidate a transaction is to spend one of its inputs, and get that input-stealing transaction into the blockchain before the transaction you’re trying to invalidate.  That’s no good if we’re trying to update a transaction continuously (a-la payment channels) without most of them reaching the blockchain.

The Transaction Revocation Trick

But there’s a trick, as described in the paper.  We build our transaction as before (I sign, and you hold), which spends our opening transaction output, and has two outputs.  The first is a 9.99c output for me.  The second is a bit weird–it’s 1c, but needs two signatures to spend: mine and a temporary one of yours.  Indeed, I create and sign such a transaction which spends this output, and send it to you, but that transaction has a locktime of 1 day:

The first payment in a lightning-style channel.

Now, if you sign and publish that transaction, I can spend my $9.99 straight away, and you can publish that timelocked transaction tomorrow and get your 1c.

But what if we want to update the transaction?  We create a new transaction, with 9.98c output to me and 2c output to a transaction signed by both me and another temporary address of yours.  I create and sign a transaction which spends that 2c output, has a locktime of 1 day and has an output going to you, and send it to you.

We can revoke the old transaction: you simply give me the temporary private key you used for that transaction.  Weird, I know (and that’s why you had to generate a temporary address for it).  Now, if you were ever to sign and publish that old transaction, I can spend my $9.99 straight away, and create a transaction using your key and my key to spend your 1c.  Your transaction (1a below) which could spend that 1c output is timelocked, so I’ll definitely get my 1c transaction into the blockchain first (and the paper uses a timelock of 40 days, not 1).

Updating the payment in a lightning-style channel: you sent me your private key for sig2, so I could spend both outputs of Transaction 1 if you were to publish it.

So the effect is that the old transaction is revoked: if you were to ever sign and release it, I could steal all the money.  Neat trick, right?

A Minor Variation To Avoid Timeout Fallback

In the original payment channel, the opening transaction had a fallback clause: after some time, it is all spendable by me.  If you stop responding, I have to wait for this to kick in to get my money back.  Instead, the paper uses a pair of these “revocable” transaction structures.  The second is a mirror image of the first, in effect.

A full symmetric, bi-directional payment channel.

So the first output is $9.99 which needs your signature and a temporary signature of mine.  The second is  1c for meyou.  You sign the transaction, and I hold it.  You create and sign a transaction which has that $9.99 as input, a 1 day locktime, and send it to me.

Since both your and my “revocable” transactions spend the same output, only one can reach the blockchain.  They’re basically equivalent: if you send yours you must wait 1 day for your money.  If I send mine, I have to wait 1 day for my money.  But it means either of us can finalize the payment at any time, so the opening transaction doesn’t need a timeout clause.


Now we have a generalized transaction channel, which can spend the opening transaction in any way we both agree on, without trust or requiring on-blockchain updates (unless things break down).

The next post will discuss Hashed Timelock Contracts (HTLCs) which can be used to create chains of payments…

Notes For Pedants:

In the payment channel open I assume OP_CHECKLOCKTIMEVERIFY, which isn’t yet in bitcoin.  It’s simpler.

I ignore transaction fees as an unnecessary distraction.

We need malleability fixes, so you can’t mutate a transaction and break the ones which follow.  But I also need the ability to sign Transaction 1a without a complete Transaction 1 (since you can’t expose the signed version to me).  The paper proposes new SIGHASH types to allow this.

[EDIT 2015-03-30 22:11:59+10:30: We also need to sign the other symmetric transactions before signing the opening transaction.  If we released a completed opening transaction before having the other transactions, we might be stuck with no way to get our funds back (as we don’t have a “return all to me” timeout on the opening transaction)]

March 30, 2015

Sleep: How to nap like a pro | BBC Future

March 29, 2015

Twitter posts: 2015-03-23 to 2015-03-29

Challenge for 2015: hackaday prize competition

So the 2015 Hackaday prize is happening, until at least August.

Somehow I’ve currently ended up involved with not one, but two entries!  The good thing is that with four months to go until the first round submission, I have been careful not to bite off more that can be chewed in the time available on weekends, or after the kids go to bed, etc. with other commitments. Along the way though it should be educational and fun, and with any luck I might at least win a T-shirt or something (some electronics test gear would be nice) … I’m under no illusion we will get anywhere near winning a trip to space!

The themes this year are is “Build Something that Matters”, around environment, agriculture and energy, with the related facet of solving a problem, and not necessarily a world-scale problem.

So my first project, of which I am making good progress, is a farm crop monitoring system for Australian conditions.  This utilises the ESP8266 wifi module and will exercise its deep sleep mode, and solar power, along with a yet to be determined Linux module for a local base station, and hopefully ISM band telemetry over long distances. I will also be helped by my neighbour who is a farmer who can use this system.

The second project, which is not my idea but that of a close friend, (but for which I am presently responsible for maintaining the page), is an Algorithmic Composting machine built out of repurposed parts and cheap electronics.  I’ll probably end up assisting with the embedded electronics, as well as keeping the documentation up to date.

I wont be posting here in a lot of detail as the contest progresses, as there is a project log built into the site intended for that purpose.  So follow along at and  instead! (And please like our projects if you have a hackaday account!)


March 28, 2015

Fedora 21: automatic software updates

The way Fedora does automatic software updates has changed with the replacement of yum(8) with dnf(8).

Start by disabling yum's automatic updates, if installed:

# dnf remove yum-cron yum-cron-daily

Then install the dnf automatic update software:

# dnf install dnf-automatic

Alter /etc/dnf/automatic.conf to change the "apply_updates" line:

apply_updates = yes

Instruct systemd to run the updates periodically:

# systemctl enable dnf-automatic.timer
# systemctl start dnf-automatic.timer


PyCon Australia 2015 is pleased to announce that its Call for Proposals is now open!

The conference this year will be held on Saturday 1st and Sunday 2nd August 2015 in Brisbane. We’ll also be featuring a day of Miniconfs on Friday 31st July.

The deadline for proposal submission is Friday 8th May, 2015.

PyCon Australia attracts professional developers from all walks of life, including industry, government, and science, as well as enthusiast and student developers. We’re looking for proposals for presentations and tutorials on any aspect of Python programming, at all skill levels from novice to advanced.

Presentation subjects may range from reports on open source, academic or commercial projects; or even tutorials and case studies. If a presentation is interesting and useful to the Python community, it will be considered for inclusion in the program.

We’re especially interested in short presentations that will teach conference-goers something new and useful. Can you show attendees how to use a module? Explore a Python language feature? Package an application?


Four Miniconfs will be held on Friday 31st July, as a prelude to the main conference. Miniconfs are run by community members and are separate to the main conference. If you are a first time speaker, or your talk is targeted to a particular field, the Miniconfs might be a better fit than the main part of the conference. If your proposal is not selected for the main part of the conference, it may be selected for one of our Miniconfs:

DjangoCon AU is the annual conference of Django users in the Southern Hemisphere. It covers all aspects of web software development, from design to deployment – and, of course, the use of the Django framework itself. It provides an excellent opportunity to discuss the state of the art of web software development with other developers and designers.

The Python in Education Miniconf aims to bring together community workshop organisers, professional Python instructors and professional educators across primary, secondary and tertiary levels to share their experiences and requirements, and identify areas of potential collaboration with each other and also with the broader Python community.

The Science and Data Miniconf is a forum for people using Python to tackle problems in science and data analysis. It aims to cover commercial and research interests in applications of science, engineering, mathematics, finance, and data analysis using Python, including AI and ‘big data’ topics.

The OpenStack Miniconf is dedicated to talks related to the OpenStack project and we welcome proposals of all kinds: technical, community, infrastructure or code talks/discussions; academic or commercial applications; or even tutorials and case studies. If a presentation is interesting and useful to the OpenStack community, it will be considered for inclusion. We also welcome talks that have been given previously in different events.

Full details:

Filed under: Uncategorized

Parallel Importing vs The Economist

Simpson-economistFor the last few years I have subscribed to the online edition of  The Economist magazine. Previously I read it via their website but for the last year or two I have used their mobile app. Both feature the full-text of each week’s magazine. Since I subscribed near 15 years ago I have paid:

Launched Jun 1997   US$ 48
Jun 1999            US$ 48
Oct 2002            US$ 69
Oct 2003            US$ 69
Dec 2006            US$ 79
Oct 2009            US$ 79
Oct 2010            US$ 95
Oct 2011            US$ 95
Mar 2014            NZ$ 400 (approx US$ 300) 

You will note the steady creep for a few years followed by the huge jump in 2014.

Note: I reviewed by credit card bill for 2012 and 2013 and I didn’t see any payments, it is possible I was getting it for free for two years :) . Possibly this was due to the transition between using an outside card processor (Worldpay) and doing the subscriptions in-house.

Last year I paid the bill in a bit of a rush and while I was surprised at the amount I didn’t think to hard. This year however I had a closer look. What seems to have happened is that The Economist has changed their online pricing model from “cheap online product” to “discount from the printed price”. This means that instead of online subscribers paying the same everywhere they now pay slightly less than it would cost to get the printed magazine delivered to the home.

Unfortunately the New Zealand price is very high to (I assume) cover the cost of shipping a relatively small number of magazines via air all the way from the nearest printing location.
















So readers in New Zealand are now charged NZ$ 736 for a two-year digital subscription while readers in the US are now charged US$ 223 ( NZ$ 293) for the same product. Thus New Zealanders pay 2.5 times as much as Americans.

Fortunately since I am a globe-trotting member of the world elite® I was able to change my subscription address to my US office and save a bunch of cash. However for a magazine that publishes the Big Mac Index comparing prices of products around the world the huge different in prices for the same digital product seems a little weird.


March 26, 2015

The Cloud : An Inferior Implementation of HPC

The use of cloud computing as an alternative implementation for high performance computing (HPC) initially seems to be appealing, especially to IT managers and to users who may find the jump from their desktop application to the command line interface challenging. However a careful and nuanced review of metrics should lead to a reconsideration of these assumptions.

read more


Job control is a basic feature of popular UNIX and Linux shells, such as “bash”.

It can be very useful, so I thought I’d make a little tutorial on it…

^C    press Ctrl-C to interrupt a running job (you know this one!)
^\    press Ctrl-\ (backslash) to QUIT a running job (stronger)
^Z    press Ctrl-Z to STOP a running job, it can be resumed later
jobs  type jobs for a list of stopped jobs (and background jobs)
fg    type fg to continue a job in the foreground
bg    type bg to continue a job in the background
kill  kill a job, e.g. kill %1, or kill -KILL %2
wait  wait for all background jobs to finish

You can also use fg and bg with a job number, if you have several jobs in the list.

You can start a job in the background: put an &-symbol at the end of the command. This works well for jobs that write to a file, but not for interactive jobs. Things might get messy if you have a background job that writes to the terminal.

If you forget the % with kill, it will try to kill by process-id instead of job number.  You don’t want to accidentally kill PID 1!

An example:

vi /etc/apache2/vhosts.d/ids.conf
find / >find.out &
fg 2
bg 2
kill %2