linux.conf.au 2009
Marching south together...

July 30, 2010

30 Jul 2010

Got bored of having to run 'make install' when hacking on Evolution, partly because libtool insanity makes it take too long — as for some reason it relinks everything as it installs it. Perhaps that was needed for FORTRAN77 programs on OSF/1, but it isn't needed on my modern Linux system. I hate libtool. But even without that, re-running 'make install' every time you change a line of code is a pain.

For a while I took to manually symlinking the libraries and executables I was working on, from my build directory into their installed locations. But I kept missing some out and that was a pain too.

My current solution, which excited mbarnes sufficiently that I felt I ought to share it more widely, is to re-run autogen.sh with the --enable-fast-install argument, then build it and run 'make INSTALL=install_symlink.sh install'. Then all files get installed as symlinks instead of being copied, and all I have to do is hack code, type 'make', and run evolution again.

The script is a dirty hack and there are much better ways to do it — some of which would even cope with filenames that have spaces in. But it works for me, and makes Evolution hacking a little easier.

Brother HL-2142

Just purchased a B&W Brother HL-2142 Laser,

Ubuntu compatibility is plug in wait and print work out the box. (10.04)

$99 from office works.

Only fault does not come with a USB cable !

July 29, 2010

The Deplorable State of Commercial Gaming

I'm really quite angry right now: half at Blizzard, and half at myself because I knew that I was going to get something like this and gave them my money anyway.

Starcraft II doesn't support LAN play. This is not just a case of needing to be online and authenticated with Battle.net to fire up a server. That would be annoying but could be poorly justified by the improved integration with your online profile.

No, Starcraft II goes out of its way to make playing on a LAN difficult even when everybody has purchased the game and has logged on to Battle.net.

Suppose you're at a LAN with a bunch of other dudes (or gals, let's be optimistic) and you're fortunate enough to have an Internet connection so that you can all get onto Battle.net. As best I can tell, you have two options for getting in a game together:

Option 1: Invitations

  • Go around and find out each other's user names on Battle.net and add each other as "friends".
  • Have the host start a private game and then invite all the relevant people to the game.
  • Other players accept the invitation.
  • Fingers crossed packets will actually get routed over the LAN (this has been hinted at but I haven't tested it.)

    (update: a friend informs me that everything goes through Singapore. Yay.)

Option 2: Public and Pray

  • Have the host start a public game, which allows anyone on Battle.net to join.
  • Tell the players what map it is.
  • Players try to connect and hope they get the right server.
  • Host kicks other players who try to join because they figure it's a public game.
  • Again, fingers crossed regarding routing.

This is seriously crap.

As the people who are reading this probably know, I help run LANs this year where we don't have Internet. I had a glorious plan to allow people to connect to Battle.net through my phone's 3G connection so that we could get some games happening, but nobody's going to want to go through all the above to make it work. Sure, I'd probably do it because I really like Starcraft but I would feel like a deluded fanatic saying "Hey you should play this game! It'll be great! We just have to do all this bullshit to make it work, but you know you want to!"

It's like a fricking iPhone. Sure, I could buy one and jailbreak it to do what I want. Sure, somebody's going to hack SC2 so it can be played on a LAN eventually. But I shouldn't have to do it. Antifeatures are lame. I may not have bought an iPhone but because of my lack of moral conviction Blizzard already has my money and they don't give a toss what I think from here on in.

I should learn to find Battle for Wesnoth exciting or something.

Perdition 1.19-rc3 Released

[Container Ship]

I have released version 1.19-rc3 of Perdition. This release fixes several problems. Perhaps most significantly, Thunderbird 3.1 can now re-connect to perdition SSL/TLS.

The code and related libraries are available as tarballs here. More information is available in the announcement email. More information about perdition here.



kexec-tools 2.0.2 Released

[Sumo]

I have released version 2.0.2 of kexec-tools. the user-space portion of kexec a soft-reboot and crash-dump facility of Linux and Xen.

There are changes since 2.0.2-rc1.

The code is available as a tarball here and in git here. More information is available in the announcement email.

kexec-tools recently moved, so the URLs above are new.

Moved

[Coral in Miyakojima] My home page and blog have moved. The new locations are:

July 28, 2010

28 Jul 2010

Things I hate today include:
  • Symbian on my Nokia N97 — for spontaneously rebooting as soon as I got off the ferry.
  • Google Maps — for not caching the map tiles I'd carefully downloaded while I was on the free ferry wireless, showing my route to the hotel.
  • Mobile phone networks — for the insane amount of money it will have cost me to re-download the same map tiles again, as I was driving.
It's almost as if it's a conspiracy — especially between the latter two.

I really need to get myself an N900 and start using maemo-mapper again. Every time I try to use non-free software, it hurts.

July 27, 2010

Dune2 on Linux

Download it from here (with a browser, it does some redirect crap). Install dosbox:

sudo aptitude install dosbox

It is a self-extracting zip file, so you can either use wine to unzip (holy nuclear option, batman) or install unzip:

sudo aptitude install unzip

Create a directory that we can ask dosbox to mount as ‘C:’, then unzip it:

mkdir -p ~/.dos/Dune2
cd ~/.dos/Dune2
unzip ~/Downloads/Dune2.exe

Get a default config file by starting up dosbox:

dosbox

and (in dosbox) ask it to write out a config file for you:

config -writeconf /home/you/.dosbox.conf

type ‘exit’ in the dosbox window to, er, exit. Add the line to mount the directory we created:

echo "mount c /home/you/.dos" >> ~/.dosbox.conf

and start dosbox again (with that config):

dosbox -conf ~/.dosbox.conf

and run Dune2 in the dosbox window (dosbox’s shell has tab completion, yay):

cd Dune2
Dune2.exe

Clicking in the window will make dosbox grab your mouse and keyboard - ctrl-f10 to escape.

July 24, 2010

iPhoto to GIMP

When I first started playing photography properly I was content to use iPhoto to do my basic post-processing. Now I'm in a mac-less environment and I like to use GIMP to edit things. Unfortunately I wasn't really sure what iPhoto was doing with my photo: I would just tweak the sliders until I was happy. That left me wondering what the "real" edit is to achieve the effect of the "Shadows" and "Highlights" sliders.

Answer: It's all in the levels editor. Go to "Levels..." in the Colors menu and you're greeted with a level-editing histogram.

  • Moving the middle triangle in Input Levels to the left is the same as increasing the "Shadows" slider in iPhoto.
  • Moving the right triangle in Output Levels to the left is the same as increasing the "Highlights" slider in iPhoto.

And with a bit of tinkering you can work out how to do other things too. Hooray.

July 20, 2010

linux.conf.au 2011 call for papers

We're seeking a wide range of papers across the whole spectrum, encompassing programming and software to desktop and userland, education, community and law...

Some typical topics (but not limited to these) include:

  • Aspects of kernel development, including recent data structures and algorithm developments
  • Database and File system developments
  • Desktop topics, covering aspects of the user experience
  • Networking topics, from device drivers to servers
  • Novice user's introduction to exploring FOSS
  • Professional development, including Software Engineering & System Administration techniques
  • Scalability, both embedded and enterprise
  • Development topics, including concurrency and toolchain advancements
  • Open Source Software usage, including business, education & research
  • Graphics & sound advancements, from low level drivers to end-user applications
  • Open Source culture, including open content creation

More information is here, and submissions close August 7.

Unlike for the previous three conferences (Melbourne 08, Hobart 09, Wellington 10), I'm not heavily involved in selections: I will be reviewing abstracts but not (co-)chairing the process. I probably won't even be at the infamous day-long meeting to finalise selections, a long awaited visit from Andrew's sister will clash.

I'm hoping to attend the conference — it's fairly safe to say by now that unlike this year, it will not clash with giving birth — but I'm not sure yet. It probably will clash with producing a PhD thesis, and ACL (computational linguistics' main conference) usually has a deadline that nicely clashes with LCA too.

Flying

[This is not my bicycle]

Due to tyre troubles with my other two bikes I was "forced" to ride my Cannondale road bike to work this morning. The first time I've ridden it since well before Hikari was born last November. Wow, that bike is light and voom it flies. (And I put the brakes back together correctly after shipping it from Sydney!)

In semi-related news, its really warm in Tokyo at the moment.

July 18, 2010

MariaDB at OSCON 2010

We’re at OSCON and today has been fabulous – I’ve just been connecting with old friends, and making new friends, and all this is what makes the travel experience completely worthwhile. If you’re at OSCON, why not come to a couple of BoF’s:

  1. MariaDB: The Community Fork of MySQL at 8pm on Monday 19/07/2010. Great for a general overview of MariaDB, for beginners to the advanced folk to come to.
  2. MariaDB: Features In-depth at 9pm on Wednesday 21/07/2010. Great if you’re a more intermediate to advanced user of MySQL/MariaDB and want to know more about the additional features MariaDB has to offer, and what else it might offer in the near future (i.e. what are you requesting).

Related posts:

  1. MariaDB 5.1.42 released!
  2. Ubuntu 10.04 LTS released, MariaDB 5.1.44/5.2-BETA VM’s available
  3. MariaDB 5.1.44 released



July 17, 2010

OpenSUSE users have a choice of database now!

A big shoutout to the openSUSE team: openSUSE 11.3 has been released! And with it comes a choice for the database users out there – you can now choose between MySQL or MariaDB.

I wrote in February 2010 that MariaDB was in the unstable repository for openSUSE, and was informed by Michal that it would be included a little while ago. The version that is shipping is MariaDB 5.1.44, and why not give it a download?

So try it. Install via zypper in mariadb.

Related posts:

  1. Recently in MariaDB #1
  2. MariaDB 5.1.44 released
  3. MariaDB 5.1.42 released!



Cuckoos and Crackers

On a friend's advice, this week I found at the library and read The Cuckoo's Egg: Tracking a Spy through the Maze of Computer Espionage by Clifford Stoll. Wow. What a read: a true story about an astronomer-programmer who as a beginning system administrator finds evidence of an intruder and ends up spending the better part of a year tracking him down to West Germany where he's involved in selling US military information to the KGB. Wow.

I think I'm a little behind the times. This book was published in 1991 and it feels like everyone involved in computing apart from me knows about it by now. The fact that the story is over 20 years old now makes the book fascinating for a number of reasons apart from the simple tracing of the cracker. Stoll goes into detail about the lifestyle at the Lawrence Berkeley Laboratory and his own thoughts on the political and social responsibilities of computer network users.

The story is from a time when, at least in academia, computer systems were relatively open. Multiple users shared computers generally in order to work collaboratively. One of his conclusions in the book is that we must work hard to maintain trust rather than put energy into abusing that trust, because of the damage it does to the network and how easily we can work together.

I guess we failed. Not that I was old enough to do anything about it but it's a damn shame to see that that's how it went. It's a fact of online life now that everything online needs to be locked down. Even if you want to be share your data, the more mechanisms you make available for that data to be shared the more software you're exposing which could potentially have bugs in it. And inevitably there are people out there who wish to exploit those bugs, for a variety of reasons. Sigh.

The second interesting point was in his epilogue discussion about the transmission of a worm: a conclusion that computer networks have robustness due to the diversity of types of nodes. A virus for a VAX can't run on an IBM system, etc. Though he couldn't possibly have foreseen it at the time, we're seeing a heavy convergence towards web applications right now. What used to be a diversity of operating systems with standard network protocols is now becoming a diversity of web browsers with standard markup and javascript.

I think he has valid point even though it was made quite a while ago now. The ability for us to have operating system/browser diversity derives directly from open standards and open implementations. Look what happens when there's a bug in Adobe's flash implementation. Oops, everyone's vulnerable. And being closed source doesn't help. Let's keep our standards open.

The final point which I found particularly interesting was Cliff's own change of opinion. Initially he took a fairly loose apathetic view that breaking into other systems for fun could be just playful or even a good thing if it exposed problems. By the end of his ordeal he decided that the very act of messing with other people's systems is damaging simply because of the goodwill and trust which is lost, not to mention the amount of time which people like him have to spend working on problems they wish they didn't have.

A great book. Go read it sometime. You wouldn't have to be a computer person to appreciate it either, but it probably helps. :)

July 16, 2010

16 Jul 2010

I've just been working on Evolution's reply code, and have added a couple more of those annoying "nag pop-ups", including this one which I expect a lot of people will appreciate when they don't get the resulting mail:
Evolution nag pop-up for replying to too many recipients

It's currently set to trigger if you hit 'Reply to All' on a message with more than 15 recipients; unless it's a mailing list message. And of course you can see that it's trivial to turn it off if you never want to see it again.

I've also taken a moment to write down and post some thoughts on the 'Reply to All' vs. 'Reply to List' debate for mailing list messages.

July 15, 2010

Turkey’s Internet Censorship

The political battle of censorship in Australia has been relatively futile so far for those of us against it. By its proponents, issues have been muddled together, sensationalist terms like "child porn" have been bandied about and claims have been made that it's safer for Australian children using the Internet. That last part is supposed to be the official reason for the policy -- or is the one written down, at least. Senator Conroy redefines the purpose of the filter every time he opens his mouth.

Those of us against it are trying to cut through this nonsense and point to the bigger issues. Suppose that they were only out to block child pornography websites. The material is obviously bad. The material is obviously illegal. But it still does not make the filter the right policy because of the level of the trust we put in the Government not to abuse their power now and in the future. (And in reality they want to block "refused classification" material, which is a very fuzzy definition which definitely includes material which is legal to own.)

So can we trust the Government? My instinct says no. However there's nothing like the benefit of hindsight. Gizmodo has recently posted a short article about what's happened in Turkey since the introduction of their filter, including a 3-minute video report [youtube] from Al Jazeera about how badly the situation has devolved. Not only are all kinds of websites being blocked for political or religious reasons, but no progress is being made in the attempt to have the laws appealed.

Could it happen in Australia? You bet. You bet our civil rights on it.

EDIT: Another highly recommended article about the flaws of censorship in general:  The State of trust: it's a one way street

July 13, 2010

13 Jul 2010

Yay Brazil!. They're making it illegal to use DRM to prevent "fair dealing" with copyrighted works, or access to works which are in the public domain. It's also legal to "crack" DRM if you're only doing it for the purpose of "fair dealing".

So, for example, it would be legal for me to crack the DRM on the eBooks I buy, which is necessary just so that I can read them. Currently I have to break the law just to be able to buy and use eBooks.

UK citizens, go here and add your vote; it's very simple to register if you haven't already done so.

kexec-tools 2.0.2-rc2 Released

[Sumo]

I have released version 2.0.2-rc2 of kexec-tools. the user-space portion of kexec a soft-reboot and crash-dump facility of Linux and Xen. The only change since 2.0.2-rc1 is to include a number of files that were missing from the tarball which caused build failures on x86_64, ppc, ppc64 and arm.

The code is available as a tarball here and in git here. More information is available in the announcement email.

July 12, 2010

LMSR Implementation Notes, two

At the end of my previous post I mentioned some thoughts on dealing with more interesting initial states ($q^0$). We’ll define our initial state by choosing the amount of funds we’re willing to lose $F$, and a set of initial prices $0 < p_i(q^0) < 1$. Unless $p_i(q^0) = \frac{1}{n}$ for all $i$, we will be forced to set $q^0_i > 0$ in some (possibly all) cases. We will treat this as implying a virtual payout from the market maker to the market maker.

The maximum loss, is then given by $C(q^0) – \min(q^0_i) = F$ (since the final payout will be $q_j – q^0_j$, the money collected will be $C(q)-C(q^0)$, and $C(q) \ge q_j$).

If we wish to restrict quantities $q_i$ to be integers, we face a dificulty at this point. Working from the relationship between $p_i(q^0)$ and $q^0_i$ gives:

\[ \begin{aligned}

p_i(q^0) & = \frac{e^{q^0_i/\beta}}{\sum_{j=1}^{n}{e^{q^0_j/\beta}}} \\

& = \frac{e^{q^0_i/\beta}}{e^{C(q^0)/\beta}} \\

& = e^{q^0_i/\beta - C(q^0)/\beta} \\

\beta \ln( p_i(q^0) ) & = q^0_i - C(q^0) \\

q^0_i & = C(q^0) + \beta \ln( p_i(q^0 ) )

\end{aligned} \]

Since $C(q^0)$ is independent of $i$, we can immediately see that the $i$ with minimal $q^0_i$ will be the one with minimal price. Without loss of generality, assume that this is when $i=1$, then we can see:

\[ \begin{aligned}

F & = C(q^0) - q^0_1 \\

& = C(q^0) - \left( C(q^0) + \beta \ln(p_1(q^0)) \right) \\

& = - \beta \ln(p_1(q^0)) \\

\beta & = \frac{F}{\ln\left(p_1(q^0)^{-1}\right)}

\end{aligned} \]

In the case where $p_i(q^0) = \frac{1}{n}$ is common for all outcomes this simplifies to the formula seen in the previous post.

Note that this is unlikely to result in a value of $\beta$ that is particularly easy to work with. However simply rounding down to the nearest representable number works fine — since $\beta$ is in direct proportion to the amount of funds at risk, this simply rounds down the amount of funds at risk at the same rate.

Likewise, keeping track of $p_i(q)$ as an implementation choice will restrict us to rational prices, and thus likely irrational values for $q_i$. However it’s likely we’d prefer to only offer precisely defined payoffs for precisely defined costs, even if only for ease of accounting. In order to deal with this, we can treat $q_i = m_i(q) + g_i(q)$ where $m_i(q) \ge q^0_i$ represents the (possibly increasing) virtual payout the market maker will receive, and $g_i(q)$ are the (integer) payouts participants will receive. In particular, we might restrict $q^0_i \le m_i(q) < q^0_i + 1$, so that we can calculate costs and payouts using the normal floor and ceiling functions and ensure any proceeds go to participants.

This gets us very close to being able to adjust the outcomes being considered dynamically; so that we can either split a single outcome into distinct categories to achieve a more precise estimate, or merging multiple outcomes into a single category to reduce the complexity of calculations.

If we look at changing the $m \dotso n$th outcomes from $q$ into new outcomes $m’ \dotso n’$ in $r$, then our presumed constraints are as follows.

First, if this is the most accurate assignment between the old states and the new states we can come up with (and if it’s not, use those assignments instead), then we need to set the payout for all the new cases to the worst case payout for the old cases:

$$ g_{i’}(r) = \left\{ \begin{array}{l l}

g_i(q) & \quad 1 \le i’ < m \\

\max_{m \le i \le n}(g_i(q)) & \quad m’ \le i’ \le n \\

\end{array} \right. $$

Also, since we’re not touching the prices for the first $m-1$ outcomes, and our prices need to add up to one, we have:

\[ \begin{aligned}

p_{i'}(r) & = p_{i'}(q) \quad \forall 1 \le i' < m \\

\sum_{i'=m'}^{n'} p_{i'}(r) & = \sum_{i=m}^{n} p_i(q)

\end{aligned} \]

And most importantly, we wish to limit the additional funds we commit to $\Delta F$ (possibly zero or negative), and thus $C(r) = C(q) + \Delta F$.

Using the relationship between $p_i(r)$ and $r_i$ again, gives:

\[ \begin{aligned}

C(r) & = r_i - \gamma \ln(p_i(r)) \\

& = m_i(r) + g_i(r) - \gamma \ln(p_i(r)) \\

\Delta F & = m_i(r) + g_i(r) - C(q) - \gamma \ln(p_i(r)) \\

\Delta F & \ge g_i(r) - C(q) - \gamma \ln(p_i(r)) \\

\Delta F & \ge \left\{ \begin{array}{l l}

g_i(q) - C(q) - \gamma \ln(p_i(q)) & \quad 1 \le i < m \\

g_i(r) - C(q) - \gamma \ln(p_i(r)) & \quad m \le i \le n \\

\end{array} \right. \\

\Delta F & \ge \left\{ \begin{array}{l l}

g_i(q) - C(q) - \left(q_i - C(q)\right) & \quad 1 \le i < m \\

\max_{m \le j \le n}(g_j(q)) - C(q) - \gamma \ln(p_i(r)) & \quad m \le i \le n \\

\end{array} \right. \\

\Delta F & \ge \left\{ \begin{array}{l l}

-m_i(q) & \quad 1 \le i < m \\

\max_{m \le j \le n}\left( g_j(q) - \left( q_j - \beta \ln(p_j(q)) \right) \right) - \gamma \ln(p_i(r)) & \quad m \le i \le n \\

\end{array} \right. \\

\Delta F & \ge \left\{ \begin{array}{l l}

-m_i(q) & \quad 1 \le i < m \\

\max_{m \le j \le n}\left( -m_j(q) + \beta \ln(p_j(q)) \right) - \gamma \ln(p_i(r)) & \quad m \le i \le n \\

\end{array} \right. \\

\end{aligned} \]

Setting where $\mu$ to be the modified outcome with maximum payout (that is, $g_\mu(q) = \max_{m \le j \le n}(g_j(q))$, $m \le \mu \le n$) and $\nu$ to be the least new price (so $m’ \le \nu \le n$ such that $p_\nu(r) = \min_{m’ \le j \le n’}(p_i(r))$) lets us simplify this to:

$$ \Delta F \ge -m_i(q) \quad \forall 1 \le i < m $$

and

$$ \gamma \le

\frac{\Delta F + m_\mu(q) + \beta \ln\left(p_\mu(q)^{-1}\right)}{\ln\left(p_\nu(r)^{-1}\right)} $$

Since $m_i(q) \ge 0$, one simple approach to satisfying the inequalities is to simply drop the $m_i(q)$ terms, giving:

$$

\Delta F \ge 0 \quad \text{and} \quad

\gamma \le \frac{\Delta F + \beta \ln(p_\mu(q)^{-1})}{\ln(p_\nu(r)^{-1})}

$$

This has the drawback of not providing the maximum liquidity for the funds thought to be at risk, however.

July 10, 2010

LMSR Implementation Notes

Some additional notes on implementing Hanson’s Logarithmic Market Scoring Rule, based on David Pennock’s post from 2006.

Usage is to is to pick $n$ distinct outcomes, such that exactly one will be true, and then to trade contracts that correspond with each outcome, so that if the outcome occurs the corresponding contract has a unit payoff, and otherwise is worthless. The market scoring rule provides a way for a market maker to set and update prices for the outcomes no matter how they might be bought and sold. While the market maker’s worst-case loss is limited to a fixed amount, $F$, this is also the usual outcome.

The scoring rule uses a cost function, defined as:

$$ C(q) = \beta \ln\left( \sum_{i=1}^{n}{e^{\frac{q_i}{\beta}}} \right) $$

At any point, if event $i$ occurs, the payoff owed to participants is $q_i$. In order to achieve any given combination of payouts per outcome, a participant need simply pay $C(q+\delta) – C(q)$ where $\delta_i$ is the participant’s desired payout for event $i$.

Prices thus vary non-linearly depending on both current payoff’s expected, and desired payoff. However a number of properties can be easily verified. First, the total payout for any event $j$ is no more than $C(q)$:

\[ \begin{aligned}

C(q) & = \beta \ln\left( \sum_{i=1}^{n}{e^{\frac{q_i}{\beta}}} \right) \\

e^{\frac{C(q)}{\beta}} & = \sum_{i=1}^{n}{e^{\frac{q_i}{\beta}}} \\

& \ge e^{\frac{q_j}{\beta}} \\

C(q) & \ge q_j

\end{aligned} \]

If we define the initial state, $q^0$ by $q^0_i=0$ for all $i$, then $C(q^0)=\beta \ln(n)$. $C(q^0)$ is the maximum amount we can lose (since we will have received the remaining $C(q)-C(q^0)$ from participants), and as such, we can define $\beta$ in terms of the funds the marked maker can afford to lose, $F$, as:

$$ \beta = \frac{F}{\ln(n)} $$

We can see that the cost of buying a payout of $p$ in all scenarios (which we will denote as $\delta = p\iota$, meaning $\delta_i=p$ for all $i$) is exactly $p$:

\[ \begin{aligned}

C(q+p\iota) & = \beta \ln\left( \sum_{i=1}^{n}{e^{\frac{q_i+p}{\beta}}} \right) \\

& = \beta \ln\left( \sum_{i=1}^{n}{e^{\frac{q_i}{\beta}} e^{\frac{p}{\beta}}} \right) \\

& = \beta \ln\left( \sum_{i=1}^{n}{e^{\frac{q_i}{\beta}}} \right) + \beta \ln\left( e^{\frac{p}{\beta}} \right) \\

& = C(q) + p \\

\end{aligned} \]

The instantaneous price of each contract is given by the derivative of the cost function, which works out to be:

$$ p_i(q) = \frac{\partial{C(q)}}{\partial{q_i}} = \frac{e^{\frac{q_i}{\beta}}}{\sum_{j=1}^{n}{e^{\frac{q_j}{\beta}}}} $$

We can directly observe from this that at any time the instantaneous prices of all the events will be between 0 and 1, and that they will sum to exactly 1. Furthermore, if we maintain a record of the values of $C(q)$ (which represents the sum of funds received from participants and the maximum loss) and $p_i(q)$, we can calculate $q_i$:

\[ \begin{aligned}

p_i(q) & = \frac{e^{\frac{q_i}{\beta}}}{\sum_{j=1}^{n}{e^{\frac{q_j}{\beta}}}} \\

& = \frac{e^{\frac{q_i}{\beta}}}{e^{\frac{C(q)}{\beta}}} \\

& = e^{\frac{q_i}{\beta} - \frac{C(q)}{\beta}} \\

& = e^{\frac{q_i - C(q)}{\beta}} \\

\beta \ln\left( p_i(q) \right) &= q_i - C(q) \\

q_i &= C(q) + \beta \ln\left( p_i(q) \right) \\

\end{aligned} \]

Note that since $0 \lt p_i(q) \lt 1$, then $\beta \ln\left( p_i(q) \right) \lt 0$ and $q_i \lt C(q)$ as expected.

If we partition the possible states into three disjoint sets, $W$, $L$ and $I$, such that

$$ \delta_i = \left\{ \begin{array}{l l}

-c & \quad \mbox{ iff $i \in L$ } \\

0 & \quad \mbox{ iff $i \in I$ } \\

g & \quad \mbox{ iff $i \in W$ }

\end{array} \right. $$

For notational convenience, we will write $p_S(q) = \sum_{i \in S}{p_i(q)}$. If we set $c > 0$ and $C(q+\delta) = C(q)$, we then can determine $g$:

\[ \begin{aligned}

C(q+\delta) & = C(q) \\

\beta \ln\left( \sum_{i=1}^{n}{e^{\frac{q_i+\delta_i}{\beta}}} \right)

& = \beta \ln\left( \sum_{i=1}^{n}{e^{\frac{q_i}{\beta}}} \right) \\

\sum_{i=1}^{n}{e^{\frac{q_i+\delta_i}{\beta}}}

& = \sum_{i=1}^{n}{e^{\frac{q_i}{\beta}}} \\

\sum_{i \in L}{e^{\frac{q_i-c}{\beta}}}

+ \sum_{i \in I}{e^{\frac{q_i}{\beta}}}

+ \sum_{i \in W}{e^{\frac{q_i+g}{\beta}}}

& = \sum_{i \in L}{e^{\frac{q_i}{\beta}}}

+ \sum_{i \in I}{e^{\frac{q_i}{\beta}}}

+ \sum_{i \in W}{e^{\frac{q_i}{\beta}}} \\

\sum_{i \in L}{e^{-\frac{c}{\beta}} e^{\frac{q_i}{\beta}}}

+ \sum_{i \in W}{e^{\frac{g}{\beta}} e^{\frac{q_i}{\beta}}}

& = \sum_{i \in L}{e^{\frac{q_i}{\beta}}}

+ \sum_{i \in W}{e^{\frac{q_i}{\beta}}} \\

e^{-\frac{c}{\beta}} \sum_{i \in L}{e^{\frac{q_i}{\beta}}}

+ e^{\frac{g}{\beta}} \sum_{i \in W}{e^{\frac{q_i}{\beta}}}

& = \sum_{i \in L}{e^{\frac{q_i}{\beta}}}

+ \sum_{i \in W}{e^{\frac{q_i}{\beta}}} \\

e^{-\frac{c}{\beta}} \sum_{i \in L}{p_i(q)}

+ e^{\frac{g}{\beta}} \sum_{i \in W}{p_i(q)}

& = \sum_{i \in L}{p_i(q)}

+ \sum_{i \in W}{p_i(q)} \\

e^{-\frac{c}{\beta}} p_L(q)

+ e^{\frac{g}{\beta}} p_W(q)

& = p_L(q) + p_W(q) \\

e^{\frac{g}{\beta}} p_W(q)

& = p_L(q) + p_W(q) - e^{-\frac{c}{\beta}} p_L(q) \\

e^{\frac{g}{\beta}}

& = 1 + \frac{p_L(q)}{p_W(q)} \left(1 - e^{-\frac{c}{\beta}} \right) \\

g & = \beta \ln\left( 1 + \frac{p_L(q)}{p_W(q)} \left(1 - e^{-\frac{c}{\beta}} \right) \right) \\

\end{aligned} \]

Since $c > 0$, $e^{-\frac{c}{\beta}} \lt 1$ and $g$ is well defined and positive. This allows us to purchase $c\iota$ and $\delta$ at a total cost of $c$, with the result that we end up losing $c$ if an event in $L$ occurs, we end up breaking even if an event in $I$ occurs, and we gain $g$ if an event in $W$ occurs.

This provides a fairly straightforward way to calculate gains for a given cost using the prices, rather than the cost function directly.

Rather than choosing a particular amount to pay for a particular gain, it’s possible to determine how much it will cost to change the prices in a particular way. We might take the same sets, $W$, $I$, and $L$ and instead decide to adjust the prices as follows:

$$ p_i(q+\delta) = p_i(q) \cdot \left\{ \begin{array}{l l}

y & \quad \mbox{ iff $i \in L$ } \\

1 & \quad \mbox{ iff $i \in I$ } \\

x & \quad \mbox{ iff $i \in W$ }

\end{array} \right. $$

If we take $p_W(q+\delta) = p_W(q) \cdot x = p_W(q) + \rho$ then since the prices always add to one, $p_L(q+\delta) = p_L(q) \cdot y = p_L(q) – \rho$ and $y = 1 – \frac{p_W(q)}{p_L(q)}(x-1)$

The corresponding $c$ and $g$ values are then

\[ \begin{aligned}

x & = e^{\frac{g}{\beta}} \\

g & = \beta \ln(x) \\

& = \beta \ln\left( \frac{p_W(q) + \rho}{p_W(q)} \right) \\

& = \beta \ln(p_W(q+\delta)) - \beta \ln(p_W(q)) \\

\\

y & = e^{\frac{-c}{\beta}} \\

c & = -\beta \ln(y) \\

& = -\beta \ln\left( 1 - \frac{p_W(q)}{p_L(q)}(x-1) \right) \\

& = -\beta \ln\left( 1 - \frac{p_W(q)}{p_L(q)}\left( 1+\frac{\rho}{p_W(q)}-1 \right) \right) \\

& = -\beta \ln\left( 1 - \frac{\rho}{p_L(q)} \right) \\

& = \beta \ln\left( \frac{p_L(q)}{p_L(q) - \rho} \right) \\

& = \beta \ln(p_L(q)) - \beta \ln(p_L(q+\delta)) \\

\end{aligned} \]

Note that this assumes each price in $W$ is multiplied by the same amount, and similarly for each price in $L$. This also has the benefit that it maintains the relative prices within $W$ and $L$.

Obviously, $I$ can be the empty set. The advantage of having outcomes in $I$ is it allows participants to make their estimates conditional. For instance the statement “If X happens, Y will happen” should warrant a gain if “X and Y” happens, a loss if “X but not Y” happens, and no change if either “not X and not Y” or “not X but Y” happen.

This can also be used if a market may need to be cancelled. This may be done by having a “cancellation” outcome such that every action a participant may take results in that outcome being in $I$. This prevents people from exiting the market place at a profit before the outcome is known, however.

Splitting and merging outcomes is an interesting possibility — if the price of “it will happen on Tuesday” is $p$, then splitting that event into two events “it will happen on Tuesday morning” and “it will happen on Tuesday afternoon”, each with price $p/2$ would allow more precise predictions. Having this happen dynamically (such as when $p$ rises above a particular limit) would allow for precision only when it’s needed.

The drawback is that it may require an increase in $F$ (but not always — once Tuesday has been split into morning and afternoon, splitting Wednesday as well can simply reuse the same extra funds). Having different “sized” regions may also require some care. Representation also becomes a possible issue. Some of the maths for handling this might also help with handling initial state $q^0$ with different prices $p_i(q^0) \ne p_j(q^0)$.

MathJax

MathJax is pretty cool — it’s essentially a client-side JavaScript implementation of LaTeX, so you can write maths in ASCII, like “x^n + y^n = z^n”, surround it with dollar signs, and have it look like:

$$ x^n + y^n = z^n $$

And, of course, you can be more complicated if you like:

$$ C(\mathbf{q}) = b(\mathbf{q}) \log\left( \sum_i e^{\frac{q_i}{b(\mathbf{q})}} \right) $$

Inclusion in WordPress is easy: you unpack the MathJax beta on your website, add a “script” line so that the MathJax javascript is loaded, and it dynamically displays the maths when the page is loaded. It also manages to do it with real fonts, so you can select bits of the equations, and not have to deal with ugly images — oh, and it zooms nicely.

Of course, there’s a downside to having a client side script redisplay the formulas, and I suspect everyone reading via RSS will have already picked up on what it is…

Perdition 1.19-rc2 Released

[David Miller]

I have released version 1.19-rc2 of Perdition. There is a single change since 1.19-rc1 to fix a crash-bug on invocation of perdition in managesieve mode. Merge madness on my part. Thanks to Filipe Carvalho for reporting it. The fix is here.

The code and related libraries are available as tarballs here. More information is available in the announcement email. More information about perdition here.

July 07, 2010

Bringing friendship back to social networking

Recently I was pondering on the Diaspora project's one month report, wondering how exactly comments on status updates were being routed.

Does the person who owns the status update receive the comment on their seed and potentially have the opportunity to review it before it is broadcast? That seems reasonable. Really, though, the comment belongs to the person who wrote it. Shouldn't their seed have the right to send it out to all its peers regardless of what the person who posted the original update thinks?

It's fairly obvious in this case that the status update seed should be the definitive source of any comments. We all appreciate the idea that if the status update belongs to you, then you should have some level of control over what content is associated with it.

The problem becomes more complicated when you take a photo of your friends and want to share it on your seeds, including tags of the identities of everybody in the photo. The way Facebook operates is that even though you own the photo, the user who is tagged has the option of removing that tag, whether or not the photo poster wanted to keep it there. Clearly this is isn't enforceable in Diaspora where you only have control over your own seed.

On one hand, the photo and its tags belong to the person who shared the photo and contributed the tags. On the other, the tagged person doesn't own anything except the identity, but it would be nice if they could have some control over being tagged. How should this system mediate this?

The critical thing is that we're communicating with our friends. I would propose a system like this: your friend's seed announces its policy on photos: (a) Please don't tag me, (b) If you tag me, please don't display it until I've reviewed the photo, or (c) Go ahead, tag me.

When you attempt to tag this friend in a photo it will check on the policy. If it's option (a) Don't tag, then it will refuse to do so (or at least make you jump through hoops. Shame on you for not respecting your friend's wishes). If it's option (b) Check first, it will send away a review request without you having to think about it. If it's option (c) Tag me, it will simply apply the tag.

Not only does this work to allow people to stipulate their tagging wishes, but it falls back on good old human respect to get along. Just because you have the technical ability to do something disrespectful to a friend doesn't mean you should. Conversely, where we can use technology to facilitate flexible interaction rather than dictate policy, we should.

QotD: Rusty Russell

Side note: when a respected information source covers something where you have on-the-ground experience, the result is often to make you wonder how much fecal matter you’ve swallowed in areas outside your own expertise.

 — Rusty Russell in Superfreakonomics; Superplug for Intellectual Ventures

July 06, 2010

Sorry for the RSS hiccups

My apologies to all the users of aggregation services which just copped a swarm of outdated posts from my blog.

I’m restoring this old blog and merging it with a couple of sites I have so stay tuned and thanks again for enduring the noise.

Perdition 1.19-rc1 Released

[Hikari]

I have released version 1.19-rc1 of Perdition, a mail retrieval proxy. This release includes numerous bug fixes and support for the manage sieve protocol.

The code and related libraries are available as tarballs here. More information is available in the announcement email. More information about perdition here.

July 05, 2010

Vote for Maria’s new name

Remember the idea behind renaming Maria, the storage engine? People regularly get confused with Maria the storage engine, and MariaDB the database. We got a whole lot of good names, and here’s a short-listed amount of names (short-listed by Monty Program employees down to about 15), and now its up to YOU to help us choose what the new name should be. Quick, go take the survey.

Voting ends on Friday, July 9, 2010, at 23:59UTC.

The winner will be announced on the first day of OSCON on July 19 2010. The winner walks away with a Meerkat net-top from System76.

Related posts:

  1. Quick notes: Monty Program Group Blog; Rename Maria
  2. Monty speaks about Maria
  3. Trying to reliably make MyISAM crash; Maria is sturdy as



Open Source Saves Malaysian Government RM188 Million

A money clipBack in January 2009, we found out that the Malaysian Government had saved about RM40 million using open source. In a little over a year, that number has been topped: over the past six years, the total costs savings are now quoted to be RM188.39 million (USD$58.54 million)! That’s a hell of a lot of money for software licenses, don’t you think?

Worth noting is that before the OSS Master Plan started, there were zero companies supporting OSS registered with the Ministry of Finance. Now more than half of the 4,000 companies do (53% is the quoted number). For more information, read the latest newsletter from MAMPU’s OSCC. Key takeaways:

  1. Saved RM188.39 million on software licenses over six years
  2. Successful OSS adoption in 691 government agencies by the end of 2009 (till April 1 2010, the number looks like it has increased to 699 agencies).
  3. In total, 95% of agencies are adopting some form of OSS solution, 87% are using it for back-end infrastructure (here its clear there’s Linux, MySQL in use), and 66% are using OSS on the desktop! (via OpenOffice.org and Firefox)*

* – Software use extrapolated from the actual OSS Master plan, and what was in the report in January 2009. I’m sure Joomla! is also used quite heavily, but never recall seeing it as the choice for CMS in the plan.

Related posts:

  1. Open Source saves Malaysian Government RM40 million
  2. Free and Open Source Software: Use and Production by the Brazilian Government
  3. Malaysian Government releases first Open Source software package – MyMeeting



July 02, 2010

Jono Bacon speaks to Oracle on the MySQL Community

Jono Bacon recently spoke with Luke Kowalski, Oracle VP in the Corporate Architecture Group, about community in the context of MySQL. I’ve known Jono for sometime now, and I first met Luke at the MySQL Conference & Expo 2010 – I found out that we have a common shared interest: Formula One racing! Jono is somewhat of an expert in online, opensource communities – he after all did write the definitive book that O’Reilly published titled The Art of Community.

The video has made its way online, and Jono wrote a brief (and you can watch the video within his post) about what was discussed. You can also get it as a podcast – just subscribe to Oracle Technology Network TechCasts in your podcatcher.

Its under 25-minutes to watch or listen to, and I’d highly recommend you to take a look if you care about community, MySQL and direction. Choice quote: “Oracle needs to make a firm commitment to acting within the culture and ethos of Open Source to have an effective, fulfilling relationship with the MySQL community”. Definitely watch the video.

Related posts:

  1. MySQL Community Rocks – look at all the contributed audio & video
  2. MySQL, with SHOW PROFILE and updated INFORMATION_SCHEMA, built from the Community tree
  3. Silona speaks about grids, databases, and open government



July 01, 2010

SMCKL #7 notes: Canon, bfm89.9, MyEG, P1 W1MAX

SMCKL’s case study night, SMCKL #7 also had some other very interesting case studies. I picked up a bunch of interesting bits.

Rachel from Canon Malaysia

  1. You can find them on Twitter at @CanonCameraMsia. They wish they had more followers on Twitter, because in comparison, their Facebook page has quite a lot more followers.
  2. They’ve got 4-5 people in the marketing department, and most of the marketing executives are very, very young. One thing that was made clear: the marketing folk at Canon are very, very passionate. Corporate policy denies them from browsing social media sites, so they bring their own laptops, and use DiGi Broadband at work, to get their social media stuff going.
  3. Their Facebook strategy has garnered them over 56,000 fans. Find them at their Facebook page. I just visited it, and realised that they have a landing tab (Explore! Canon), so they have invested in Facebook a little. Kudos! To bring traffic, they’ve also decided to engage Canon users: they change their Facebook profile picture on a daily basis. As a reward, you get your avatar, and name displayed on their Facebook page for a day. If you are the monthly winner, you actually get a prize – a printer or a camera! Canon has found this to be a rather successful campaign and it clearly works for them.
  4. They also have a presence on YouTube – iLikeCanonCamera. There are commercials, and more. Now with more video on some of the cameras, like the 7D and 5DMkII, they’re also looking at using Vimeo for video.
  5. They are going to build a virtual community at: EOS World. They already make use of newsletter blasting. They also mention they have a EOS Buddy Club, but that requires some qualifications (i.e. you have to own some of their upper-end DSLRs). The EOS World is open for all. I learned something new with regards to the Buddy Club – I qualify to join!
  6. They do blogger engagement: Brian, aka goldfries, Kim Ong, a blogger called Ah Beng (no link because I couldn’t find it via Google easily), and Namewee.
  7. Something they believe in: under-promise, over-deliver

Freda Liu from bfm89.9 (popular radio station)

  1. They do their own surveys, so it may be a little different in comparison to what other radio stations that pay external organisations to come up stats do. With over 500+ respondents, they found that 46% of BFM listeners earn more than RM5,000 per month, and over 45% of listeners live in homes with monthly income greater than RM10,000.
  2. Their most popular listener age group? 26-30. The next highest are those aged 30-35.
  3. They upload all their podcasts to their website. This is true for all their shows. They get over 7,000 podcast downloads, per week! They might be the most successful Malaysian based podcast channel, for what it’s worth.
  4. They want to give more personalisation via their website in due time. Currently they tweet at @BFMRadio, about upcoming interviews, interviews released as podcasts, and they do try to ensure they are conversational (true – this is one aspect I like about their account), and they do run contests.

KK Chin, MyEG

  1. First up, KK Chin is very funny! MyEG is not a government agency, in fact, they are a public listed company, established in 2001. The reasons they are on social media? “You’re there, we’re there”.
  2. They have about 800,000 users in the MyEG database. They process 3,500 road tax renewals daily (this was five months ago); now, at the end of June 2010, they do 7,000 per day! In comparison, JPJ does about 22,000/day. Incidentally, you can also renew your insurance online, and they’re seeing uptake of 120-150 daily renewals.
  3. They had a Facebook page, started in November 2009, iwth over 8,000+ fans. However in mid-May 2010, Facebook decided to delete the page. They have no idea why.
  4. Their Twitter account started at the same time. They look for words like “myeg”, “road tax”, and “saman”. They will then reply and try to help you! That’s what I call customer service, plus a great customer acquisition strategy. From what I gather, they occasionally also give away free road tax renewals, and more via Twitter/Facebook.
  5. MyEG has crazy ad spend on TV, as well as on AMP radio stations. In comparison, they spend close to 8-10% on social media.

Kenny Wong, P1 W1MAX

  1. P1 believes they were the first to have a digital-specific customer care team in Malaysia. The team has about 8-9 people to engage on social media!
  2. They know being a service provider they do get some flack. They have spent the last 6 months expanding their network, and now believe they will come back on SM a lot more!
  3. They help sponsor Project Alpha, and 15Malaysia. 15Malaysia had some interesting stats: over 510,000 downloads, over 3.2 million streamed, over 15 million page views, nearly 94,000 Facebook fans, and over 1,600 Twitter followers. The most popular video (by viewer-ship and being shared) was Potong Saga.
  4. P1 harnessed YouTube for their ads at one stage. They had spent their limited budget on creating videos, that there was no more money for buying air-time. YouTube was their free distribution mechanism.
  5. Kenny mentioned that if you’re in customer service, working for a telco/ISP< you definitely like pain! You'll get more brickbats, rather than accolades. Sometimes, the same sort of person will condemn, but they will also complement - its about how much people value and resonate with the brand/service. Remember to take the hits. Laugh at yourself. And do that well.

Closing thoughts

Would like to say that SMCKL was very well organised. I see these events just getting larger and larger, and it just gets better, and better. I also enjoyed every single presentation given. I learned something new from every one of them. Yesterday, we also received some schwag: Cziplee sponsored a Moleskine, Papa John’s gave a 50% discount voucher, and Crocs provided a RM20 discount voucher, valid forever ;)

The question about metrics was never really answered. Kenny mentioned brandtology, radian6, and red dot. What is the cost acquisition of a fan? What is the ROI on fans? What’s a fan worth? More fans does not necessarily mean a loyal customer base. Having irrelevant non-targeted visitors also don’t necessarily help.

I also wanted to ask: why invest heavily in Facebook, when they may just zap your page? Sure, go there because people are there. But maybe acquire fans via Facebook, and migrate/sync them to a self-hosted BuddyPress or something?

Related posts:

  1. MyEG, JPJ, VicRoads on road taxes: the difference is trust?
  2. Video cameras and Linux Redux – Canon MV920
  3. Crocs Malaysia and race based market segmentation



June 30, 2010

Crocs Malaysia and race based market segmentation

Update – Thursday, July 1 2010, 5.38pm UTC+8: What happened, happened. What was said, was said. Oftentimes, a slip of the tongue, really is no fault of the mind. Maybe I was a little too harsh here, so when reading this 1,000-word essay, just keep that in mind. I’m sure the MarCom Manager did not actually mean what she said. The calibre of her presentation was good, and we learned some new things about their digital marketing strategy. In fact, you might just want to go read the other case studies (Canon, BFM 89.9, MyEG, P1 W1MAX)! To err is human, to forgive divine. Let’s all take a chill pill, and have a beer or something ;-)

I graced a SMCKL meetup tonight. I haven’t been in about a year (scheduling conflicts); it was fabulous to catch up with all my friends, learn new things and I have quite some notes that I’ll talk about soon. But this is about my relationship to Crocs, and Crocs Malaysia, at large.

I love my Crocs Yukon. In fact, I love it so much, I pretty much buy a new pair once every six months! You might wonder why. The soles tend to wear out, not evenly, just around the midfoot, usually towards the forefoot. But this quick wear has never been a problem to me, because I love my Crocs; I wear them everyday. Crocs™ Yukon | Crocs, Inc.

If I’m on my feet, my Crocs are wrapped around them. The current pair (in its six months with me) is world travelled – it’s been to: Singapore, London, Thailand, Finland, Iceland, Germany, the United States, and China. Its been on numerous planes (Dopplr suggests my carbon total stands at 17,414kg CO2). Its been through freezing winters and rather hot tropical weather. Crocs have been on me when I meet deputy-ministers, ministers, and even a certain prime minister. I wear Crocs to meetings. I wear Crocs when I give talks. Some friends refer to my pairs of Crocs as “government Crocs”. We always have a chuckle at that.

The bottom-line is that it’s quite hard to find me in a pair of shoes. I’m usually always in Crocs. The very rare occasion of me wearing shoes has me wearing a very comfortable pair of Geox shoes, or an amazing pair I picked up from Marks & Spencer.

Back to SMCKL #7 – we had the Marcom Manager for the company that imports/distributes Crocs in Malaysia come speak to us. She mentioned the name of the parent company, but I never did get it, so for all intents and purposes, this is defined as “Crocs Malaysia“. Her presentation was rather interesting, and a few points I took away from it:

  1. They invest heavily in Facebook – they have the regular Crocs Malaysia page, as well as a Crocs Ideas page (which isn’t linked form the main page). They have over 14,652 fans that like their page. Content is largely what’s coming out in the stores, and what’s to become available.
  2. There’s some Twitter engagement (@crocsmalaysia), but it does not seem as popular as Facebook (other speakers had similar comments).
  3. They’re doing blogger engagement. For the year 2010, they’ve picked five: FeeqSays, Rebecca Saw, Marcky.my, MissyCheerio, and Christopher Tock. These bloggers are known as their “official bloggers”.
  4. While the arrangement isn’t made clear with the official bloggers, it is made clear that the official bloggers get free Crocs, in exchange for writing about how Crocs have influence on the daily life of the blogger. Bloggers are basically selling stories, pushing experiences, where their Crocs get involved. I don’t know if there’s a minimum post count/amount of engagement required, or its something you do in passing (which is why I said the arrangement isn’t clear). I do however think that with the proper metrics, story telling and sharing experiences, totally rocks.
  5. To engage with their readers, these official bloggers also get to run competitions on a weekly basis, and the rewards are free Crocs for winners. Good, it gets more Crocs out there, especially to the non-believers ;)

Everything seemed like it was going well, and she was doing a fabulous job at presenting her case study. Until the moderator asked her: How do you choose your bloggers?.

She does not go for “A-list bloggers” like Kenny Sia, or Nicolekiss. A-list Malaysian lifestyle bloggers. She went on to tell the audience that each blogger has value in them. I take it that everyone has a story to tell, and everyone has their own tribe. Then she talked about Feeq. She said that his target market is the Malay market. She did not want to only target the Chinese market. Crocs Malaysia targets all market segments: Malay, Indian and Chinese.

Wait, what? There’s race based segmentation and targeting in footwear? In social media? Online? We live in a time, where Malaysia is no longer just Malaysia; its the cliché of 1Malaysia. And here the faux pas, stated in front of a large audience, the yardstick used, is in which racial segment a blogger is popular in.

I immediately reacted:

Clearly disappointed that @crocsmalaysia is profiling/targeted by race. So much for being Malaysian. Time to find new footwear #smckl

I am trying to break your heartThere was another tweet which wondered what the difference in race brings? And whomever is behind the @crocsmalaysia account stated: it is just our way to reach out to all consumers of all races thru our bloggers. One might think that this is something only I took offence in; lucky for me, I was not alone. People came up to me, replied to me, and even direct messaged me on this very topic.

Social media, and the utility that it travels on, the Internet, does not discriminate against race, colour, creed, nationality, gender, and more. You should choose a blogger because she has a wide Malaysian audience! In Malaysia however, it seems ingrained that there will always be profiling, separation, and a system in where we don’t have one nation, but at least two separate nations. Maybe we only have the incumbent political parties to blame for this.

As an aside, I think it is important to distinguish what you say to a close group of friends, versus what you say on Twitter, versus what you say to a large audience of folk (i.e. when you’re speaking in public). When you start representing a brand, you’ve definitely got to start thinking about how you articulate yourself. A topic to talk about elsewhere.

Either way, I cannot remember when I first shelled out RM209 for my first Yukon. I expect it must have been late 2007, or at latest early 2008. My (extended, truly Malaysian) family is also big on Crocs. However, I think its time to wean myself off from Crocs, and find non-discriminatory footwear.

Related posts:

  1. Silly Malaysia – race cards pulled out, what happens when oil ringgit runs out?
  2. Malaysia stops censoring bloggers
  3. Malaysia Today Mirror



June 29, 2010

It's freezing

I hope Debian squeeze is not going to be released without the trivial fix for bug #564464 being applied, just because of any impending freeze. The regression itself wasn't introduced until near when the freeze was mooted, and still hasn't been fixed 6 months later.



Not having a beep that can be tuned in any way with "xset b" is a bit boring for those of us who don't let our desktop environment replace the useful beep signal with stupid noises.

June 27, 2010

Climate Change==Earth tilt

I was given a new explanation for climate change yesterday. A couple of guys had heard from the old timers (read, senile old fools) that the sun used to set in a different location when they were kids living in the same house, so the Earth must be tilting. When I explained that astronomers can pick up unexpected changes at the level of about 1/1000 of 1/3600 of a 1 degree, and we haven't picked up any such changes over the last few decades, they explained that astronomers must just be in cahoots with the climate scientists.



I love small country towns.



In unrelated news, everyone here is expressing surprise that the latest poll has Labor moving ahead again, possibly in response to Julia Gillard being sworn in. "No one I knows will ever vote for her!".



Dammit, I wish I didn't live in a safe Nationals seat.

June 24, 2010

Highlighting trailing whitespace in vim

This config snippet configures vim to highlight trailing whitespace in a horrendous red, making it easy to spot and remove.

:highlight ExtraWhitespace ctermbg=red guibg=red
autocmd Syntax * syn match ExtraWhitespace /\s\+$\| \+\ze\t/

Even better would be to only highlight it on lines that have been otherwise modified since the last commit…

June 23, 2010

The semiotic web

Quite a long time ago I read a fascinating article on semiotics and user-interface design. My recollection is that it made the argument that computer user interfaces could be broken up into roughly three branches: “menus”, where you have a few options to choose between, and that’s it; “WIMP paradigm” where you’ve got windows, icons, menus and a pointer and can gesticulate to get things done; and “command oriented” where you type commands in to have things happen.

While the WIMP paradigm is obviously pretty good, it’s restricted by its “metaphoric” nature: you have to represent everything you want to do with a picture — so if you don’t have a picture for something, you can’t do anything with it. In effect, it’s reduces your interaction with computers to point-and-grunt, which is really kind of demeaning for its operators. Can you imagine if the “communication skills” that were expected of you in a management role in business were the ability to point accurately and be able to make two distinct grunting noises?

On the other hand, if your system’s smart enough to actually do what you want just based on a wave of your hand that is pretty appealing — it’s just that when you want something unusual — or when your grunts and handwaving aren’t getting your point across — you can’t sit down and explain what you want merely with more grunts and pointing.

Obviously that’s where programming and command lines come in — both of which give you a range of fairly powerful languages to communicate with computers, and both of which are what people end up using when they want to get new and complicated things done.

It’s probably fair to say that the difference between programming languages and command line invocations is similar to essays and instant messaging — programs and essays tend to be long and expect certain formulas to be followed, but also tend to remain relevant for an extended period; an IM or a command line invocation tends to be brief, often a bit abbreviated, and only really interesting exactly when it’s written. Perhaps “tweet” or “facebook status update” would be a more modern version of IM — what can I say, I’m an old fogey. In any event, my impression is that the command line approach is often a good compromise when point-and-grunt fails: it’s not too much more effort, but brings you a lot more power. For instance,

$ for a in *.htm; do mv "$a" "${a%.htm}.html"; done

isn’t a very complicated way of saying “rename all those .htm files to .html”, compared to first creating a program like:

#!/usr/bin/env python
import os
for name in os.listdir("."):
    if name.endswith(".htm"):
        os.rename(name, name[:-4]+".html")

and then running it. And obviously, one of the advantages of Unix systems is that they have a very powerful command line system.

In any event, one of the things that strikes me about all the SaaS and cloud stuff is that there really isn’t much a linguistic equivalent to the command line for the web. If I want to do something with gmail, or flickr, or facebook I’m either pointing and grunting, or delving deeply into HTML, javascript, URLs, REST interfaces and whatever else to make use of whatever arbitrary APIs happen to be available.

A few services do have specialised command line tools of course — there’s GoogleCL, various little things to upload to flickr, the bts tool in devscripts to play with the Debian bug tracking system, and so forth.

But one of the big advantages of the web is that you aren’t meant to need special client side tools — you just have a browser, and leave the smarts on whichever web server you’re accessing. And you don’t get that if you have to install a silly little app to interface with whichever silly little website you happen to be interested in.

So I think there ought to be a standard “command line” API for webapps, so that you can say something like:

$ web www.google.com search -q='hello world'

to do a Google search for ‘hello world’. The mapping from the above command line to a URL is straightforward: up until the option arguments, each word gets converted into a portion of the URL path, so the base url is http://www.google.com/search, and options get put after a question mark and separated by ampersands, with regular URL quoting (spaces become plusses, irregular characters get converted to a percent and a hex code), in this case ?q=hello+world.

The obvious advantage is you can then use the same program for other webapps, such as the Debian BTS:

$ web bugs.debian.org cgi-bin bugreport.cgi --bug=123456 --mbox=yes
From mech...@...debian.net Tue Dec 11 11:32:47 2001
Received: (at submit) by bugs.debian.org; 11 Dec 2001 17:32:47 +0000
Return-path: 
Received: from gent-smtp1.xs4all.be [195.144.67.21] (root)
	by master.debian.org with esmtp (Exim 3.12 1 (Debian))
	id 16Dqlr-0007yg-00; Tue, 11 Dec 2001 11:32:47 -0600
...

It obviously looks cleaner when you use the shorter url (web bugs.debian.org 123456), although due to the way the BTS is setup, you also lose the ability to specify things like mbox format then.

Of course, web pages are in all sorts of weird formats, too: having Google’s HTML and javascript splatter all over your terminal isn’t very pleasant, for instance. But that’s what pipes are for, right?

$ web chart.apis.google.com chart --cht=p3 \
    --chs=400x150 --chd=t:2,3,5,10,20,60 \
    --chl='Alice|Bob|Carol|Dave|Ella|Fred' | display



chart

It’d probably be interesting to make “web” clever enough to automatically pipe images to display and HTML to firefox and so on, depending on what media type is returned.

Obviously you can use aliases just like you’d use bookmarks on the web, so saying:

$ alias gchart='web chart.apis.google.com chart'
$ alias debbug='web bugs.debian.org cgi-bin bugreport.cgi'

lets you type a little less.

Anyway, I think that makes for a kind-of interesting paradigm for looking at the web. And the “web” app above is pretty trivial too — as described all it does is convert arguments into a URL according to the given formula.

Things get a little more interesting if you try to make things interactive; a webapp that asks you your name, waits for you to tell it, then greets you by name is made unreasonably difficult if you try to do it on a single connection (with FastCGI and nginx for instance, the client has to supply the exact length of all the information you’re going to send before it will receive anything, and if you don’t know what you’re going to need to send up front…). Which means that so far my attempts to have web localhost bash behave as expected aren’t getting very far.

The other thing that would be nice would be passing files to remote web apps — being able to say “upload this avi to youtube” would be more elegant as web youtube.com upload ./myvideo.avi than web youtube.com upload <./myvideo.avi, but when web doesn’t know what “youtube” or “upload” actually means, that’s a bit hard to arrange. After all, maybe you were trying to tell youtube to do the uploading to your computer, and ./myvideo.avi was where you wanted it to end up.

Anyway. Thoughts appreciated.

June 20, 2010

HOWTO: Make your videos 10 minutes in length for YouTUBE

I recently made quite a number of videos, that span about 0.5hr or 1-hour in length. YouTube accepts videos that are 10-minutes long and up to 2GB in size. Blip.tv allows FTP uploads, but your filesize cannot exceed 1GB per video. An hour long video from the camera is about 2GB in size. This is quick guide as to how I split videos.

QTCoffee exists to solve this problem. Quick, simple install on Mac OS X, and I was on my way to creating split videos. The command I used:

splitmovie -o outputfile.mov -self-contained -no-fast-start -duration 10:00 sourcefile.avi

Now, in that directory, I get outputfile-1.mov, outputfile-2.mov, and so on. Easy as pie. Its donation-ware, but a commercial license will set you back USD$10. It occasionally reminds you to donate by popping up nagware in your web browser at random once every 10 times its used (I’ve not used shareware since Windows 3.1, and nag screens were a lot different back then). You can easily remove it (it tells you how, even!), by executing: defaults write com.pair.3am.QTCoffee BugMe NO .

Now YouTube is happy, and you can concatenate the files together as a playlist.

QTCoffee is ideal for me, because its scriptable, and can work on a large directory of videos. I wonder what the Linux variant for this is?

Related posts:

  1. YouTube, an MTV replacement
  2. Online videos, to a VCD, via Linux
  3. Looping images to the length of some audio



MyEG, JPJ, VicRoads on road taxes: the difference is trust?

On Twitter today, Christopher Tock stated:

If @myegclub cld have a service to notify road tax of a car nearing expiry date, I’m sure it’ll get more ppl to register under them! ;) #fb

I replied:

Instead, why not send new road tax one month before expiry by mail, and get payment via @myegclub ? @spinzer @nikicheong

Christopher then asked about the situation in which people don’t pay, and they receive their registration labels by mail. And Niki Cheong said that “many people try to find their way around paying for stuff here. ingrained in our lives”.

VicRoads website It got me thinking. In Victoria, Australia, VicRoads sends out registration labels and the certificate every year, about a month before expiry. You then head online, and pay for your registration using your credit card. I believe you might also be able to pay for it in cash at various locations (post office, etc.), though I’ve always done it online. You never ask – what if I don’t pay for it. You just do. I’m sure there are stiff penalties for lacking payment. But the process is easy – it comes to you, there’s no need to remember when things are expiring, and you can pay for it online, all without leaving the comfort of your home.

In Malaysia, JPJ does not remind you when your road tax is expiring. The onus is on you to know this. You then have to head to the post office to pay for it and get the registration label, and settle your insurance at the same time. Or you can avoid all this by sending it to your car’s service centre and they’ll take about two days to sort this out for you. Insurance can be paid via credit card, but your road tax needs to be in cash (or so UMW-Toyota tells me).

JPJ websiteThe government of Malaysia is heavily pushing their e-government services portal – MyEG. You can now renew your road tax online, thus getting rid of the whole visiting the post office or your manufacturer’s service centre. This is similar to what VicRoads does – you enter your registration number and it will tell you if you have a payment to be made or not. This is a step forward – except for the fact, that there is no reminder for you to renew your road tax!

Will it be hard to implement? Of course not. Part of the registration for MyEG includes you giving away your identity card (IC) number. A simple change to the database can ensure that you can tie several cars to your account and you can thus make payment on them. MyEG will know when the road tax is expiring, and a simple SMS message might suffice (bulk SMS costs are cheap). And if they’re more trusting, mailing out of the labels themselves!

But I guess Niki is right – it is ingrained that Malaysians might want to find away around the system, if the labels were sent out, pre-payment. Take a look at petrol kiosks: in Australia, you pump your petrol first, then go in to make the payment. You don’t think about running away after getting your full tank. You do this no matter how much petrol costs. In Malaysia, you pay for your petrol first. Either at the counter, or via credit card authorisation. No payment, no petrol.

I sincerely hope MyEG/JPJ figure this out at some stage. As we move towards a more knowledge-based economy, and the goals of the government certainly include getting Malaysia more in-tune with the rest of the First World, it probably makes sense that service delivery takes a notch up.

As a quick aside, do look at both the VicRoads and the JPJ website. Look at how the information is architected. How its presented. Its an interesting comparison. I wonder how many web designers go through this sort of thought process, when thinking about designing websites that are targeted for mass consumption of government services.

Related posts:

  1. SMCKL #7 notes: Canon, bfm89.9, MyEG, P1 W1MAX
  2. Filing taxes online in Malaysia
  3. Messaging matters



June 16, 2010

Happy 7th birthday, Linode!

Perhaps this is a familiar refrain for some readers… but I’ll say it again: Linode rock! On their 7th birthday, they are giving current and new users a huuuge RAM upgrade —  around 42% on average. From humble beginnings as a Linode 64 five years ago, my Linode 512 is now a Linode 768. Yikes!

Good news. Linode’s getting older.”

“… and Leon is getting laaaaarger!”

Already blessed with plenty of memory available for cache (which is very handy on a VM with relatively slow I/O), I now have even more headroom:

June 14, 2010

Ways to spend my time

Ideas already for my unexpected surfeit of time in 2010!

Joining the Sydney Recorder Society. I played recorder a lot in high school, through to seventh grade AMEB. I have very rarely played since, nor in fact really missed it, but that doesn't mean that I wouldn't enjoy playing again. I've also been interested in improving my singing and perhaps joining a choir for many years but up until now I've had money or time for music, never both.

My mothers' group is joining an indoor netball tornament. People tend to salivate at the idea of me playing defence for their netball teams; I'm not actually terribly good at anything other than casting a long shadow, nor do I find netball itself the most fascinating of sports, but it doesn't sound like it will be a terribly serious team and playing a season would probably be pleasant enough.

Finally doing some sewing on the machine we were given more than a year ago.

Working on the children-on-the-Internet project described here.

June 13, 2010

Nokia N900 gets a firmware update – PR1.2 for Maemo 5

Nokia N900 I’ve been using the Nokia N900 since March 2010, when Nokia decided to give one to every person that attended the launch. Its mostly been my roaming phone so far, and its been places ;)

Naturally, I found some problems which I very quickly wrote to Glen, who heads Product Management at Nokia Singapore/Malaysia/Brunei. I’m happy to state that these problems have been fixed in v10.2010.19-1 (PR 1.2) – see the changelog for a more complete list! This was released May 25/26 2010, worldwide, and I’m only now getting to update it. Let’s go through some of them…

Short codes (USSD) leading to “incorrect number”

I could not dial numbers that are formatted like *131#. The error message thrown at me: “incorrect number”. These codes are known as Unstructured Supplementary Service Data (USSD) codes.

I discovered this immediately within 24-hours of using the phone, largely because I had a Starhub prepaid card in it, and pretty much everything related to account management could not be done. DiGi in Malaysia uses this for checking your balance, and so on as well. Naturally, this was not something only I found – it was quite a problem on the forums too.

Pleased to report it has been fixed in PR 1.2.

Web browsing – landscape only?!?

Nokia N900Web browsing only works in landscape mode, and not portrait mode. I understand Nokia is pushing this connected device mantra, and its like your mini-computer replacement, but clearly, sometimes I want one-handed operations.

Today, it works. Not automatically – go to the web browser, look under Options – Settings, and make sure you select Enable Rotation. It takes a little over 1.5 seconds to rotate between landscape/portrait, and to be honest this seems a little slow, but it is at least fixed in PR 1.2.

Improved communications

You can now have video calls in Skype and Google Talk. You can also start using Facebook IM chat. Ovi Chat still remains. A killer? BBM chat integration!

Some things were however, not fixed, and I guess we might wait a few months before the next update happens.

  1. Copy via selecting text on the screen works in the Notes application. In the Messages application (Conversations), you can’t select using the screen, you have to use the keyboard (shift [uppercase] + up/down/left/right). This is still an inconsistent UI bug, and the messages app needs fixing
  2. More in what I consider inconsistent UI: the +/- hardware buttons are reversed when it comes to volume and zoom using the camera application. I know there is no +/- button per se, but I’d still regard this as a bug.
  3. Not a large bug in my book, but there is still no Multimedia Messaging Support (MMS).

The upgrade for me happened over the air. Half way through the upgrade I was a little worried about what might happen to my pictures. But it seems there is User Data Preservation (UDP), so I’m not complaining. Everything remained after the update – photos, contacts, text messages and more.

From a developer perspective, you’ll find that Qt 4.6.2 is now pre-installed. What does this mean to you? Check out the Qt Labs blog post on Qt 4.6.2 for Maemo 5. Kinetic scrolling is one of the things that excites me, with this change.

Anyway, if you’re a Nokia N900 user, get this firmware update. While it was non-destructive for me, your mileage may vary, so you might want to make backups.

Related posts:

  1. The Nokia E72: Quick impressions
  2. MeeGo: Quick thoughts on the Moblin and Maemo marriage
  3. Nokia 5800 XpressMusic Comes With Music



Participate in the Web of Trust Community Survey

Do you use Web of Trust (WOT)? Its a great plugin, available for Firefox, Chrome and other browsers, that help you browse the web in a safer fashion. From June 11 to June 18 2010, you can fill up the community survey, and stand a chance to win an Apple iPod Nano.

Related posts:

  1. NanooNanoo
  2. Apple launches the Malaysian Online Store
  3. Donating to Firefox add-ons



June 12, 2010

Reading habits

I intended to write this as a comment at Matt Zimmerman's post on ways he reads, but it got rather long.

Let's start with books. I also don't read as much as I used to, but I am trying to do more of it and less of other reading. I was struck by Kate Harding's post on reading:

... that’s a wonderful thing, especially for people who for various reasons can’t be physically present everywhere they might like to be, or who find it much easier to be social this way. But for me, the blessing and the curse of it is, I spend much more of my life than I used to thinking about what I’m going to say next. I’m composing a comment in response to what I just read instead of sitting with it; I’m having “chats” with friends where there can be no pleasant silences without one of us wondering if the other is still there; if I’m observing the world around me, half the time I’m thinking, “How do I make this a funny tweet?” When I was writing for Broadsheet, I read other feminist blogs desperately looking for fodder, rather than just taking it all in because it’s smart and interesting — which is exactly what got me interested in them and made me want to start my own in the first place.

All that thinking up something to say gets fucking exhausting.

I'm not going to insert the mandatory I love the feel and smell of paper thing about books here: I for one couldn't give a toss about it and, except for the heavy metal aspect (and what an aspect it is), bring on the e-reader revolution. I will happily remove bookshelves from my home and hang nice things on my walls. But the thing about books is that, allowing for 95% of everything being crap, they're planned, revised, edited, checked and they have a lot of space to say what they're saying. There are exceptions, but the general rule is that I get a lot more out of one good book read over a few days than I do out of 100 good blog posts over that time.

I'm trying to work out what to do about news. The trouble with news is that I do need an editor: I like to know what's going on in the world but I don't naturally find out about it in my normal activities. I find out things from social justice blogs, which are important to me, and I find out things from the Sydney Morning Herald's website, and there's a lot of things in between I am missing out on. I tried Google News, but I think the cramming of all that news onto one page makes me run and hide. I actually suspect the answer here is TV news bulletins and I'm thinking of adding, say, the ABC's and SBS's evening bulletins to my life on a regular basis. Then I know roughly what's going on and there's plenty of detailed print journalism to turn to when I want to follow something up.

I read a lot of email still, although for years I've been limiting (non-work-related) mailing lists to a 75% test: if I am not reading 75% of the posts to the list, I unsubscribe. Regular readers of technical mailing lists will immediately understand how few mailing lists I am subscribed to now.

I was until recently fairly firmly on the mailing list site of the mailing-list-versus-web-forum debate. But I've realised that this is really more about tools, that is, mail readers are more mature than forum interfaces *and* you can use your favourite mail reader for all lists. Each forum has its own, bad, UI, on it's own, regularly crashing, server.

But some of the features of forums, especially but not only the ability to move or delete or edit posts or entire threads after publication, are useful for high volume discussion. I'd love to see work on development of both standards and tools for more moderated threaded discussion that does not bind as tightly with a single UI. (I've used Usenet/NNTP. It's not what I'm looking for.) Really I'd love to do that work, even, but it's not a one-person job, buy-in is needed from software developers and users.

At the moment I follow a few web forums, mostly related to parenting things. I resist becoming too actively involved (ie, I'm not a regular poster at any and certainly don't want to moderate, I keep the relationship to a state where I can regularly take breaks of months from a given forum and no one notices).

I read a lot of blogs (really, a lot). They get subjected to the 75% test too, largely, at least if they update frequently. About a year ago I gave up trying to be basically completionist: if I went away and you blogged during that time, I didn't read it.

For a long time now I've been a fan of personal life-blogs over most other genres. I want to keep up with the educated, researched, niche blogs like Language Log or LWN (OK, the latter doesn't think of itself as a blog, but it's in my feed reader, so to me it is) but I find it difficult to be in the mindset to read it as I go through my reader and I can't think of a good model for setting them aside and going through a bunch of them, especially since I do web reading at my desk. I also want to keep up with hypertextual discussions on social justice issues, but that also easily becomes a second full-time job.

I used to like the big aggregators, but now someone needs to do a highlights column. I care deeply about my baby and my PhD, but I don't really care about the life milestones of, say, a given random Ubuntu developer. If someone else could pick the top three technical blog posts of the week and I could just read them, I'd prefer that.

I read less and less of microblogs or Facebook and I think it's going to stay that way. I feel a bit bad about it, since I like writing a microblog, I just don't like staying on top of my stream. I'm very over the 140 character limit too, it's too easy to get into needless arguments because my teeny sentence missed a nuance and then I have to clarify with someone, 140 characters at a time. I read direct replies to me, and every so often I surf over and read the most recent 50 or so items I'm subscribed to and that's about it.

There's things about Facebook I like (more generous character limit, reply threads, the 'Top News' sorting) but I do intend to leave. Just, people keep announcing the birth of their babies exclusive to Facebook. Knock it off!

I don't really find shared links as useful as Matt does, possibly I need a better tool for it. But I think the theme of most of this is that really, I am turning to edited content, sometimes by pros and sometimes by very smart people who spend a lot of time on the 'net. I am not cut out to be an editor in that sense, at least, not most of the time. Probably no one is.

Stuff I'm against, privacy edition

The ALP's proposed mandatory 'clean feed', see Save the Net, No Clean Feed, Open Internet and, especially if you are going to vote in Victoria in the 2010 Federal election, Filter Stephen Conroy.

Recording of email and correspondence history for Internet users in Australia:

Currently, companies that provide customers with a connection to the internet don't retain or log subscriber's private web browsing history unless they are given an interception warrant by law enforcement, usually approved by a judge. It is only then that companies can legally begin tapping a customer's internet connection.

In March 2006, the European Union formally adopted its data retention directive (PDF), a directive which the Australian Government said it wished to use as an example if it implemented such a regime.

...

The EU regime requires that the communications providers from certain EU member states retain necessary data as specified in the Directive for a period of between six and 24 months.

One internet service provider (ISP) source told ZDNet Australia that the Australian regime, if implemented, could go as far as recording each URL a customer visited and all emails.

But, just when you decide to vote for the Liberal-National Coalition (or MAYBE NOT)... Youth privacy at risk under the Mad Monk:

At the heart of the near-universal support for adolescent health privacy is an extensive body of data. The research shows that the greatest barrier to young people seeking medical help is the fear their parents may find out.

...

In Australia, "mature minors" are authorised to make decisions about their medical treatment. A mature minor is a tween or teen with sufficient understanding and intelligence to understand the nature and consequences of the medical intervention proposed, and to give informed consent to it.

While all those under 18 must be accessed on such criteria, it is generally assumed that those over 17 are mature minors, that those 14 to 16 are reasonably likely to be, and that those under 14 may not have capacity to consent, particularly in relation to more serious treatments. The requirement for confidentiality is a corollary of the mature minor framework.

Never one to let evidence muddy the waters of ideology, the now Opposition leader Tony Abbott was part of a government that in 2003 lifted the age at which information about a child's healthcare visits could be accessed by their parents from 12 to 14. As Health Minister he vigorously argued for this threshold to be lifted again, from 14 to 16. Had he succeeded, an entire group of Australians would have been denied independent and confidential medical care, despite most qualifying for it.

Apple opens up Podcasts, iTunes U in Malaysia

I just noticed this: Apple now not only sells you apps in the AppStore in Malaysia, but you can also download podcasts and iTunes U content, using your Malaysian iTunes Store account. This is a big change, considering the directory was not available, this time last year. I believe this change is mostly international, across all iTunes stores.

Incidentally, you can now also buy iPad apps in the AppStore (yet, you cannot get an iPad from Apple or any of their authorised resellers; people are generally price gouging you now with their parallel imports). So what do you get in the Malaysian AppStore today?

  1. Apps: iPhone, iPad
  2. Podcasts
  3. iTunes U podcasts

So what’s missing? Music, Movies, TV Shows, and Audiobooks. I guess that’s stuff we should look forward to, in the future.

This could be big for podcasters in Malaysia/Singapore. Audio, video content can go into the directories. We’re getting faster Internet. And there are a lot of iOS-devices out there, which now may have yet another reason to use iTunes (and maybe purchase apps at the same time). Kudos Apple, we’ve waited long enough.

Related posts:

  1. Podcast directory not available in iTunes Malaysia
  2. A plea to Apple
  3. The Apple Store Malaysia Phone Experience



Resource Rent Maths, take 2

My previous post apparently didn’t do the economics for the resource rent analysis quite right — it seems that the idea is a cleverer company would be able to use the resource rent tax to find cheaper sources of funding, which changes things…

The idea then would be that you start your mining project seeking 60% in risky funding (they get whatever profits you make and the totality of the loss), and 40% in risk-free funding (they get the same return as they would if they invested in government bonds, whether the project succeeds or fails). That’s as opposed to the current approach of seeking 100% in risky funding.

So say you’ve raised $5B. You spend your $5B doing surveys, setting up your mine, etc. Failure here means you declare bankruptcy and the government gives you enough money to pay back the $2B of risk-free investment, plus interest, presuming the Greens don’t have their way. On the other hand, your mine might be a success, and you might, eg, start getting $1.5B in revenue, against $500M in expenses. At this point you first have to pay your “super profit” tax, which is, apparently 40% of:

  • gross receipts: $1.5B
  • less depreciation: assuming 20 year expected life, 5% of 5B = $250M
  • less running expenses: $500M
  • less “normal return” on debt/equity: 6% of $5B = $300M
  • totalling: $450M

So $180M on resource rents. You then pay corporate income tax of 30% (eventually 28%) of:

  • gross receipts: $1.5B
  • less depreciation: assuming 20 year expected life, 5% of 5B = $250M
  • less running expenses: $500M
  • less resource rent: $180M
  • totalling: $570M

So $171M ($159.6M at 28% in 2014 or so).

You then pay the risk-free return to your risk-free investors, which is 6% of $2B or $120M. (Actually, this might be tax deductible too)

So after paying expenses ($500M), resource rents ($180M), income tax ($171M) and the risk-free dividend ($120M), your $1.5B of earnings is down to $529M. Issuing all that to your risky investors, gives an annual return of 17.63%, fully-franked.

That compares to doing things the current way as follows: you raise $5B of risky investment; your mine succeeds and makes $1.5B in revenue, against $500M in expenses. You just pay company tax at 30% after expenses and depreciation, so that’s 30% of $750M, or $225M. That leaves you $775M to pay in dividends, which is an annual return of 15.5%, fully-franked.

That, obviously, is an entirely convincing investment. It relies on the government refunding the $2B of “risk-free” investment in the event that the mine falls apart, though — which, as I understand it, is the part of the plan the Greens oppose. But otherwise, the above’s fairly plausible.

The difference in those sums — profit rising from 15.5% to 17.63% is due to the level of depreciation in the above sums. If those formulas for calculating the rent and company taxes are correct, then your return on investment increases by two-thirds of your annual depreciation compared to the initial investment and decreases by a fifth of the risk-free rate. In the above case, annual depreciation was 5% of the entirety of the initial investment, and the risk-free rate was 6%, which implies an improvement of 2/3*5%-6%/5 which is the 2.13% we saw.

In reality, you’d probably need to offer a higher return to your “risk-free” investors — because if you didn’t, they’d probably just by bonds directly from the government in the first place. And if I’m not mistaken you still need to repay the principle for your risk-free investors over the life of your mind. So hopefully that simply evens out in the end.

There’s not a lot of difference in that scenario to having the government borrow enough to maintain 40% ownership in every mining operation in Australia. They’ll then receive 40% of the after-tax profits, and have to pay interest on their borrowings at the long term bond rate, which would mean (in the above example) getting $225M in company tax, then $310M in franked dividends, then paying out $120M in interest costs for a total of $415M extra per-annum. That’s more than the total of $351M in receipts in the above example, I think due to the depreciation deduction in the resource rent tax calculation.

Mechanically, there’s a few differences: the company has to gain two sorts of investment (risky shares and risk-free bonds, for instance), if it fails it has to go to a lot more trouble to pay back the risk-free investors (getting the tax office to issue a refund in cash), and the government gets to keep it mostly off its books (doesn’t have to raise funds directly, investment losses turn into tax refunds).

In any event, that should make it easier for mining companies to raise funds — they only need to raise 60% of the amount at the risky level, for the same return they previously offered.

I don’t see anything stopping you from being tricky and doing a two stage capital raising: raising $3B of risky funds to do exploration; and if that fails repaying your investors 40% ($1.2B) of their capital — then doing the risk-free fund raising to get enough cash to start production. The initial fund raising then has a chance at a 17% ongoing return, or a 60% loss — compared to currently having a chance at a 15% return or a 100% loss. Again, that should make it easier to raise funds for new projects.

On the other hand, I also still don’t see anything stopping you from transferring your profits. Say you’re a public investment company. You’ve got plenty of money from offering superannuation products or what not, and you want to get into mining because you hear it gives a high return for your investors. So you allocate a few billion to start a mining company, which does some prospecting and opens a mine. That works out, and it starts making super profits. You decide you want to reduce your tax, and get more dividends. So instead of having one privately held subsidiary mining company, whose balance sheet looks like:

  • Revenue: $1500M
  • Expenses: $500M
  • Resource rent tax: $180M
  • Company tax: $171M
  • Dividends: $649M

you decide to invest in a transport company as well. Hopefully one that’s already making a decent profit, but paying a bit more than market value works too. You then have them make an agreement that the mine will exclusively use your transport company for the next 10 or 20 years, for whatever excuse satisfies appropriate laws. Then have the transport company seriously jack up the price. Your balance sheets should then look like:

Mine Mine change Transport change Total change
Revenue $1500M - +$700M +$700M
Expenses $1200M +$700M - +$700M
Resource rent tax $0M -$180M n/a -$180M
Company tax $15M -$156M +$210M +$54M
Dividends $285M -$364M +$490M +$126M

And voila, your resource rent tax has been reallocated to your dividends (except for the 30% that goes to company tax, of course). It doesn’t have to be a transport company, either — any private company that you can buy outright, that isn’t hit by the resource tax, and that you can find some excuse to make your exclusive supplier of a necessary product/service will do fine. And even better, as far as I can see, even when you get rid of all the resource rent proceeds the government was hoping for from your mine, they’ve still covered 40% of your initial risk…

June 10, 2010

The end of 2010

Although there's always a possibility, it seems that regular childcare will not be available for my son Vincent until early 2011 (yes, I've had his name down for ages, although not since birth, since I was pretty sick afterwards).

What this means: my supervisor and I have agreed that I can't resume my PhD studies if I am doing fulltime baby care. My experience so far meshes with this: I can poke at my code and read the literature, I can't really pound on it in forty-five minute chunks a few times a day, and it would be a waste of both of our time if I was to be formally re-enrolled now.

So it looks like I have six months or so to kill, and I'm thinking about what to do with that time. I may or may not have some paid work lined up for some hours of the week. I'm hoping not to spend the rest aimlessly noodling around on the 'net, well, not all of the rest. Options include more paid work (if you for some reason have paid work that can be done by a — skilled! —  programmer/writer/researcher in forty-five minute chunks, get in touch), hacking, writing. Looking forward to coming up with some projects!

June 09, 2010

June 08, 2010

What do American digital content sellers have against the rest of the world?

I live in the “rest of the world”. I do not reside in the USA. Why is it I cannot get content that I’m willing to pay for?

As a preamble to this, you might want to read an article in The Economist: Cupertino’s cold warriors: What has Apple got against eastern Europe?.

The iTunes store is a major pain point. I can buy Apple hardware in Malaysia, be it iPhone’s, iPod’s, Macs, and more. But when I visit the iTunes store, I can only purchase apps for the iPhone (the iPad store opened recently). Why can’t I purchase music/movies, legally? So you’ll say why not visit Amazon’s MP3 store. Bam, I can’t make a purchase there either.

The iPad’s come with iBooks. Its a fabulous book reading application. I cannot purchase books from the iBook store, but I can purchase ePub formatted books, say, from O’Reilly’s Safari bookstore. What about the Kindle – its available in quite many locations, but only where AT&T is present – so some rather “odd” countries show up in the availability lists. Why can’t I purchase books, legally?

I’ve been a long-time subscriber to audible. Audiobooks mean I don’t have to take a trip out to the bookstore. Chances are, I might get audiobooks cheaper than the dead-tree versions available in Malaysia. Today I got an email from Audible, telling me there’s a book I might be interested in. True enough, I was interested in The Facebook Effect: The Inside Story of the Company That Is Connecting the World. Who couldn’t be, after listening to the most amazing The Accidental Billionaires: The Founding of Facebook: A Tale of Sex, Money, Genius and Betrayal?

Of course, I get greeted with the magical message:



Download The Facebook Effect: The Inside Story of the Company That Is Connecting the World | David Kirkpatrick | The Facebook Effect: The Inside Story of the Company That Is Connecting the World Audio Book unabridged | Audible Audiobooks | Audible.com

That basically reads:

We’re sorry. Due to publishing rights restrictions, we are not authorized to sell this item in the country where you live.

Sorry David Kirkpatrick, I tried to give you some money, but apparently, you’ve decided to shaft the rest of the world.

Sadly, this situation is not much different in a first world country like Australia. Sure you get the iTunes Music Store (you are paying a premium, in comparison to exchange rates), and eventually the iPad becomes available for sale, but its always behind the US. So its not an isolated third-world 1Malaysia problem. Of course not – Singaporeans still suffer from the lack of the iTunes Music Store, and they’ve been a first world, industrialised nation, since 1996.

When will American content sellers realise that the Internet lacks boundaries?

When will they realise that limiting based on geolocation (Android marketplace) or credit cards (iTunes store) is so 20th century? Incidentally, there’s a petition out there for Google: Enable paid apps for all countries on the Android Market!. Its okay Google, you’re not the only retarded one here – the BlackBerry AppWorld is no better; PayPal is available in Malaysia, but you can’t buy apps either.

You push globalisation to the core, yet you refuse to embrace it.

Give us the content that we want to pay for. It doesn’t matter where we are located. Really.

Related posts:

  1. Apple opens up Podcasts, iTunes U in Malaysia
  2. Podcast directory not available in iTunes Malaysia
  3. Digital Media Consumers



Safari 5: Quick Impressions

I rebooted my MacBook Pro today, because I wanted to try out Safari 5. The short version of all this is: I came off quite impressed. DNS prefetching, the Web Inspector, and Safari Reader (strips ads/formatting) are killer features.

First up, its fast. DNS prefetching makes a difference if you browse the web like I do – I have a folder of bookmarks called “Daily”, and in there it has sites like Techcrunch, Techmeme, Hacker News, Slashdot, and a few news sites. You can imagine I’m usually loading up links elsewhere, and if an article is too lengthy, I usually just use the Read Later widget and send it to Instapaper.

There is now a free Safari Developer Program (as opposed to the US$99 paid iPhone or Mac developer programs). It allows you to sign your extension (beyond that, I don’t see the value – guess why its free?). There is supposed to be an Extension Gallery, but I guess this will be around in due time.

As someone that frequently uses the Web Developer extension in Firefox, I have to say I’m quite impressed with the Web Inspector that comes built in with Safari 5. Its more visual. Timelines, Resources, looking at HTML5 storage containers (local databases), and more is quite useful.

I mentioned that I like using Instapaper above. What is its killer feature? Scraping text from a page, images if need be, and giving you a clean reading interface. Safari comes with Safari Reader, which is very Instapaper like. The blog you read has got ads, but a fairly simplistic (aka boring) design. With Reader? Articles look a whole lot better – I can tell I’ll be using this feature a lot. Its not just about the fact that it strips ads – its a much cleaner, interesting interface. I wonder if it’ll catch on?

before reader



Before Safari Reader
Safari Reader



Using Safari Reader

Memory usage seems to be better than Firefox 3.6.3, with multiple tabs open. I know the Firefox team are hard at work to make magic happen here, but Safari seems to have the edge at the moment. I haven’t used Chrome enough to know if it compares.

What do I miss? My Firefox add-ons. Here’s to the Safari Extension Gallery growing to become like addons.mozilla.org. Give Safari 5 a twirl, you probably won’t be disappointed. And if you’re into reading about every whizz-bang feature, go check out the feature list.

Related posts:

  1. O’Reilly to offer DRM-free ebooks…
  2. The Nokia E72: Quick impressions
  3. Nokia N97 – Quick Impressions



Where did everything go?

Time for a clear out of all the old content, and start again. The Etch-A-Sketch end of the blog so to speak.

After a long period of inactivity, I’ve decided to resurrect this blog mostly to post in progress work and things cruelly rejected from Tech Liberty.

Unlike the last incarnation, there’s going to be less personal blogging and posts. There won’t be status updates on my non-cat, or updates on any projects I’m working on. Those will all remain where they are today, and this will remain purely on tech issues for tech audiences.

That’s the plan anyway.

We’ll see how that pans out!

MariaDB 5.1.47 VMs for OpenSolaris, Ubuntu

Mark’s done an excellent job again, getting VM’s of MariaDB 5.1.47 (release notes, changelog) out there for download. He’s also improved the bandwidth available at the box hosting these images. Get it for Ubuntu 10.04 or OpenSolaris 0906.

Related posts:

  1. Ubuntu 10.04 LTS released, MariaDB 5.1.44/5.2-BETA VM’s available
  2. VirtualBox images for MariaDB
  3. rpm -q –changelog in Debian | on IRC (or adventures in the land of #ubuntu)



Resource Taxes

So it seems that taxing oil and gas is the only significant result that’s going to come out of the Henry Review, and that probably means I should work out an actual opinion on it.

As I understand it, the Resource Rent Tax as proposed by the review is meant to be a different way of charging for non-renewable resources extracted from Australia — coal, oil, gas, uranium, whatever. The aim being to increase the government’s share of the value, while maintaining profit incentives to actually find, extract and sell it. And the way the Henry review recommends achieving that is for the government to make themselves 40% partners in the investment, with their initial capital contribution being made by tax concessions, and then receiving dividend payments worth 40% of profits via the tax system. See Nicholas Gruen’s take for more on this line of thinking.

Of course, the government’s cheap — it’s not going to actually put money up front to become a 40% partner like anyone else would; it’s obviously hoping to get all the benefits without any of the risks. But that’s not economically sound — it would mess up the incentives for investment, effectively making investing in Australia 66% more expensive [0].

(And, of course, the government’s not going to be an ordinary investor either — just buying shares in mining companies would be way too straightforward…)

Instead the government’s contribution is in the form of payment of the state taxes and tax deductions, and because that’s not as valuable as actual money, their payback only kicks in when the endeavour starts making lots of money (where lots is defined as more than you’d make just loaning to the government).

On that basis the maths would go like this: in the first year you spend $3B to setup a mine, but don’t make any revenue yet. The government gives you $2B in tax credits you can use later (or possibly against other projects you’re working on). Your mine starts production, earning its first $1B ($1.5B in revenue, 500M in expenses). You then owe $280M in company tax, and another $288M in resource tax. You deduct those from your tax credits. You can presumably then pay out $432M as fully franked dividends to your investors; I’m not sure about the remaining $568M (if it’s not, $568M in unfranked dividends is equivalent to about $400M in franked dividends, the difference going to the government via the investors’ income tax). Anyway, that goes on for three and a half years or so until your tax credits have been all used up — you’re either paying out $1B in dividends a year to your investors (a 33% return) and no tax, or $832M in dividends (28% return) and $168M in tax (16.8% of earnings). After the three and a half years are up, you switch to $432M in dividends (14.4% return) and $568M in tax (56.8% of earnings). Presuming the initial investment is entirely unrecoverable (the trucks you bought wear out over the life of the mine, it’s cheaper to demolish the buildings and rebuild than try moving them to your next mine, etc); that would mean over the first three and a half years investors recover either 110% of their investment or 97% of their investment, and then earn a 7% return.

With just company tax, the same scenario would have resulted in $700M in fully franked dividends each year (23.3% return), so investors would get 93.3% of their money back after four years, and then earn 23.3%.

Except, of course, things aren’t actually that simple either, because, AIUI, some of the ongoing costs will get counted as well, so the $500M in annual expenses might mean up to an additional $333M in tax credits each year, which would not be very sound — but if some of those expenses are for “expanding the mine” they possibly should be counted as additional “investment”. Additionally, interest is earned on unspent tax credits at the government bond rate, but that would be pretty insignificant in the above example. And it’s possible the government doesn’t plan on providing tax credits worth 40% overall, but only puts in 40% of the previous investment (which would be worth 28.5%).

There’s also the “super profits” aspect — I can’t see how that’s intended to be calculated. It could be simply via the interest rate on unspent tax credits: if you’ve got $2B in tax credits, and earn the 6% bond rate on that for an additional $120M in tax credits, then you could just spend the “interest” to reduce your $288M annual resource rent liability to an $168M annual liability in perpetuity. The $120M saving then is the resource rent tax (40%) on the non-super part of the annual profits (6% of 5B). Of course, if you work things that way, you don’t have the few years of no/low tax. I wouldn’t have thought the tax office would let you work things that way, either, to be honest; but economically it’s probably meant to be treated as an equivalent outcome. Anyway, that totals to a 55.2% tax on annual earnings in the above example.

Beyond that there’s the effect on risky projects. If a mine doesn’t turn out to make money, at the moment you lose lots of money. With the government being a 40% partner, you still lose that money, but you get 66.6% of it back in tax credits. And if you didn’t “lose” the money, so much as paid your nephew (or subsidiary company, whatever) to do some prospecting for you, well, hey, that’s pretty neat, right? The risk there is pretty simple: the government wants to be counted as an investor in all the profitable mining companies, without actually exercising any judgement on what’s likely to be a good company and what’s not. And if you’ve got an investor with no judgement, people are going to take advantage of that.

Of course, being the government you can write the laws to your own advantage — so you can claim all the profits when things go great, and disclaim any losses when things go bad. That seems to be the Greens plan:

Resources Minister Martin Ferguson said the government was committed to keep a controversial plan to reimburse miners for 40 per cent of their losses.

But with opinion polls predicting the Australian Greens gaining the balance of power in the Senate after the next election, the government may have to scrap this.

Greens leader Bob Brown has said that while he supported the resource profits tax in principle, he did not want miners being rebated for their losses.

That goes directly against the positioning of the government as a “co-investor”, though, compare to Ken Henry’s reported comments or Terry McCrann’s expansion from senate testimony.

And really, nothing in the calculations above actually had anything to do with resources — just investment; you can invest in a restaurant too, and except for scaling down the numbers, the same calculations and arguments would apply. If it was really about the resources, it would make more sense for the resources themselves to be government’s initial contribution [1] to the investment. But that just goes straight back to charging royalties on whatever’s dug up, which is the system we’ve already got.

As far as sovereign risk goes, that seems a pretty simple calculation. After, what, 30 years of Hawke, Keating, Howard and Howard 2.0 Rudd, people might’ve expected pretty simple and sound economic policies — floating the dollar, privatising the banks, independent reserve bank, compulsory superannuation, the GST, free trade agreements, low inflation, gradual lowering of income tax rates, pretty good handling of the Asian currency crisis in the ’90s and the recent financial crisis. What are the odds the government will suddenly start doubling the amount it taxes various companies? Before the “super tax”, you might’ve said pretty low. Now, not so much. Will it happen again? Who knows — but I bet more people would guess it would now, than would have previously. So yeah, Australia’s sovereign risk seems way higher.

Ultimately, this is looking more and more to me like one of those ideas that sounds great at the height of a boom (“look, those people are making lots of money, gosh I wish we were those people!”), but that turns out too clever by half, and all the little complexities involved in turning theory into practice end up biting you in the butt.

[0] If (eg) you currently have to invest $10B in setup costs for every $2B profit per year your mine makes; then with the government taking 40% of that, $10B would only get you $1.2B in profit. To get $2B in after-resource-tax profit, you’d need $3.3B in before-resource-tax profit, which would mean a $16.6B in setup costs; a 66% increase.

[1] Though that runs into the “which government?” problem — the resources are owned by the states, and it’s the federal government that wants to collect more money… And that in spite of the state governments having the bigger budget problems at the moment…

June 06, 2010

Why I’m excited about Palm’s webOS

I have been a Palm fan for a very long time. My first PDA was a PalmPilot Professional, which eventually led to the m100 (AA batteries!), my first Palm OS phone, Pia’s hand-me-down Tungsten W, the Treo 650 and finally, my last (real) phone: the Treo 680. I’m even vaguely sad that I missed the final outing of Palm OS on the Centro.

I was disappointed when Palm split into two companies, but perhaps it was ultimately a good thing… PalmSource became victim to an almighty corporate Sarlacc, to be digested slowly over a thousand years, while Palm became nimble and determined to win out of necessity.

The ill-fated Foleo was a public relations disaster, but in retrospect, a concept way ahead of its time — consider the explosion of the netbook market soon after, and where the iPad is today (albeit generations of mobile technology beyond the capabilities of the Foleo).

Which brings us to webOS, released a year ago today on the Palm Pre. Having waited impatiently all that time (often running to the emulator to get my fix), I finally acquired one last week.

It is a truly delicious user experience. I won’t go into too much detail on this front — as with most things, you must use it — suffice to say that it carries the soul of Palm OS in a 21st century vessel.

(Okay, one thing on this: When it comes to the basic functions of a smart phone — calls, and contacts — I’m convinced this thing has both the iPhone and HTC’s Sense UI for the Desire beat. Synergy is seamless and awesome.)

Despite it’s youth, webOS is an incredibly promising and fast-moving platform… and in stark contrast to other “mobile Linux” competitors, it’s not just a bunch of goofy shit piled on top of a heavily molested Linux kernel: What runs on your phone is an utterly recognisable Open Source stack and an utterly recognisable web stack. Sure, there’s a layer of proprietary Java gumpf shoved between the good bits, but even that is getting thinner.

To some extent, webOS is the GNOME Mobile platform with a user interface and services layer built for the web generation. It’s D-Bus, GStreamer and PulseAudio under WebKit. Mojo applications, written in JavaScript, talk to services via JSON APIs, and native apps integrate into the web-based user interface via — get this! — NPAPI plugins.

Crucially, webOS will grow and improve along with the web. Everything you’re seeing in the web world — faster JavaScript, hardware accelerated CSS animation, massive growth of the JavaScript ecosystem (consider all the frameworks, CommonJS modules, nodejs event-based server, etc.) and all the incredible new APIs popping up — will find a place in webOS. Check out the Palm Developer Day keynote and podcast from Dion Almaer and Ben Galbraith for more on the near future of webOS.

On the awesomeness of Dion and Ben at Palm? I’ll just quote James Governor:

Palm tried to use Apple’s trick of secrecy first rather than investing heavily in developer good will and playing the open card. It didn’t work. Palm realised its error last year and did something incredibly smart – it hired Dion Almaer and Ben Galbraith to develop a new, web-savvy, strategy around its platform.

Using open technology as the bricks and mortar of a platform is not particularly amazing of course. Everyone’s doing it. But Palm are making friends and influencing developers by having an impressively open attitude to devices, too. Your store-bought phone — with a bit of Konami code action and the freely downloadable webOS SDK — is already “rooted” for you. Just log in. ps afx? cat /proc/cpuinfo? top? Your “first command” habit is most likely catered for. :-)

That openness has encouraged an incredible amount of community activity. The most Open Source savvy group dedicated to the platform is WebOS Internals. Initially, they published all sorts of juicy information about the innards of webOS and the Pre… but have now rallied around distribution of “home brew” Open Source patches and apps, using their own package management interface, Preware. They’ve even published an updated kernel which supports overclocking, temperature sensors and more advanced power management than the original! It’s wonderful stuff.

Then HP bought Palm.

Despite some messaging hiccoughs (now resolved), this is an incredibly exciting move for Palm and webOS fans. I’m hoping it gives Palm the reach, resources and relationships to go global, accelerate improvement of the platform, and ship some terrific new hardware to make their software shine… that said, it better not be an almighty corporate Sarlacc!

Damn it feels good to be a Palmster.

June 04, 2010

Building a Status Screen, part 3: Display

This is the final installment of a 3-part series on building a visual status screen, or dashboard, for your company. If you missed the previous parts you can find it at:



 Building a Status Screen, part 1: Hardware



 Building a Status Screen, part 2: Collecting Data



In the first installment I showed how to physically mount LCDs on a wall, and in the second I showed how I go about collecting data from various systems (both internal and external) in a central location.



The final step is taking that data and displaying it on the status screens, and then updating the display whenever the data changes. There are a huge number of ways this could be done depending on what sort of system you are accustomed to working with, but since I normally do web application development I decided to take the simple way out and make the status screens using dynamic web pages. Yes, when I'm swinging this PHP hammer every problem looks like it could be solved with a web page!



I'll start by showing you the simplest, most brute-force approach that should be easy to get going. It's a bit crude but it'll get the job done, and later I'll mention some enhancements that can make it more elegant.



In the last installment we ended up with all the relevant data being stored in simple text files on a central machine. With all the data in one place it's easy to create a dynamic web page that is served by that machine and displays the data neatly formatted to suit your LCDs, and by displaying it in a browser like Google Chrome that can run in full-screen mode it can look really pretty.



You'll need some kind of scripting system running on your server to generate dynamic web pages, and there are many alternatives so I won't go into detail here. I'm using PHP but you could do the same thing with Perl, Python, ASP.net, or many others. It's really up to you.



Let's start with just about the simplest case possible, which is a page that loads a single value (the current temperature, in this case) from a text file on disk, displays it, and reloads itself periodically.





<html><head><title></title>

<meta http-equiv="refresh" content="10" />

</head>

<?php

$temperature = file_get_contents( "/home/statusscreen/data/data-weather-temp" );

echo "The temperature is " . $temperature . "&deg;";

?>

</body></html>


That's it! Not so hard, right? If you put that code into a file called something like "screen.php" inside the web tree on your server and load it up in a browser, you'll see a mostly blank page with the current temperature displayed on it - assuming your data collector script is updating the "data-weather-temp" file as described in Part 2.



The "meta" tag at the top is a special tag that tells the browser to reload the page periodically, and in this case it'll reload every 10 seconds. If you leave the page open in your browser and alter the value in the "data-weather-temp" file you'll see the change reflected in the web page after a few seconds.



Of course this is a very minimal example, and if you've done any programming it'll be obvious that there are many ways to improve the code above: for example, before attempting to retrieve the contents of the file there should be a check to make sure the file actually exists. Then once the value has been extracted it should probably be cast to a numeric value just in case something nasty ends up in the file. The page itself will also be totally unstyled, simply displaying default black text in the top left corner of the page on a default white background. You'll also notice that the entire page is reloading every single time, when really it's only the temperature value that should need to be fetched from the server.



It's a start though, and with some further development it can end up as quite a decent system. To test it you can open the Google Chrome browser on the computer connected to your wall-mounted status screen, press F11 to go to full-screen mode, and leave it open. The page will continue to reload every 10 seconds so by editing the script on the server you'll be able to see the effect of your changes almost immediately. You can also preview the page directly on your laptop or desktop, of course, but having it open on the machine and screen combination that you'll ultimately leave running can be handy because it may have a different resolution to your own computer. Normally when developing web pages it's a good principle to structure it in such a way that it looks good in different browsers at different resolutions, but in this case you're creating a web page that will only ever be loaded up by one computer running that specific browser at a known resolution, so you can tune it to look exactly the way you want without caring about how well it works on other computers.



The next step is to expand the script to include other bits of data that you want to display on the same screen. In my case I wanted a simple 4x3 grid layout for one of the screens so it made sense to put the different parameters into a table. The result could be something like this:





<?php

$datapath = "/home/statusscreen/data";

$temperature = file_get_contents( $datapath . "/data-weather-temp" );

$brillianz = file_get_contents( $datapath . "/data-ops-br" );

$sb4 = file_get_contents( $datapath . "/data-ops-sb4" );

$projnum = file_get_contents( $datapath . "/data-projects-projnum" );

//(etc)

?>

<html><head><title></title>

<meta http-equiv="refresh" content="10" />

</head>

<table>

<tr>

<td><?=$temperature?></td>

<td><?=$brillianz?></td>

<td><?=$sb4?></td>

<td><?=$projnum?></td>

</tr>

</table>

</body></html>


That's functional, but still ugly. We're still dealing with default text on a white page in an unstyled table.



But now you can apply some styling to make it look pretty. By putting class names and IDs on the various elements and loaded an external CSS file you can now make your plain page look as classy as you like. In my version I applied styles to just about everything using CSS: the font size, font colour, text shadow, page background, borders and margins, and cell backgrounds you see in photos of the screens are all just styled with CSS.



At this point you have a system that is not only perfectly functional, but is actually starting to look pretty sweet as well. You could leave the system running exactly like this and be happy.



If you want to take things a little further though there are many things you can do. For example, rather than load the entire page every time you could have the page only load once and then use AJAX to periodically fetch and update data for the individual cells. In my version I have a bunch of Javascript that fetches values and then applies a "fade" effect to update from one value to the next, making the visual transition much smoother than a jarring reload of the whole page.



Because I have three screens on the wall I actually have three pages defined. One uses the simple grid layout described here, while one loads a web-based energy monitoring system created at IVT to display real-time power consumption in different parts of the building, and another displays bar graphs using data collected from various internal systems such as our bug tracker and package autobuilders. What you want to display is totally up to you.



The energy monitoring system is actually based on slightly different technology to what I've described here. It's still a web page, but it uses Adobe Flex to display real-time data that updates dynamically without having to reload the whole page.



I've deliberately become more vague towards the end because this is where you really have to decide what data you want to display and how you want it to appear. I built a system that suits my requirements, but it may not suit yours.



If you've built a status screen for your company I'd love to hear about it, so please drop me an email!



June 03, 2010

Ubuntu’s Bleeding Edge, an act of niche journalism

I’ve been struggling to write anything of consequence at all recently… but apparently the best way to get back on the horse is to write what you know, so I have embarked on what might be described as an act of niche journalism.

Ubuntu’s Bleeding Edge is a new blog about the action, intrigue and occasional romance of the Ubuntu development branch, for everyone who thinks Release Day parties are only meant to celebrate the start of a new release cycle. It’s an easy way to keep up-to-date about the goings-on of Maverick Meerkat… particularly if the idea of running an ever-changing, alpha-quality operating system scares the bejesus out of you.

The story so far…

It’s a nice way to tackle a problem in bite-sized chunks, have a ready excuse to write regularly, and recall how much I used to enjoy those six month roller-coaster release cycles. It sure helps to have an enjoyable subject that I know pretty well, too!

Check it out, and subscribe to the feed if you’re keen to keep up with the ups and downs of Maverick.

Update: My latest devel branch update covers the removal of aptitude from the default desktop install and the switch to building for i686, both of which have become minor controversies.

June 02, 2010

larch: migrating mail between IMAP accounts

I recently had to move several gigabytes of email (not my own, work-related) into Google Apps (Gmail). As best I can tell, the way most people do this is that they grit their teeth and they open up a graphical email client and drag folders one-by-one. It's a one-off job for most people.

There were a couple of reasons I didn't want to do that. One is that I was on my parents' DSL connection at the time and pushing gigabytes of data through someone's DSL is a violation of good guest principles, at least in Australia. The other is that we have over 500 folders in the account I'm talking about: that's a lot of mouse pain.

Anyway, here's your answer, if you are in the same position as me. After substantial searching, at least for this kind of tool, I came across larch, which is a Ruby command-line tool for IMAP-to-IMAP moves, most tested on Gmail and essentially designed for the "move my mail archives into Gmail" use-case. It's much more mature than most of the one-off scripts people have thrown up on the 'net. It certainly seemed robust over this volume of mail, although I did have to run it a couple of times to get past a few errors (it does not re-copy already copied mail, so re-runs are fast). It deserves more search juice.

If you wanted to keep two accounts permanently in sync, offlineimap would be the tool of choice, although the manual still seems to regard IMAP-to-IMAP syncing as not as robustly tested as its core mode of operation, which is IMAP-to-Maildir.

June 01, 2010

Using your iphone as a modem under Debian

Thanks to the awesome work of Diego Giagio (for writing it) and Paul McEnery (for packaging it for Debian), using your iPhone as a modem under Debian is about 60 seconds work:

  1. Install ipheth-dkms (the kernel module side of things) and ipheth-utils (the userspace pairing daemon).
  2. watch the postinst build the kernel driver for your current kernel
  3. plug your phone in
  4. dmesg|grep iPhone should show something like:

    [867025.370421] ipheth 3-2:4.2: Apple iPhone USB Ethernet device attached

    and you’ll find you have a new Ethernet interface

  5. enable tethering on the phone (Settings -> General -> Network -> Internet Tethering)
  6. now the ethernet interface is running a DHCP server - select it with nm-applet or ifup it or whatever you normally do

May 31, 2010

Facebook Zucks

Facebook Sucks (Blue)

Facebook sucks? Nah, man: Facebook Zucks!

Facebook Zucks Facebook Zucks (Blue) Facebook Zucks (Twotone)

Download the whole CC-BY collection — includes SVGs! (ZIP, 89KB)

May 30, 2010

May 29, 2010

OLPC support from the Prime Minister of Australia

We at OLPC Australia celebrated our first birthday with a massive bang — a black-tie gala event held at the Museum of Contemporary Art on Sydney Harbour. It was a wonderful night of celebrating Australian Indigenous art, music, culture and food. Corporate sponsors generously donated to the cause.

Of special note was our keynote speaker. Prime Minister Kevin Rudd voiced his support for our mission, extending deductible gift recipient status to all donations made to us. Also in attendance was the Assistant Treasurer and other representatives of state and federal government.

Myself and other members of the OLPC Australia team were fortunate enough to meet with Kevin before the official proceedings commenced. He took the time to converse individually with each of us. I related my experiences in Dhalinybuy, where every child has their own computer on the Internet. This ratio of 1:1 access is almost unheard of even in city schools. I was pleased to see one of our anecdotes make it into his address, not very long after our conversation.

It’s an indescribable feeling knowing that you’re on the radar of the highest political office in the land. We are a small team and have a long way to go, but I firmly believe that we are on track to empower remote communities across Australia.



©2010 Sridhar Dhanapalan.

This work is licensed under a Creative Commons Attribution-Share Alike 2.5 Australia Licence.

Creative Commons BY-SA Licence

.

OLPC Australia in East Arnhem Land

I am writing this from Dhalinybuy School in remote Australia. What’s even more impressive is that I am typing this on a production-model OLPC XO-1.5!

For those who don’t know yet, in March I started full-time as the Technical Co-ordinator at One Laptop per Child Australia. This basically means that I manage the technology surrounding the XO laptops, XS server and so on.

We are in East Arnhem Land, Northern Territory, this week for OLPC deployments, as well as training in Yirrkala School and the Yirrkala Homelands Learning Centres (HLCs).

There are eight HLCs in all, spread over a wide area. The closest one is close to two hour’s drive away from Yirrkala, almost entirely on dirt road. Yirrkala itself is quite remote - about 13 hours drive (again, almost entirely on dirt) from Darwin. It’s generally easier to fly to these locations (which takes at least four hops if you’re coming from Sydney), especially right now as a tropical cyclone (which barely missed us a month ago when we were out this way) destroyed many of the roads.

In collaboration with the University of Western Sydney (UWS), and with some assistance from the Northern Territory Department of Education and Training (NTDET), we have formed teams and spread out over the eight HLCs to work with children, teachers, schools and communities.

I’ll have to go into my work at Yirrkala School at a later time, but here at the HLCs we have managed some impressive feats, if I do say so myself! For instance:

  • this is to the best of my knowledge the world’s first deployment of the new XO-1.5 devices, and we’re doing it across all eight HLCs at once
  • children can write in their own language, as we installed Yolngu Matha fonts
  • we have taught teachers and students to create their own e-books using Scratch, using pictures they take with the camera and content we loaded onto the XOs beforehand

In addition, I worked with Ian Cunningham from NTDET to produce an inexpensive and simple means to deploy wireless access points to these remote communities. These are Linksys WRT-54GL devices flashed with DD-WRT. We configured each such that they will just work when plugged in. The HLCs that have satellite Internet can have their access points managed from anywhere on the NT Schools network.

I left our setup to the UWS students (none of whom are technical) on my team, and they were able to successfully set up the access point and create a workable area for the XOs to be charged.

Most of the HLCs have their electricity supplied entirely by local generators, which are normally rationed to run at night. Dhalinybuy school has its own smaller generator. This is enough for the basics, but insufficient for the four desktop PCs that they have. Consequently, these computers are rarely used, and the teachers tell me that they are too difficult to manage anyway. Being battery powered and far more power efficient, XOs are far more suitable.

We have successfully deployed XOs to every school-age child in Dhalinybuy. I’m still out here, so I don’t yet know the status of the other HLCs. I am, however, confident that they are operational, given the ease at which we got things going here.

Through the access point, every XO (and hence every child) can collaborate and share their activities in Sugar. This also facilitates an Internet connection for all the XOs, through the NT Schools network. They are now open to a wider world of information and communication.



©2010 Sridhar Dhanapalan.

This work is licensed under a Creative Commons Attribution-Share Alike 2.5 Australia Licence.

Creative Commons BY-SA Licence

.

May 26, 2010

Privacy through peer-to-peer

We have a tricky problem to solve as we begin to use the Internet on more and more devices: making our data available on all of them. These devices include desktop computers, laptops, phones, tablets, consoles which play movies and music, cars with telnet ignition interfaces, etc. I believe that the only tenable long-term solution is a convenient and secure method for sharing this data peer-to-peer.

Right now Internet users are tending to solve the problem by delegating responsibility for their data to a third party who is supposed to keep it secure and supply it back to the user on request. This way you can upload data from one of your computers and download it (even automatically) on the others when you need it. For things like your files, email and contacts there are services available such as Dropbox, Apple's MobileMe and Canonical's Ubuntu One.

Some kinds of data are more sensitive than others. As a matter of policy I dislike making my data available to other parties unless it's necessary and I do it explicitly. It is well known amongst those who use email encryption that the encryption should be used all the time so as to not draw special attention when you do send something sensitive. This is also one example of keeping your data your own as a matter of course. Another advantage is that you use the same workflow all the time.

Suppose, as has happened before, I need to work with a digital copy of an art piece where all hell would break loose if it was leaked or shared. It would be downright stupid of me to upload it to somebody else's computers where I'm no longer directly responsible for the data. If I don't upload it, it can't leak, so the most sensible thing is not to do so. There's unlikely to be a problem even if I did use someone else's storage, but I don't want to take the risk. Now I need to start using special methods to move the data around such as an external hard drive or scp. How inconvenient. All I really want to do is to easily copy stuff over a network to the other computers without a middle man.

The trouble is it's a pain to set up that kind of service on the Internet right now. It won't really be viable on a large scale until IPv6 takes off in a big way. When that happens every device on the Internet will be individually addressable (and hopefully not excessively firewalled). Over the next couple of years this will become more and more attractive, particularly as the speed of Internet connections increases.

But why stop at ultra-sensitive data? Many of us store things like our email, instant messenger logs and social networking data on other people's computers. Google might even store your music and stream it to you. If we have a convenient method of synchronising our own data on our own computers we may as well do so and thereby eliminate the risks of losing direct control of the data. And we get an awesome backup system for free.

So we have this neat Diaspora thing coming out in a few months. It's going to provide decentralised peer-to-peer encrypted social networking. It'll hopefully be a little like Facebook with absolute control over to whom your data is given. I'm going to love playing with and possibly seriously using Diaspora but I doubt it will deliver enough convenience to steal many Facebook users, at least for a while.

I'm seeing something more here. The Diaspora devs explain that they want to make an extensible framework where anybody can create any sort of application or payload for sharing between peers. I reckon they're seeing a bigger picture where peer-to-peer is a necessary part of keeping our data private on the Internet. Initially, though, they're creating a secure social networking tool. That's a great idea because that's where users are hurting the most right now.

Diaspora or a project like it must succeed. I refuse to acknowledge a future of computing where users are expected to surrender data to others to get their work done on the Internet. It may not matter even most of the time. There is always a place for your own computer, your own data, your own control.

The cloud is getting more convenient all the time. With a little resourcefulness and work, peer-to-peer systems between our own computers can be convenient too. Then we won't have to throw out our old expectations of privacy for the sake of that convenience.

May 24, 2010

May 21, 2010

Clarke and Dawe on the world’s lending merry-go-round

Clarke and Dawe on the 7:30 Report last night, for my international readers:

How can broke economies lend money to other broke economies who haven’t got any money, because they can’t pay back the money the broke economy lent to the other broke economy and shouldn’t have lent it to them in the first place cause the broke economy can’t pay it back?”

You’re wasting very valuable time, Roger.”

May 20, 2010

Fonts

I’m amazed that Helvetica, a rather beautiful font, costs USD$741. I know fonts take a lot of time to make, and generally cost a lot of money (a conversation I had with Keith Packard sometime in 2004 cemented this into my head). Recently, I read in Monocle about Japanese font-maker Morisawa and figured that fonts are truly, really expensive.

Apple ships Helvetica. But as a “dfont” package. Good thing there’s a utility called DfontSplitter (Windows and MacOSX only). It allows you to extract the font as a TTF. But legally, you might not be able to use it for Web Embedding either.

Today, I discovered Cufón (source). Its a simple web utility – upload the font, and it generates the font in a format, and then there’s a little rendering engine that pops out in JavaScript for your usage. Very handy. Now websites can have embedded fonts, and things start looking the way you designed them to look.

As an aside, I used to generate favicon’s using an image editor. Nowadays, it seems like you might as well just move to the web – one less use for desktop software – use the favicon.ico Generator.

Related posts:

  1. Google Documents, Percipitate
  2. Sharding for the masses: Introducing the SPIDER storage engine (OpenSQLCamp @ FrOSCon)
  3. Acrobat.com Office, Buzzword Review



May 16, 2010

Climate Servoing considered harmful

It turns out that the earth will be quite uninhabitable in 300 years time. But before anyone gets any brilliant ideas that perhaps we can get out of this bind with a bit of engineering such as spraying sea water into the clouds, let me just say that part of my day job is tightening up a servo control loop for a 250 tonne telescope and 550 tonne dome after new motors and controllers were installed][1]. While you're still learning how a system responds to your servo, you often send it into nasty oscillations. When I can hear the preload pinion banging up against the gears, I know I'm causing abnormal wear. But the telescope responds on timescales of seconds. I can fix the loop in the order of an hour everytime the guy who built the motors controllers changes some parameter of the motors such as the torque demand headroom, now that we're heading into winter and we find the gearbox oil is changing viscosity. When the atmosphere responds on timescales of centuries, it's going to take a few more centuries before we learn how to control the oscillations. Please don't send the climate into oscillations. I like being able to live in Australia.



[1] While I was riding up to Qld on Friday, it suddenly occurred to me as to why we're been having the motors shut down on us for the past couple of weeks, along with some associated nasty symptoms. I then proceeded to write some code in my head instead of concentrating on the road and the rider in front of me. Turns out the motor controllers aren't velocity servoing as advertised, but are instead internally integrating the velocity demand fed into it back into a position, and trying to servo to that position. That's not what we want or need, and it's not helpful dagnamit. Especially as the gearbox oil cools and become more viscous during winter. I'm not detuning the servo all year around just because the gearbox oil is cold for a couple of months of the year and it takes longer to spin up to speed.

May 13, 2010

The Gold Standard

I’ve been trying for a while now to figure out why I dislike the gold standard — that is, pegging currency against the price of gold. I think currencies are fundamentally arbitrary — they’re a convention that needs to be (roughly) agreed on, but whether that’s $1 for a ham sandwich or $1000 for a ham sandwich doesn’t make much difference, as long as everyone agrees which of the two it is. By that argument, saying $1 is worth 23 miligrams of gold should be fine. Now sure, that breaks down if you suddenly get a huge increase in supply of gold — if someone posts a video on youtube how to turn grass clippings into pure gold, you’re going to get some pretty serious (and worse, uncontrollable) inflation and it’ll cost more than 230 miligrams of gold (or a mower load of grass, or $10) to get a big mac and coke. But worries about vast new sources of gold is probably not a realistic objection, at least until we have asteroid mining.

So I think there must be something else to justify opposition to the gold standard, and in particular that at some point you have to argue that not being able to inflate your currency whenever you want is actually a bad thing.

To some extent the consequences of being unable to devalue your currency/inflate you money supply is playing out in Greece now, and depending on whose explanation you believe, was a cause of the Great Depression.

In both cases, the theory goes that crushing debt (war reparations, too much spending) and an inability to actually pay that debt can’t go on forever, and inflation is an easy way to get out of that. At any rate, easier than bank failures, easier than government defaults, and easier than going to war. Basically, inflation turns into a way to force everyone to forgive their debtors by a given percentage, rather than having to pick some people who get nothing back, while others get everything.

On the other hand, inflation only “really” helps with long term debt — if you have to pay someone a million dollars in ten years time, a 15% annual inflation rate lets you pay it all back by doing the equivalent of $135,000′s work in each of the last two years, even if you do nothing for the first eight years. But if you owe someone a million dollars in two years time, and can only earn $135,000 each year, you’d better hope for inflation of over 540%, or you’ll want to start bankruptcy proceedings now.

That’s essentially what happens with debt that has a variable interest rate (or rolling debts) — the lender guesses what inflation’s likely to be, and says “ok, I won’t send the boys around to collect my $1M today, but next week you owe me an extra 1%”. Which of course means inflation isn’t going to do you much good if your debts are all short term (135B euros of Greek bonds due within the next five years, a five year fixed rate mortgage, credit card debt, etc) — the people loaning you money have been clever enough to factor in the possibility of inflation and still make you pay what you owe.

In theory, all that’s fine and proper: you shouldn’t borrow more than you can pay back, and there should be some negative consequences to living off promises you never make good on.

In practice, people get into situations where it’s simply impossible to pay back a debt. Whether that’s a thin veneer over slavery in the form of debt bondage, or managing to spot a bunch of uncovered short sellers who were willing to commit to selling (in effect) more than 100% of a company, or something else.

There’s probably no simple solution to that — people are always going to want to buy now and pay later, people are always going to try making that “pay later” part as expensive as possible, and people are always going to make mistakes in estimating what’s possible: all of which leads to people getting into more debt than they can actually pay. In that view going to the gold standard to stop government getting themselves out of debt by printing money just makes it harder to get out of too much debt, it doesn’t actually decrease the factors that get people (or governments) into debt in the first place, and thus actually makes things worse, not better (at least overall: people holding long term government bonds whose worth might be inflated away have every reason to like the idea).

Of course, the main resolution surely has to be liquidation/bankruptcy proceedings, where creditors only end up getting a percentage of what they’re owed adjudicated by some trusted third party, the debtor gets put on a list of bad people who don’t pay their debts reliably, and otherwise everyone goes back to living their lives, usually including the bankrupt individual. That approach seems a lot better than the pure debt-market approach of having risky debts become increasingly short term and increasingly expensive until either someone rich comes along and provides a bailout, or there’s a global recession.

this service terminates here. all change please. all change.

This blog has come to an end. This has been a long time coming, but basically I feel that I can no longer support Livejournal with my content and money. Secondly, lots of people on the technical Planets don't enjoy reading the things I write about my life.



I haven't stopped blogging though! I'm moving my personal life over to danni @ Dreamwidth, who are awesome. If you want to be part of Dreamwidth, I have some invite codes; comment here and hopefully I can hook you up.



I have also started a geek blog on blogs.gnome.org: the aptly named scrambled tofu. This is the blog that should be syndicated to any Planets.



I won't be cross posting; this is the end of the tape. When I'm feeling procrustinatey, I'll send around emails to ask for my Planet subscriptions to be updated, or if you're a Planet administrator, and super keen, here's the feed. Leave a comment below and I won't bug you with email :o)

May 12, 2010

May 11, 2010

Donating to Firefox add-ons

I went to get Flashblock installed in my browser after a hiatus, and saw something interesting:



Firefox plugin donations

That’s the add-ons page telling me that you can now make a small donation to the developer of the plugin so they can continue development on it. There’s a suggested amount (in this case $5), and when you do click on Contribute, you’re given options: one-time suggested donation, any price you’d like to give, or a recurring monthly contribution. All this is fulfilled quite simply via Paypal. I actually know of one of the developers of Flashblock – a Malaysian developer, Philip Chee.

Money in a capI find Flashblock useful. Donating definitely makes sense. Then I immediately thought of WOT – Web of Trust. I use the service on a daily basis. Would a donation of $10 to WoT make sense? Absolutely. It makes the web safer for me to browse. It adds that added layer of security before I go ahead and click on a link.

Do you donate to the add-ons that you use the most? Did you even know you could donate to add-ons for Firefox? I would totally appreciate some comments on this.

Disclosure: I have a professional interest in Web of Trust.

Related posts:

  1. Firefox Download Day
  2. Migrating Firefox/Thunderbird from Linux to OS X
  3. itunes, fedora, firefox… flock next?



May 10, 2010

The Internet: A Democracy

Tonight I appeared briefly on ABC's programme Q&A which was this week about Internet filtering. I asked of the panellists a question which read:

(John Gilmore famously once said, "The Net interprets censorship as damage and routes around it." However,) "The Net" is not some robot but the result of how everyone uses it.

Since the Internet is effectively a democracy in its own right, do you believe that any nation should have the right to impose regulations upon it? Could it have its own sovereignty?

On the broadcast the part in parentheses was omitted. As I watched the panellists attempt to answer the question (or dodge the question) I quickly realised that apart from Brett Solomon, the intent of my question was mostly missed. In hindsight I see that by trying to fit my question in a small number of words I obscured my meaning.

Nonetheless the word "sovereignty" in particular brought up some good discussion about how sacred the freedom to do what you like on the Internet is. An on-screen tweet also suggested that it was more like anarchy then democracy, citing /b/. This is a fair point but still not quite what I meant.

I would like to elaborate a little about what I did mean. I do not necessarily think that the following is true. It is more of an interesting thought experiment for me, which is why I attempted to pose it to the Q&A panel.

The Proposal

The Internet is evolving all the time and I consider this to be very much like a democratic process. I'm going to stretch this analogy a bit. Instead of a piece of land on which we live, we have a relatively boring network over which we can make connections between computers using generally the TCP/IP suite with IPv4 and IPv6.  There is a lot of complicated technology which makes it happen but fundamentally that's all we're getting. An empty block.

On one level standardisation is generally democratic. Most Internet standards are developed by working groups with input from the entire industry -- the community and the businesses which run and use the Internet. We have requests for comment, and draft and approved specifications. Best of all, these working groups are doing this work knowing that it has to meet the needs of the stakeholders. Their standard has to gain general acceptance. A political party's policy must also gain general acceptance for them to be elected. Otherwise we go and use a different standard or vote for a different party, or make our own.

Standards are mostly about the Internet at the level of those who implement it. On another level there is a form of democracy in what services Internet consumers choose to use. Lots of people use email. Lots of people use peer-to-peer services. Lots of people use web servers. Lots of people use Facebook. The services which have server and client software, those which are supported, are those which have gained widespread popularity. Nobody is forced to use any of them, but it is inefficient to work against the grain because you cannot communicate with others as easily. This process of being driven to accept what the majority wants is also a democratic feature of the Internet.

The major point where this analogy falls apart is that democratic governments in the physical world can outlaw certain activities. This is not enforceable on the Internet because whatever the "crimes" are, they can be obscured easily. You can't encrypt your stealing of a car, but if someone invented an invisibility cloak that would be a similar effect.

Yet by and large this global community of Internet users feels that the benefits of this freedom outweigh the disadvantages of not being able to outlaw activities. This is where John Gilmore's famous quote applies. He originally made the statement about USENET but he stands by it to this day in a more general sense too. If censorship is applied to the Internet users change which services they use; they change how they use the Internet and therefore its form and features in order to circumvent that censorship. By a process of consensus, users do not want the censorship to happen. If the majority of users wanted to embrace censorship or any other arbitrary restriction, the changes necessary to work around it would not gain traction. People would say, "Stuff Tor. Proxies for rent? Hah, they'll be out of business in no time." More people would block Tor users because they feel that the trouble caused through anonymity is not outweighed by the ability to circumvent censorship.

The Internet community is big. It has representatives in the form of users from every nation. It is a community which is saying in a more-or-less democratic way that Governmental censorship is unwanted.

In that context, who is the relatively small Australian Government to say, "We know better than the Internet community. We should apply blacklisting for RC content."?

It is not even a question of the technical implementation of the policy. It is a restriction upon the Internet community which has by majority decided that it does not want the restriction. But the Internet community is much larger than the Australian community or Government, so there may be an argument to say, "The Internet overrules the Australian Government. No filtering." If so, should national Governments of the physical world respect that?

In saying so it is not suggesting that anarchy should be permitted on the Internet either. Crimes on the Internet are generally crimes in the physical world too.  Let the nation which has sovereignty over the physical space in which the criminals reside and the crimes are taking place enforce the laws of that area. However, they must not disregard the wishes of the (larger therefore more important) Internet community in their enforcement.

Well Then

Am I crazy, onto something or simply logically invalid? I haven't yet decided. Given that the Internet is a shared global resource and medium I think we have to at least settle on the issues of what individual nations may or may not do before we can fully evaluate censorship or national law enforcement proposals.

Wikipedia available in print via PediaPress

Book vendorOver a year ago, I found out about Wikitravel Press from Jani Patokallio, at BarCampKL 2009. I was pretty excited since it was based on the open content Wikitravel, it was printed on demand, and it was updated on a regular basis. Imagine getting accurate travel guides when you were going somewhere (as opposed to a dated Lonely Planet guidebook)?

When I was young, I used to pore through a set of encyclopaedias from Grolier, that my late grandfather had purchased sometime in the ’60’s. These were wrongly disposed off, during a move, and the last time I encountered a traditional encyclopaedia, was a few months ago, albeit a digital version. For me, Wikipedia won.

Naturally, I’m excited to see that you can now get Wikipedia entries, printed in book-form now. PediaPress now offers custom books to all users. Check out the book creator on Wikipedia.

It got me thinking. Would we eventually see textbooks being printed out from Wikipedia? I took a look at some of the pre-prepared books, and found one for College Mathematics: Algebra. Its over 400 pages of dense mathematical content; about the typical size of a math textbook. Coursework preparation will be more open, and quality will be crowd-sourced.

More interesting for me, and other developers using MediaWiki? The tools for creating your own press are available at the PediaPress Open Source Repository. Therein lies a Python library for parsing the MediaWiki articles (mwlib), another library for writing the PDF documents from MediaWiki articles (mwlib.rl), and a bunch of extensions to collect MediaWiki articles and output them to PDF, XML, or even OpenDocument format (ODF).

What does that mean? If you’re using MediaWiki to create documentation, you can quite likely create a printed manual pretty darn easily. No mucking around with writing documentation in DocBook XML… you can use the friendly MediaWiki markup and syntax.

Related posts:

  1. On Wikipedia and traditional Britannica
  2. Photograph of Melanie C in Wikipedia
  3. OpenOffice.org 2.3 exports to MediaWiki



May 09, 2010

Long term storage of digital memories

For once, and with trepidation, I actually do ask for advice!

My parents have several digital cameras and a digital video camera. It would be nice to have access to family photos and movies past their next hard drive crash and/or computer replacement. But all consumer digital media seems unreliable enough for long or even medium term storage of family photos and similar.

I could organise some kind of backup routine for them, but the developing and nagging and testing and recoveries involved would annoy all three of us; contrary to the common parent-of-geek mythology my parents are no slavish devotees of my computing advice and like to keep a great deal of control themselves. And I get bored with that stuff. I want more set-and-forget: ok, stick your photos here and they'll be there ten years from now.

One possible solution would be something like here, upload them to [some remote hard drive of mine]. Advantages: I have, unlike quite a few commercial entities, managed to hang onto a great deal of my digital files for ten years. Disadvantages: the software for this largely seems to suck in that it will be difficult to give them the motivation in terms of "and it will be shared with family really easily!": lots of creating galleries and so forth required to make things findable. I use F-Spot and Gallery and both of them leave sad dingy marks on my soul sometimes. I value the resulting organisation enough to do it, they probably do not, at least initially.

So should I just tell them to pay for a Flickr account? Or maybe Picasa? Or some similar website I probably haven't heard of? I realise that stuff in the big wet cloud is vulnerable to both commercial failure of the provider and catastrophic failure on the part of their sysadmins, but at least it's not vulnerable to my boredom or that of my parents (although Flickr at least is vulnerable to forgetting to pay Flickr). What has good desktop tools with the fewest clicks between plug camera in and digital files are safely squirreled away and as a bonus available for viewing by selected people?

I realise none of this is fail-safe, and if they aren't interested in preserving the data it ultimately won't happen, I just want to lower the barriers as much as possible and give it the best chance.

If you have thoughts and don't know how to get in contact, email per this.

May 07, 2010

Blonk! Blonk! Blonk! (or, why I will never use a Mac)

(because I'm exceedingly lazy and would prefer to use something that's easier to use)





I was searching to see whether there was any attempt whatsoever to solve Java's focus problem wherein it incorrectly uses XSetInputFocus() to steal the focus instead of just letting the window manager set policy like Dhog intended. Thereby playing not-nice with focus-follows-mouse window managers.



I believe the same bug is the cause behind OpenOrrifice's showstopper (for me) and general indicator of programmer competence, bug #19489, and a similar bug in opera (that has been blamed on Java before, but which can't be because I've turned java off in the browser).



I didn't find the answer to that, but what I did find was this.





I know it doesn't sound like a big effort, but programmers are really, really lazy, and they like to minimize motion. They'd use feeder tubes if the Health Department would let them.

...

That's what not having autofocus is like to people who've been using it for the past 10 to 30 years (in my case, 20 years). BLONK! BLONK! BLONK! I'm serious. It's that bad. Not exaggerating even a tiny bit.

...

It's that whole "we designed it the right way for everyone" mentality that turns off so many would-be Mac users. [editor's note: anyone else reminded of Gnome here? The comments about undocumented interfaces come to mind here too]

May 06, 2010

Nvidias CUDA

Actually got around to looking a CUDA on Nvidia cards at work yesterday.

Got the SDK and sample code up and running on the mac pro, next thing to do is find some maths problem to do on it which will be faster than on a CPU / Grid. well at least that it will be faster given a better GPU the NVIDIA GT 120 isn’t that fast considering there is 2 Xenon quad core chips 2.4ghz. on the mac then there at 6 macs connected to the Xgrid. so it will be a bit hard to do a direct comparison.

Also had a informal meeting with the lab manager from physics we discussed how that for ever increasing amount of remote courses at the university how he plans to do physics labs for the astronomy course next year.

Ended up sending him to have a look at the UTS remote labs http://remotelabs.eng.uts.edu.au/ and told him to keep us posted since it’s all of interest.

So any ideas how to show that CUDA and GPU computing is useful ? or how to conduct an astronomy lab when your students are in Queensland ?

May 05, 2010

Viewing attachments when using mutt remotely

Yes, that's right, I'm still in the dark ages and do not yet use Gmail for my email. Even though it has IMAP and everything. I still use Mutt.

I almost always use Mutt locally, using offlineimap to sync IMAP folders to local maildirs. This means I don't usually have the problem of being unable to view non-text attachments. However, for the next little while I'll be using Mutt on a remote connection.

Don Marti has one solution to this, which assumes that you are accessing the server with Mutt on it via SSH (probably true) and are easily able to create a tunnel to your local machine, which is trivial if you are using a commandline ssh client, but while you can do it with PuTTY I figured it was just annoying enough that I might not bother. (And I doubt you can do it at all with those web-based SSH clients.)

My alternative assumes instead that you have a webserver on the remote machine that has mutt on it. It then just copies the attachment to a web-accessible directory, and tells you the URL where you'll be able to find the attachment. It's thus a very trivial script (and I doubt very much it's the only one out there), but perhaps using mine might save you fifteen minutes over coming up with your own, so here it is:

copy-to-dir.sh (in a bzr repo)

Sample output is along these lines when you try to view an attachment in Mutt:

View attachment SOMEPDF.pdf at http://example.com/~user/SOMEPDF.pdf
Press any key to continue, and delete viewable attachment

In order to use it, you need to:

  1. copy the script to the remote machine where you use mutt;
  2. make it executable;
  3. edit it to set the OUTPUTDIR and VIEWINGDIR variables to something that works for your setup;
  4. set up a custom mailcap file much like the one Don Marti gives, that is, put something like this in your ~/.mutt-mailcap:
    text/*; PATH-TO-SCRIPT/copy-to-dir.sh %s
    application/*; PATH-TO-SCRIPT/copy-to-dir.sh %s
    image/*; PATH-TO-SCRIPT/copy-to-dir.sh %s
    audio/*; PATH-TO-SCRIPT/copy-to-dir.sh %s
    
  5. set mailcap_path = ~/.mutt-mailcap in your ~/.muttrc file.

Something like this probably could work for Pine and other text-based email clients used remotely too, but I'm not sure howi because I don't use them. And if someone wants to document this in a way that assumes less pre-existing knowledge, go ahead.

Also, making your attachments web-accessible means that they are, well, web-accessible. I've set up a HTTP Auth-protected directory using https for this, you should think about your own setup too.

Creative Commons License

"Viewing attachments when using mutt remotely" by Mary Gardiner is licensed under a Creative Commons Attribution 2.5 Australia License.

May 04, 2010

4 May 2010

apenwarr:

What was wrong with the SOCKS server that SSH provides? Playing transparent proxy tricks is cute, but why not make it work using SOCKS and then it would be more generically useful?

Better still, you can use an otherwise unused corner of IPv6 address space for your dynamic proxying so you aren't messing with the client's Legacy IP routing at all.

May 03, 2010

telepathy-glib GObject-Introspection and TpBaseClient

In Telepathy libraries such as telepathy-glib, telepathy-python and telepathy-qt4 there are two kinds of API. There's what we call low-level API, which is a direct wrapping of the D-Bus API exposed by Telepathy components and automatically generated from the Telepathy specification; and high-level API, which is hand-written and calls the low-level API internally with the goal of making it easier to implement things with Telepathy.



It's relatively easy to expose the low-level API into your language by writing an output module on top of our specparser and using your language's native D-Bus support. However writing a high-level API is a pain in the neck, and requires you to redo a lot of work that's already been done once.



Basically this sucks, so over the last couple of weeks a few of us have been working to support GObject-Introspection for the telepathy-glib high-level API. This means that we can now expose telepathy-glib functionality into languages such as Javascript (via GJS) and Python (via PyGI). We're choosing not to bind the low-level API that telepathy-glib exports, mostly because it's not very pretty and languages have better ways to expose this (N.B. this does create possible ordering problems because you are likely to have two separate DBusConnections). We're also working on extending telepathy-glib to make more things easier.



One such extension is TpBaseClient and its subclass TpSimpleObserver, which are designed to make it significantly easier to write a Telepathy clients such as Observers (TpSimpleHandler is coming soon).



The introspected bindings are not yet production ready, we're still going through and checking everything is annotated correctly, and we've also exposed a few limitations in GJS and PyGI (e.g. GArray support) that don't yet have fixes merged.



Still, putting this all together, it's now possible to write the basis of a Telepathy Observer in Javascript:

const Tp = imports.gi.TelepathyGLib;
const Mainloop = imports.mainloop;

function observe_channels(observer, account, connection, channels,
                          dispatch_op, requests, context, user_data)
{
    print("observe_channels");

    print("account = " + account.get_object_path());
    print("connection = " + connection.get_object_path());

    for (let i in channels) {
        let channel = channels[i];

        print("channel = " + channel.get_object_path());
    }

    if (dispatch_op != null)
      print("dispatch_op = " + dispatch_op.get_object_path());
    else
      print("dispatch_op = (null)");

    for (let i in requests) {
      let request = requests[i];

      print("request = " + request.get_object_path());
    }

    context.accept();
}

let dbus = Tp.DBusDaemon.dup();
let observer = Tp.SimpleObserver.new(dbus, true, "JSObserver", true,
                                     observe_channels);

observer.add_observer_filter({
  "org.freedesktop.Telepathy.Channel.ChannelType": Tp.IFACE_CHANNEL_TYPE_TEXT,
});

try {
    observer.register();
    Mainloop.run("");
} catch (e) {
    print("ERROR: " + e);
}
All of this work is now merged into telepathy-glib master, and will be released as part of the telepathy-glib 0.12.

Henry Tax Review, post release

And here was me thinking forming an opinion on yesterday’s tax review would be hard. Turns out, not so much: the review itself was really well done, pretty much what you’d hope for from a professional public service; the government’s response, on the other hand, was impressively gutless.

The most interesting recommendation (to me) in the Henry Review was the changes to personal income tax, which I’d summarise as:

  • Raise the tax-free threshold from $6,000 to $25,000
  • Change the official income tax rate to 35% for up to $180,000 per annum (ie, almost everyone), and leave it at 45% for above that
  • Drop the Medicare Levy, Low Income Tax Offset, etc, and just have a single rate
  • Fringe benefits tax should be simplified (particularly for cars), moved to market valuations, and taxed progressively rather than always at the top marginal rate
  • Introduce a standard deduction for work related expenses to simplify filing

At first glance, I thought the 35% rate seemed high (it’s currently 15% to $35,000 and 30% to $80,000 and 38% to $180,000). But graphing the rates seems to dispute that thought:

Tax Rates

There is some loss — people earning between $35,000 and $65,000 pay $250 more in tax per year, which then rises $1,000 more per year at $80,000 dropping back to parity at $113,000. People earning more than that get a small tax break that eventually levels off at a flat $2,000 tax benefit for people earning $180,000 or more. At the other end of the scale, people earning between $18,000 and $30,000 pay between $450 and $1500 less tax per year, which seems sensible. And of course, everyone benefits from having a simpler tax system, and (in theory) not having to pay an accountant for help filing your return. And the marginal tax rates become both easier to understand and generally the same or lower, which hopefully means less people are in the situation of thinking “well, I could work a few days a week, but I’d end up with less money that way, so I’ll set at home and watch Oprah instead”.

(Caveat: those numbers aren’t strictly right — they’re based on the current marginal rates and the LITO; so they don’t include the Medicare levy, and probably other things. This is why I’m not the treasury department. But I think it’s a fair indication of what the effect would be)

It’s not clear to me what Labor’s planning to do with the recommendations here — they haven’t accepted them, but they didn’t officially reject them yet either. Presumably they’ll have to say something, sometime, about it, but I don’t see any advantage to waiting if they were going to take this and run with it. I guess that makes them an exercise in cowardice: doing something about it would be too hard, as would finding actual flaws with it, so let’s just ignore it and hope we get re-elected anyway.

The company tax changes seem similarly motivated — dropping two percent over five years? Is anyone seriously going to pay attention to that? I don’t think so; and the Henry review’s recommendation was, in my opinion, much less subtle: dropping from 30% to 25%, the idea being merely to stay in line with international trends, particularly those for small economies. I suppose I can appreciate taking some time to cut the rate, but not if you’re also only going to cut it by what looks like a token amount.

As far as I can see the only reason that recommendation even got the token support from the government that it did was that the Henry review explicitly linked it with the 40% resource rent tax — recommending that the 25% company tax and the 40% resource tax be balanced to maintain an overall 55% tax rate (25+(1-25)*40=55). I can’t say I understand the resource rent tax (or the “super profits tax” as the government calls it) — but then I don’t understand the motivation for it either; if you get $90B of profits, how do you only pay $10B in “resource taxes” when you should be paying at least 30% company tax on profits, which would be $27B? Or are we not counting some tax receipts, in order to make the profits sound more unfair? The numbers all sound very shoddy there.

And of course, the government is using the “super profits tax” to pay for superannuation concessions, which is a clever sound bite I’m sure; while the Henry review was recommending they be tied into infrastructure spending, which seems like an actual logical link (Losing non-renewable resources? Spend the proceeds on stuff that will last…) But a $700M infrastructure fund, versus a $9,000M resource rent tax doesn’t sound like an impressive match to me.

As far as simplification goes, there seems to be lots in the review’s recommendations, and pretty much none in the government’s changes. Whether it’s justified or not, the resource tax is a bunch of extra regulation, that’s not accompanied (as far as I can see) by any reduction in regulation. I guess I’m not terribly surprised, but that was the one election promise that I was actually impressed by and that I figured the government might be willing to keep.

May 01, 2010

Ubuntu 10.04 LTS released, MariaDB 5.1.44/5.2-BETA VM’s available

DocklandsA big congratulations to Ubuntu for the release of 10.04 LTS. While I haven’t had the chance to upgrade, I see everyone on Twitter and in the blogosphere say they are really like the Lucid experience.

A couple of days ago, I made mention that there were VirtualBox images of MariaDB out there. Turns out there were so many downloads, Mark has had to upgrade his Internet connection!

Anyway, to the point: Mark has created Ubuntu 10.04 LTS VM’s with MariaDB 5.1.44 and MariaDB 5.2 BETA. Don’t hesitate to download them, and send feedback.

Have a good weekend playing with Ubuntu 10.04 LTS Lucid Lynx release, and enjoy using MariaDB on it.

Related posts:

  1. VirtualBox images for MariaDB
  2. MariaDB 5.1.44 released
  3. MariaDB 5.1.42 released!



Henry Tax Review

The Henry Tax Review is supposed to be released tomorrow. Since that might warrant a blog post, and possibly even some criticism, I thought it might be interesting to note down some criteria beforehand to remove one avenue for bias.

One issue for regulatory reform is whether changes make the entire system simpler or more complex — more complex regulation potentially handles trickier situations more “fairly”, but at the same time forces everyone to incur the cost of understanding all the complications, even if only to be sure they don’t apply in their situation. The Rudd government made an election promise to that effect:

Labor believes that when making new regulations, governments should remove an existing regulation and should design rules with small businesses in mind. We call this approach ‘think small’. It will require government departments and agencies to better understand the realities faced by businesses on the ground. Labor will adopt a ‘one-in, one-out’ principle for federal government regulation. This means that when a new regulation is proposed it must be accompanied by a proposal to remove an existing regulation.

There’s a deregulation group as part of the Department of Finance, but I haven’t seen much talk either way as to how this promise has been holding up. In theory, based on this principle, the Henry review should be proposing about as much reduction in regulation as new regulation though.

One of the obvious ways to reduce the complexity of the tax system would be to remove the various GST-free categories of goods (unprocessed food, etc). It would probably be appropriate to compensate that with a small increase in some welfare payments.

It’s probably also one of the few changes to the GST that’s within the review’s purview, given the clause in its terms of reference that goes “The review will reflect the government’s policy not to increase the rate or broaden the base of the goods and services tax (GST); preserve tax-free superannuation payments for the over 60s; and the announced aspirational personal income tax goals”. It’ll be especially interesting to see how true the Henry review has stayed to that policy, compared to the conclusions being drawn from Rudd’s hospital plan on a backflip there.

Personally, I quite like the “Reform 30/30″ proposal, which involves a massive simplification of both welfare payments and income tax. Supposedly it would boost government revenue by $15B per year, which is a significant fraction of the $125B in income tax or $43B in GST received in the 2008/9 financial year. On the other hand it comes at a cost of not giving welfare bonuses to people doing good things (having kids, buying houses, studying, etc) and taking less account of various other ways in which you might be rich other than having a high paying job (rich parents, rich spouse, money already in the bank, nice house, etc).

Presumably anything like that would be a non-starter politically, but some movement in that direction ought to be plausible. There’s been some talk for a while now about having a simplified tax return, so that you can just tick a box and accept whatever the ATO says rather than fill out a bunch of forms — basically heaps easier and quicker, but you don’t get to claim lots of deductions. Given the ATO’s electronic systems and reporting of interest payments by banks, and PAYG contributions by employers, that ought to be pretty plausible to setup, and might start paving the way for cutting out lots of personal tax deductions — why keep them if barely anyone’s using them, after all?

That, at least, is kind-of like cutting welfare payments — a tax deduction for $1000 is roughly the same as a receiving a cheque from the government for $300 if you’re at a 30% tax rate. Of course that means that deductions are being more considerate of the welfare of people paying more tax, which is similar to being more considerate of the people who lease need consideration.

I can’t see how the Henry review will be able to recommend much in the way of cutting welfare expenditures in general ($125B of expenses in 2008/9), but they’ve at least been told “The review should take into account the relationships of the tax system with the transfer payments system and other social support payments, rules and concessions, with a view to improving incentives to work, reducing complexity and maintaining cohesion”. So maybe there will be some ideas on this.

Maybe this will also mean the Ergas review will be revealed soon too. It looks like it’s even more out there than the 30/30 proposal, with a roughly flat 20% income tax, raising tax on income from superannuation, and taxing the family home. I’m pretty surprised that there’s anything out there more wacky than what the Liberal Democratic Party came up with, but maybe that’s due to its progenitor — supposedly Turnbull ordered the review as shadow treasurer without bothering to even tell Brendan Nelson. Still, it would be interesting to be able to compare the reasoning and recommendations to those of the Treasury-Secretary’s in tomorrow’s report.

April 30, 2010

Quick notes: Monty Program Group Blog; Rename Maria

A couple of things:

  1. There is now a Monty Program Group Blog. Its brand new, and in a company where most people spend time on writing tonnes of code, expect posts to be sporadic, but of great technical nature. We’ll also cover things like events, conferences, etc. i.e. where can you meet a Monty Program person. Do subscribe to our RSS or ATOM feed.
  2. At the MySQL Conference recently, Monty announced the rename the Maria engine contest. The competition is still running, and the winner gets a System76 Meerkat NetTop. Some interesting names have already shown up. Monty says, “Make it tell a story”.

Related posts:

  1. Monty speaks about Maria
  2. Trying to reliably make MyISAM crash; Maria is sturdy as
  3. Monty is the world’s first MySQL fellow



April 27, 2010

VirtualBox images for MariaDB

Coming from a great MariaDB contributor, Mark, is:

  1. MariaDB 5.1.44 / 5.2.0 Beta Binaries for Solaris 10 SPARC, and Debian GNU/Linux SPARC. Mark does a fabulous job of building these binaries, and he does them really quickly. If you’re on the SPARC platform, give it a go. Send some feedback, also.
  2. Mark has also spent some time developing virtual machines. All you need to get started is download VirtualBox. Mark provides an OpenSolaris 0906 + MariaDB 5.1.44 VM as well as an Ubuntu 10.04 LTS + MariaDB 5.1.42 VM.
  3. It is expected by the end of this week, when Ubuntu 10.04 LTS is released, Mark will upgrade the image to include MariaDB 5.1.44.

Thanks Mark! This is some fabulous work. Go give his VM’s a try and send feedback. Would you like to see any other VM’s? Any other distributions?

Related posts:

  1. Ubuntu 10.04 LTS released, MariaDB 5.1.44/5.2-BETA VM’s available
  2. MariaDB in Gentoo; updates for Solaris/Debian SPARC
  3. MariaDB 5.1.47 VMs for OpenSolaris, Ubuntu



April 25, 2010

once past the town girls may remove their gloves

Seem to be doing weekendly trips into the country.



Last weekend Steph, Jo, Em and I went up to Edgar's Mission just outside of Kilmore. Steph wrote it up.



This weekend Jo, Em and I drove out to Daylesford. Where it is clearly (English) Autumn.



autumn colours #2


I say (English) Autumn, because Australian trees are not normally deciduous.



Daylesford does a pretty good job of catering to vegans. We had lunch at the Harvest Café and then afternoon tea at the Himalayan Bakery (which is apparently for sale).



cherry danish


Next weekend we're going to try and make it out to Ballarat for rollerderby.

April 24, 2010

O’Reilly MySQL Conference Awards 2010

The O’Reilly MySQL Conference & Expo 2010 is over. I hope all of you had a good time. I have plenty of blog posts and thoughts lined up about this, but first, I’d like to point out something that has become a tradition, that was continued in 2010: the O’Reilly MySQL Conference Community Award Winners.

Conference award winners

Tim O’Reilly was kind enough to hand out the awards this year. In case people were wondering, the awards were pewter wine goblets from Royal Selangor.

Selection of the award winners happened via voting from the alumni of winners, and was all done in a rather short period of time. Kudos to the entire team that voted. Now for the winners…

O’Reilly MySQL Community Member of the Year 2010

  1. Mark Callaghan is known for his work in leading a MySQL engineering team first at Google, and now at Facebook. In addition, the panel appreciated his insightful and always tasteful blogging, ranging from insightful benchmark reports to open source community advocacy.
  2. Kai ‘Oswald’ Seidler is a developer of XAMPP, a multi-platform LAMP stack, especially popular amongst Microsoft Windows users. Many users get their first contact with the AMP (Apache-MySQL-PHP) platform using XAMPP!
  3. Daniel Nichter created the Hack MySQL Kit, hacks on Maatkit and heaps of other software. He’s also a fabulous MySQL DBA.

O’Reilly MySQL Application of the Year 2010

Twitter was unanimously voted to be the application of the year in 2010.

Panellist Marc Delisle described his use of Twitter recently:

“Seven weeks ago I was in Niamey, Niger during the coup d’état. While borders and the airport were closed and a tank was patrolling on my street, I took refuge at the Canadian embassy where Twitter users updated me on the situation, almost minute by minute.”

O’Reilly MySQL Corporate Sponsor of the year 2010

  1. Rackspace received the award for hiring many of the core Drizzle developers, enabling them to work full-time on the MySQL fork. Rackspace also contributes to open source projects like MariaDB, Drizzle and more, providing hosting.
  2. Percona has over the last years hired many valuable MySQL contributors, and have a lot of consultants and developers extending MySQL and tools around it. Percona’s team blog on MySQL performance is also highly regarded within the community.

Another picture from the excellent James Duncan Davidson:



annual MySQL awards

Related posts:

  1. o’reilly mysql conference & expo 2010
  2. MySQL Conference Updates – The day before
  3. MySQL Conference Update: Grid is up, go promote and register!



April 22, 2010

Making UConnect suck less with urlyconnect

The University of Tasmania has student wireless Internet access, a network called UConnect. (Great pun.) This network has sucked by varying amounts over the three and a half years I've spent there as a student.

It's some sort of 802.1X beastie. Setting it up on GNU/Linux is mostly a trial-and-error affair. My current settings are WPA2 Enterprise security, PEAP authentication, MSCHAPv2 inner authentication, and an AusCERT-certified identity whose root certificate isn't included in any normal root certificate packs.

They don't actually tell you what the settings are. The current way you're supposed to set it up on Windows or Mac OS X is to run some java-ish program which asks you for your administrator password and proceeds to mess with your settings such that it should work. Furthermore, if you're running Windows and it can't detect antivirus software it forces you to install McAfee Antivirus before it will mess with your settings. But that's okay, because McAfee is awesome.

Nonetheless it can be made to authenticate and have your computer on the University network. At this point you have access to the Internet, sort of. You can make DNS requests, or open TCP connections to destination ports 80, 110 and 443. Previously you had to explicitly go through proxy.utas.edu.au to get your web traffic out, but this year they brought in transparent proxies (edit: Not true. See comments). This is a minor improvement, but still doesn't do much for my IMAPS email, Google Talk, or anything else I want to do.

The general technique I've used for this other traffic is an ssh tunnel to my computer at home, using corkscrew where necessary to get through an HTTPS proxy. ssh -D will set up a SOCKS proxy on the local computer, causing all connections to be made via the remote computer (my computer at home). All I have to do is tell pidgin and Thunderbird and possibly firefox to use a SOCKS proxy at localhost and I can use the Internet how I like.

I wish I could say that having negotiated all these challenges I can relax and concentrate on my studies. This year they introduced a new problem: connections drop out. After some random interval of normally 2-30 seconds most TCP connections will stop having packets go either way. New connections continue as normal. I have no idea what routing problem is responsible for this but it's damn annoying.

Even HTTP requests can be affected by this. Sometimes I'll load a wikipedia page and half the maths images won't load. There is one little bit of respite: if a connection lasts for longer than about 30 seconds, it will tend to live more or less forever. Weird. This gives me a solution to web browsing: I keep on trying to establish an ssh tunnel to my home computer, and once one has managed to stay established for over half a minute, I can start using it to do web browsing through the SOCKS proxy.

This is where I've been at for the last seven or eight weeks but it's getting seriously frustrating killing ssh connections until I manage to establish a tunnel which stays up. I wanted a program to automate this process. Let's make a lot of ssh connections, poke them every ten seconds for half a minute, then grab one that's still alive and start another channel with that ssh TCP connection.

This can be facilitated using the ssh ControlMaster option (-M -S). Each of the simultaneous connections puts a socket in /tmp. Then when I want to use a particular TCP connection I launch another ssh process with -S to indicate the socket to use and also with the -D option which enables the SOCKS proxy behaviour on that connection. Then I want that first background connection to hang around until I'm done with the second, then die automatically.

So I've written a program that does just that: urlyconnect. If you're at UTAS you too might find it useful. I've tested its functionality on Debian and Ubuntu GNU/Linux but I see no reason why it shouldn't run on a Mac. If you use Ubuntu, note carefully the instructions in the README about the required version of OpenSSH.

Well, that's that done. Time to go and do something related to my degree.

April 21, 2010

Altus Metrum web store opens

Bdale has just opened up the Garbee and Garbee web store to sell Altus Metrum electronics including TeleMetrum and TeleDongle.

The TeleMetrum boards aren’t quite ready to ship, so they’re marked as ‘out of stock’ for another couple of days. That’s because we’ve decided to switch GPS antennas to improve GPS reception and we won’t have enough of those until Friday to start shipping boards. The TM boards themselves are working great and we’ve flown them several times.

We’ve already sold a couple of TeleDongle boards for use without TeleMetrum. These boards contain the CC1111 transceiver chip along with an 8-pin connector that can be used for USB, SPI or serial communication. This adds long-range wireless digital communication to almost any project. Steve Conklin has posted his plans for these boards. We sell TeleDongle as either bare boards or packaged in a pretty blue box with a USB cable attached.

All of the hardware designs are licensed under the TAPR open hardware license and software under the GPL. All of the tools used to design the hardware and build the software are also free software (already packaged for Debian, of course) so you can take these designs and do whatever you want with them.

Lew Zealand

Yes, this is quite belated. I’ll explain why in a subsequent post.

linux.conf.au this year was in Wellington, New Zealand. It just keeps getting better! It’s always great meeting people you otherwise only know online. I was especially impressed by the OLPC NZ team.

Immediately following linux.conf.au, I jumped on a plane to Christchurch to embark on a week-long tour of the South Island. Long story short, it was the time of my life! I made some amazing friends. I also saw and did incredible things, including:

  • awe-inspiring views of glaciers, glacially-formed landscapes, turquoise-coloured rivers and lakes, beautiful skies and more
  • helihike: a helicopter trip onto a glacier, then hiking on it
  • a night on a boat on Milford Sound, probably the most beautiful place on Earth
  • every extreme activity I could get my hands on, including:

I have most of my photos online now:

I think what surprised me most was how adventurous I can be when I’m not in my ‘natural habitat’. I’m not normally a thrillseeker at all, but in NZ I made the decision to take a holiday from myself as well as from work and home. I even made a concerted effort to not touch computers at all. My phone was offline for most of the trip (I was using it as a camera). I never thought that being cut-off could feel so liberating.



©2010 Sridhar Dhanapalan.

This work is licensed under a Creative Commons Attribution-Share Alike 2.5 Australia Licence.

Creative Commons BY-SA Licence

.

April 20, 2010

'net hiatus again

I think I say this too often to be believable, but I am again declaring an incomplete 'net hiatus so that I have time for both my baby and some part-time work on my PhD. What this means specifically:

  • no IRC;
  • almost no time in blog comments;
  • considerably less blogging for Geek Feminism, although I'm not going to stop entirely (I write fairly seldom for Hoyden in any case, so cutting that down isn't on the cards); and
  • very little time on identi.ca/Twitter/Facebook.

A week or so back I started dropping people from my identi.ca/Twitter lists and will probably drop very many more. If you tweet more than a few times a day and/or if you regularly live-tweet events, you will almost certainly go, I just can't follow you in a way that's sensible for me. Others may go as well. (Please, no 'splains about Twitter lists. I know they exist and how to use them.) If you notice me unfollowing you... it's not you, it's me. On Facebook I'll not drop friends, since it's harder to re-add them, so there I will use lists.

I'll still be reading blogs and such, but I've reset my feed reader down to twice daily checks. I can be reached via email in the usual way.

April 19, 2010

Orangutans at the Nestlé shareholder meeting

Bravo Greenpeace Switzerland! At Nestlé’s annual shareholder meeting 2010 last week, you descended from the ceiling in the middle of the presentations with flyers and a banner asking for the company to take responsibility for their reckless actions in Indonesia.

Thousands of square kilometres of forest are cleared every day so that companies like Nestlé can make vast sums of money off consumers.

Orangutans asking Nestlé for a break

Meanwhile, Orangutans outside the venue were protesting Nestlé and asking for a break (copying Nestlé’s own slogan “Have a Break! Have a …”). The Orang Utans are pushed towards extinction by capitalist interest.

One of my closest friends was part of the act, and he recounts breaking into the ventilation system before sawing through the ceiling, and descending on a rope. The police detained them for more than 24 hours, but the message has been sent.

Bravo!

Read more (and watch videos of the spectacular descent) on the Greenpeace webpage, the Greenpeace press announcement and their blog (all in German), or on 24heures (in French). Planetsave has a decent coverage in English.

NP: Emerson, Lake & Palmer: Brain Salad Surgery

Units and prefixes

Lots of people get units wrong. Please don't be one of those people.

This is a tutorial introduction to getting your computing units right. The history of these issues is covered in extensive detail at Wikipedia.

File or Data Size

Everyone knows that computers store things as 1s and 0s. These are called bits. (Short for binary digits).

All modern computers represent things like letters and numbers with a group of 8 bits, which is called a byte. That is; there are 8 bits to every byte.

Convention is to use a lower case b to refer to quantities of bits and an upper case B to refer to a number of bytes. 220B refers to 220 bytes.

If we want to talk about lots of bytes there are SI prefixes available to us:

  • k (kilo) 1,000
  • M (mega) 1,000,000
  • G (giga) 1,000,000,000
  • T (tera) 1,000,000,000,000

So it is perfectly okay to talk about 936 kB, 1.24 Mb, 50 GB, referring to kilobytes, megabits and gigabytes respectively. It is not okay to talk about 936 mb because the prefix m (milli) means one-thousandth of a bit. You can't have parts of a bit. Please don't use the wrong case.

Binary prefixes

Confusingly, often people use terms like kB, MB and GB while meaning something slightly different.

When a computer stores a bunch of bytes in memory it tends to group them into chunks of powers of 2. This is because it needs to be able to give each spot in memory an address and it's easier to build hardware conforming to:

All addresses of the form 000000xx xxxxxxxx are stored in chip 1, 000001xx xxxxxxxx are stored in chip 2, etc.

In this example every chip has 10 bits of address space and can store 1024 bytes. If you were lazy, you would say that each chip stores 1 kB of data. But that means 1,000 bytes, not 1,024!

Similarly, with 2^20 bytes you get approximately one million: 1,048,576. So you can't say that this is a "megabyte" really, because that would mean exactly one million bytes.

There are two main solutions to this problem. The first and most common is to ignore it. This entails using kB, MB, GB, etc. and letting the person reading it guess exactly how many bytes you mean. This works most of the time because usually an approximate file size is good enough. Will the file fit on my flash drive? About how long will this take to download? Near enough is good enough.

But I'm an engineering student so I find this imprecision annoying, and sometimes we need to be explicit about what we mean. For this we have a method developed by the International Electrotechnical Commission:

  • Ki (kibi) 1,024
  • Mi (mebi) 1,048,576
  • Gi (gibi) 1,073,741,824
  • Ti (tebi) 1,099,511,627,776

If you ever see numbers like 3 KiB, 2.3 GiB, etc., you know exactly how many bytes they really mean. Better yet, if you always use those forms to represent the powers of 2, you also know that the ordinary SI prefixes always mean powers of 10.

The only downside is that it sounds ridiculous to say "mebibytes" out loud. But hey, it might be cool some day. It's only been around since 1996. If you look carefully, some computer software will express quantities in MiB or GiB. DC++ is an example.

Transfer Speeds

There are two main things to say about speeds. First of all, a transfer speed is how much data you can transfer in a given amount of time. Therefore the speed is an amount of data per second.

When speaking out loud it's convenient to talk about a "one-point-five megabit connection" but when written it should be "1.5 Mb/s".

Secondly, standard Internet connection speeds are quoted in bits per second. When you download files in your browser it usually shows you bytes per second. If you have a 1500 kb/s connection, to estimate your download speed in bytes per second you need to divide that number by 8, which is 187.5 kB/s. In reality it won't be quite that fast, but there's a big difference in the number depending on whether you're using bits or bytes.

Conclusion

I'm not as pedantic as some people. I have a lecturer who will happily write distances in Mm (megametres) instead of thousands of kilometres like almost anybody else would do.

All the same, the chances are you'll run across these numbers quite a lot and if you know exactly what they mean, more power to you. I will also be happier if I am reading something you've written and don't have to make mental conversions.

Finally and most importantly: sometimes it can be ambiguous. There is such a wide range of internet connections available these days that a download speed of 1 MB/s or a download of speed of 1 Mb/s could both be reasonable. If you're working out how long something will take to transfer you can't rely upon common sense to guess the correct form.

April 17, 2010