Planet Linux Australia
Celebrating Australians & Kiwis in the Linux and Free/Open-Source community...

January 28, 2012

Why Everyone Must Oppose The Merging of /usr and /

As co-editor of the last edition of the File Hierarchy Standard before it merged into the Linux Standard Base, I’ve been following the discussion about combining the directories  /bin, /sbin and /lib into /usr/bin, /usr/sbin and /usr/lib respectively.  You can follow it too, via the LWN discussion.

To summarize, there are two sides to the debate.  The “pro” side points out:

  1. Nothing will really change for users, as symlinks will make old stuff still work.
  2. There are precedents in Solaris and Fedora.
  3. The weak reasonings used previously to separate / and /usr no longer apply.
  4. Separate /usr has become increasingly unsupported anyway.
  5. Moving to /usr will enable genuine R/O root filesystem sharing.

The “anti” side, however, raises very salient points:

  1. Lennart Poettering supports it.
  2. Lennart Poettering is an asshole.

Fellow Anti-mergers, I understand the pain and anguish that systemd has caused you personally, and your families.  Your hopes and dreams crushed, by someone with all the charm of a cheese grater across the knuckles.  Your remaining life tainted by this putrescent subhuman who forced himself upon your internet.

Despite the privation we have all endured, please find strength to stop this nightmarish ravaging of our once-pure filesystems.  For if he’s not stopped now, what hope for  /usr/sbin vs /usr/bin?

Walktime Blog #19: No dominant Open Source home automation projects?

If you're just getting into home automation and go searching for Open Source HA projects, you'll discover there aren't any obviously dominant players - more a mix of partly-developed personal projects that aren't very portable. Why is that?

View or comment directly on YouTube: www.youtube.com/watch?v=Ywulcl_PCUM

Links for this ep:

 * Allison Randal: www.twitter.com/allisonrandal

 * Desktop Home Hacks: www.youtube.com/watch?v=a8asl5SsGy4

 * Mister House: misterhouse.sourceforge.net

 * Open Remote: www.openremote.org

 * SuperHouseTV: www.superhouse.tv

January 27, 2012

linux.conf.au (LCA) 2012

Well, it’s that time of year again – my annual geek-week at linux.conf.au. Every day there were many interesting talks to attend and many I had to miss. I am currently catching the ones I missed by watching the LCA 2012 videos.

Keynotes and Open Source DSP

The Bruce Perens keynote had many good points on why open source is becoming essential to our security and well-being in the 21st century. These themes were expanded by the other keynote speakers. Bruce stated that “open software is the only credible producer of software”, we can choose to be “slaves to tools or the people who control the tools”. Watch the talk for more information on these memes.

I am interested in “the art” of presentation (as distinct from the content) so was also interested in Bruce’s presentation style from that perspective. He appeared on stage in a suit, which is uncommon in a geeky crowd. It raised a few eyebrows. However Bruce then explained that “a suit at LCA” was a theatrical device to underscore the point that we, as an open source community, should be facing outward. Open source communities are good at talking to people within our community, but our image outside of that community is poor and misunderstood (e.g. not many people understand their email is relayed via open source, or how it helps security problems and can help preserve democracy). Our external image must be improved.

Another great point by Bruce was how Open Source is now solving tough, previously opaque problems that were traditionally considered too hard due to patents or specialised knowledge. All you need is one guy to really get into and understand the problem. Suddenly the voodoo evaporates. People then know it’s possible – a problem that our peers have solved always appears easier. This one guy publishes source and shares what he has learnt. Others start to hack the code. Bruce, much to my embarrassment (!), actually cited my DSP work as an example, for example Oslec for echo cancellation and projects like Codec 2. When Oslec started I had many people tell me “it can’t be done”, “you need a DSP chip to do it in hardware”, or “it’s all covered by patents”. The truth is that Open Source DSP algorithms are now out performing closed source competitors. For example the Opus guys have developed a world-beating open source audio and voice codec. More on that below.

Actually I really enjoyed all of the keynotes. Paul Fenwick spoke on how our mind works, including topics like “decoy choices” and the “planning fallacy”, and how playing Tetris can help with traumatic experiences. I also recommend you watch the keynotes by Jacob Applebaum and Karen Sandler.

It was great to see Jean-Marc Valin and Timothy B. Terriberry in person, presenting on the Opus Codec. I wonder if this will be the “last” audio codec. It’s open source, royalty and patent free, will be an IETF standard, codes speech and music signals from 6,000 bit/s up, and outperforms other codecs like MP3.

Codec 2 talk

As I mentioned above I am interested in how conference presentations work. Like a lot of my work, I am inclined to experiment. Try something different. Hack it. A presentation on Codec 2, like many presentations at LCA, has a strong technical component that is hard for the average LCA attendee to follow. My Codec 2 work is an extension of my PhD research in speech coding. It took me 3 years to get my head around speech coding for the PhD. So how do I communicate Codec 2 topics to a smart, but non-speech-coding aware audience?

One way to handle this is “tutorial” style. You spend about half the talk bringing people up to speed on your technical topic. Enough to explain in the second half how you applied Linux or open source to this field. This is a common approach at LCA. It can work well, but also means a lot to absorb for the audience. This can be a challenge after a day (let alone a week) of great ideas and intellectual stimulation at a conference like LCA.

Instead of the tutorial approach I hit on a different idea. Rather than confine my talk to Codec 2 and DSP theory, I tried approaching Speech Coding from a variety of tangential topics that matter to LCA attendees. I talked about codec patents, how Ham radio relates to Open Source, and finally a really easy to grasp graphical explanation of how the sinusoidal model used in Codec 2 works. I left out a bunch of DSP topics, and didn’t even put up a block diagram of the codec. I wanted the audience to walk away knowing 3 or 4 things about speech coding really well, rather than try to cover the entire, technically deep, acronym rich subject at a shallow level.

This worked well. Really well in fact – my Codec 2 talk was voted the best of the conference and I was asked to repeat it later in the week. Wow! This was especially amazing for me as the voting is done by the attendees. A nice way to start 2012 for me, after working through some tough personal issues in 2011. Here are the slides for my Codec 2 LCA 2012 talk in Open Office format.

How Good Was Your Conference Talk?

It is important to me that my talks are well received. For me it’s part of my job and I take it seriously. Here is what I look for. This applies mainly to conferences with multiple parallel threads where people have a choice in what they attend:

  • Did I fill the room (or nearly so)?
  • Were people still asking questions at the end of your allocated time? Extra points if people come up to you afterwards and ask more questions. Even better if you get hustled off by the conference organisers because the next speaker is overdue to start.
  • Did some of the more popular speakers/major contributors to the conference attend the talk?
  • Was the applause loud and enthusiastic?

Oh, and I also like to make my talks short and leave more time than usual for questions. For example in a 50 minute slot I will time my talk to be 30 minutes rather than the nominal 40, allowing 20 minutes for questions. I feel strongly that the audience should drive a good chunk of the talk through their questions. This feels much better to me than running over time and not allowing enough time for questions.

21 Second Lightning Talk

Lightning talks are a fun part of LCA. These usually last 5 minutes. I had an idea for a lightning talk on my Electric Car that I wanted to try. I figured I could get my talk done in 10-15 seconds. Yes, I was experimenting with presentation styles again. I wanted to use lots of slides connected with just a few words (normally we are encouraged to do it the other way around). This year I managed to get a lightning talk slot and presented the talk. You can see it on the lightning talk video starting at 50:20. From when I start to when I stop talking is 21 seconds, not quite the sub-15 seconds I was aiming at. I always was a bit talkative. However the applause was pretty loud so I think the idea worked!

Codec 2 Hacking

While at LCA Jean-Marc did some great LSP vector quantiser work for Codec 2 and explained some of the techniques involved. This was very useful, and will be part of Codec 2 soon. Thanks Jean-Marc!

At the end of the conference Bruce Perens (left), Timothy and Jean-Marc (right), came to stay for a few days at Nuria’s (centre) house. It was really nice to have them all, we did some good work on Codec 2, and the dinner-time conversation (fuelled by Nuria’s fine lasagne and BBQ) was fascinating. As Bruce pointed out, there is great value in the small number of open source speech coding guys meeting face to face.

I was a bit nervous travelling in the same car with Jean-Marc and Timothy. People working on open source voice codecs are rare – so we figured we had 60% of the world’s open source codec guys in one car!

pylint and hiding of attributes

I recently came across the pylint error:

E:  3,4:Foo.foo: An attribute affected in foo line 12 hide this method

in code that boiled down to essentially:

class Foo:

    def foo(self):
        return True

    def foo_override(self):
        return False

    def __init__(self, override=False):
        if override:
            self.foo = self.foo_override

Unfortunately that message isn't particularly helpful in figuring out what's going on. I still can't claim to be 100% sure what the message is intended to convey, but I can construct something that maybe it's talking about.

Consider the following using the above class

foo = Foo()
moo = Foo(override=True)

print "expect True  : %s" % foo.foo()
print "expect False : %s" % moo.foo()
print "expect True  : %s" % Foo.foo(foo)
print "expect False : %s" % Foo.foo(moo)

which gives output of:

$ python ./foo.py 
expect True  : True
expect False : False
expect True  : True
expect False : True

Now, if you read just about any Python tutorial, it will say something along the lines of:

... the special thing about methods is that the object is passed as the first argument of the function. In our example, the call x.f() is exactly equivalent to MyClass.f(x). In general, calling a method with a list of n arguments is equivalent to calling the corresponding function with an argument list that is created by inserting the method’s object before the first argument. [Official Python Tutorial]

The official tutorial above is careful to say in general; others often don't.

The important point to remember is how python internally resolves attribute references as described by the data model. The moo.foo() call is really moo.__dict__["foo"](moo); examining the __dict__ for the moo object we can see that foo has been re-assigned:

>>> print moo.__dict__
{'foo': <bound method Foo.foo_override of <__main__.Foo instance at 0xb72838ac>>}

Our Foo.foo(moo) call is really Foo.__dict__["foo"](moo) -- the fact that we reassigned foo in moo is never noticed. If we were to do something like Foo.foo = Foo.foo_override we would modify the class __dict__, but that doesn't give us the original semantics.

So I postulate that the main point of this warning is to suggest to you that you're creating an instance that now behaves differently to its class. Because the symmetry of calling an instance and calling a class is well understood you might end up getting some strange behaviour, especially if you start with heavy-duty introspection of classes.

Thinking about various hacks and ways to re-write this construct is kind of interesting. I think I might have found a hook for a decent interview question :)

Walktime Blog #18: Interview with Marco Ostini at LCA2012

Despite Australia's strong involvement in the early history of space exploration, we're now lagging way behind many far smaller nations in our commitment to the industry. At linux.conf.au 2009 in Hobart, Marco Ostini gathered together a group of like-minded people to create Lunar Numbat: a project to develop Open Source space technology as part of the White Label Space team competing for the Google Lunar X-Prize. At linux.conf.au 2012 in Ballarat I managed to pin him down long enough to film this interview.

View or comment directly on YouTube: youtu.be/6PQ6mIEmKfc

Links for this ep:

 * Marco Ostini: www.twitter.com/marcoostini

 * Lunar Numbat: www.lunarnumbat.org

 * White Label Space: www.whitelabelspace.com

 * Google Lunar X-Prize: www.googlelunarxprize.com

Walktime Blog #17: Interview with Matt Evans at LCA2012

At linux.conf.au 2012 I had the good fortune of meeting up with old friends and also making new ones, including the amazingly talented Matt Evans. Matt did a great talk called "Hack Everything: Re-Purposing Everyday Devices", so I caught up with him briefly afterwards to talk to him about it.

View or comment directly on YouTube: www.youtube.com/watch?v=BIQl0u-YZ5s

January 26, 2012

Sigh, so that happened

So two things happened today.

The first was a near riot that necessitated the evacuation of the Prime Minister and the Opposition leader from an awards ceremony meant to highlight the work of emergency workers.

The second was WIN Television blowing it again.

If you haven't caught up on the events today then I heres a brief timeline I've put together from various sources:

  • Earlier today in Sydney, Tony Abbott made comments indicating that he thought it was "time to move on" from the thinking that led to the Aboroginal Tent Embassy being set up forty years ago.
  • Tony Abbott travels to Canberra to attend the awarding of the inaugeral National Emergency Medal along with the Prime Minister.
  • People at the Tent Embassy hear of Tony Abbotts comments and are, it could be said, greatly miffed. They are then told that he is actually in Canberra, not far from the Tent Embassy itself.
  • A number of people decide that they're going express their anger at the Opposition Leader and Prime Minister. They surround the restuaraunt, a glass walled building and start yelling and slamming on the glass.
  • Police are called and the Prime Ministers security detail advises her that the situation is getting dangerous and that she should leave. The PM agrees and also indicates that they should escort the Opposition Leader as well, as he doesn't have his own security detail.
  • The PM and Opposition leader are rushed out of the building surrounded by the biggest scrum of security personel. The PM trips on the way out and it appears the opposition leader is helped along by the security personel via the method of grabbing his belt and pushing.

That's it in a nutshell I think. I've probably missed something but that's what I've managed to pull out of the media.

Needless to say as I watched the tweets fly by with all sorts of claims and details (the original tweets I had seen claimed that the PM had been crash tackled by a protester), I waited for any sign that the MSM had picked up the story. It didn't take long for the papers to run with it, and the TV networks at least squeezing in an extra news bulletin to cover the basic details.

Well, most of the networks.

WIN Television's Wollongong station couldn't even be bothered following channel 9 by putting on an extra news bulletin after Wriddhiman Saha lost his wicket. Instead WIN simply continued on with it's pre-planned ads and it wasn't until it broadcast the 9 news bulletin at 6pm that the news finally reached the airwaves.

I basically have three things to say:

To Tony Abbott, on this day of days, a day which a large part of the indigenous community views as the day they started losing their rights, to suggest that perhaps the Tent Embassy wasn't needed was either a dog whistle of epic proportions, or simply a case of shoving both feet in at the speed of sound.

Oh and as for the snide little raised eye brows and snark when Gillard offered to remove you from the situation, that just makes you look like a petulant little child.

To the protesters, you did yourself and your cause no favours today. You managed to rise to the bait and make yourselves look like a rabble. And getting cute by offering to give the PM back her shoe tomorrow if she comes by the Embassy just really doesn't help.

Finally, to WIN Television. Convergance is coming, and it's not going to be nice to those who aren't ready for it. You've known about this for years now, unless you start doing something soon, you will be left behind.

Anyway, that was  my Australia day, how was yours?

Blog Catagories: 

Shadow of a Dark Queen







ISBN: 9780006480266

LibraryThing

I read this book before LCA 2012, but never had a chance to mention it here. It was the first return to Midkemia for me since I read the Krondor's Sons books. This book is set a lot later, and there is very little reuse of characters between The Riftwar Saga or even Krondor's Sons. The only real overlap is the presence of Pug briefly. This book over does its "dirty dozen" aspects, with much of the book focusing on the military training of criminals. The rest of the book feels like a rushed military adventure in a far land, and could have done with some more attention. However, the book isn't terrible, and I thought it was ok overall.



Tags for this post: book raymond_e_feist midkemia combat crime fantasy sword_and_sorcery

Related posts: Daughter of the Empire; The King's Buccaneer; Servant of the Empire; The Riftwar Series; Silverthorn; A Darkness at Sethanon; Mistress of the Empire; Prince of the Blood; Magician: Master; Magician: Apprentice; Raymond E Feist's Empire Trilogy; Polar City Blues; The Stainless Steel Rat Sings The Blues; Dawnspell: The Bristling Wood; Caves of Steel; Currency; Tipping point: windscreen washers; You Can Be The Stainless Steel Rat; Belgarath the Sorcerer; The Adventures of the Stainless Steel Rat; Mona Lisa Overdrive
Comment Recommend a book

QotD: Jon Corbet on linux.conf.au and Linux Australia

In summary, LCA remains unique in its combination of strongly technical talks, freedom-oriented and hands-on orientation, wide variety of topics covered, and infectious Australian humor. There is a reason some of us seem to end up there every year despite the painful air-travel experiences required. Linux Australia has put together a structure that allows the conference to be handed off to a new team in a new city every year, bringing a fresh view while upholding the standards set in the previous years.

– LWN’s Jon Corbet on linux.conf.au, An LCA 2012 Summary

DeVeDe on Perfect Pangolin

This currently seems a little confused about the version numbering of the libblah-extra-number packages, so...

apt-get --ignore-hold install devede libavcodec-extra-53 libavdevice-extra-53 libavfilter-extra-2 libavutil-extra-51 libavformat-extra-53 libpostproc-extra-52 libswscale-extra-2

apt-get download libav-tools

apt-get download ffmpeg

apt-get download devede

dpkg -i --force-depends-version libav-tools_0.8-1ubuntu1_i386.deb

dpkg -i --force-depends-version ffmpeg_0.8-1ubuntu1_all.deb

dpkg -i --force-depends-version devede_3.21.0-0ubuntu2_all.deb

Congratulations Colin Benjamin - and Thank you

I have ambivalence about Australia Day - and the often unquestioned celebration of an invasion of Australia at the expense of the indigenous nations who made this continent their home. But Australia Day has also become an opportunity to celebrate the contributions of today's citizens.

I'm excited and honoured to see my dad, Colin Benjamin, in today's honours lists "for service to the community through roles with social welfare organisations, and to business". He follows in the footsteps of my grandfather, Eric Benjamin, who was awarded the same medal in 1988 "in recognition of service to social welfare".

Ruth and Eric, my grandparents, were refugees. They came to Australia with assisted passage aboard the TSS Jervis Bay, escaping Europe and the rise of fascism before the start of World War II. They were incredibly lucky. Most other members of their families did not survive the war. Eric's mother and sister ended up in Sobibor, his father ended up in the Polish Ghetto at Lodz.

Ruth was arrested and spent months in solitary confinement, before being transferred to the Moringen concentration camp. She was released and fled to the Netherlands, which is where she met Eric. They married and headed to Australia, and were watched by our security agencies. I don't know what happened to the rest of her family in Europe, except her brother survived, only to die in the spanish civil war. At university I had the opportunity to create a short video documentary about her.

Service to the community. Seem such simple, innocuous words. When we give safe harbour to people fleeing persecution and war, we give them a chance to make a difference to their new world. Our new world, Australia, has benefited from the efforts of these people. So while we must acknowledge the wrong done to the indigenous peoples of the great southern continent, we should also speak out against those who seek to demonise refugees and asylum seekers - who also come on boats, as my grandparents did.

Dad sometimes talks about "degrees of freedom" and the role entrepreneurship and business plays in people having control over their lives. When society has a systemic bias against you and your circumstances, or institutionalises barriers for your personal advancement and freedom, then creating your own work is crucial.

Thanks Dad!

Here's the citation.

MEDAL (OAM) OF THE ORDER OF AUSTRALIA IN THE GENERAL DIVISION

Dr COLIN BENJAMIN

For service to the community through roles with social welfare organisations, and to business.

Director-General, ‘Life. Be in it’ International, since 1993; Director, from 1983.

Co-Chairman, ‘Life. Be in it’ Australia Ltd, since 2007.

Director, ‘Life. Be in it’ Health Promotion Institute, 2011.

Director, Victoria Branch, Australian Association of Social Workers, since 2011.

Founder and Chairman, Australian Register of Counsellors and Psychotherapists, 2009-2010.

Former Chief Executive Officer, Psychotherapy and Counselling Federation of Australia, 2007-2010.

Member, Association of Professional Futurists.

Federal President, Australian Social Welfare Union, 1976.

Federal President, Australian Association of Social Workers, 1975.

Executive Director, Victorian Council of Social Service, 1973-1975.

Director of Education and Welfare, City of Sunshine, 1969-1972.

Senior Social Worker, Sunbury Mental Hospital, 1966-1969.

Deputy Director-General, Department of Employment and Training, Victoria, 1980-1982.

Director of Regional Services, Department of Community Welfare Services, 1977-1979.

Director of Research and Social Policy, Department of Social Welfare, 1975-1976.

Chairman, Marshall Place Associates, since 2004.

Chief Executive Officer, The Horizons Network, since 1990.

Director, Futures Division, Ogilvy and Mather Worldwide, 1983-1989.

Links January 2012

Cops in Tennessee routinely steal cash from citizens [1]. They are ordered to do so and in some cases their salary is paid from the cash that they take. So they have a good reason to imagine that any large sum of money is drug money and take it.

David Frum wrote an insightful article for NY Mag about the problems with the US Republican Party [2].

TreeHugger.com has an interesting article about eco-friendly features on some modern cruise ships [3].

Dan Walsh describes how to get the RSA SecureID PAM module working on a SE Linux system [4]. It’s interesting that RSA was telling everyone to turn off SE Linux and shipping a program that was falsely marked as needing an executable stack and which uses netstat instead of /dev/urandom for entropy. Really the only way RSA could do worse could be to fall victim to an Advanced Persistent Attack… :-#

The Long Now has an interesting summary of a presentation about archive.org [5]. I never realised the range of things that archive.org stores, I will have to explore that if I find some spare time!

Jonah Lehrer wrote a detailed and informative article about the way that American high school students receive head injuries playing football[6]. He suggests that it might eventually be the end of the game as we know it.

François Marier wrote an informative article about optimising PNG files [7], optipng is apparently the best option at the moment but it doesn’t do everything you might want.

Helen Keeble wrote an interesting review of Twilight [8]. The most noteworthy thing about it IMHO is that she tries to understand teenage girls who like the books and movies. Trying to understand young people is quite rare.

Jon Masters wrote a critique of the concept of citizen journalism and described how he has two subscriptions to the NYT as a way of donating to support quality journalism [9]. The only comment on his post indicates a desire for biased news (such as Fox) which shows the reason why most US media is failing at journalism.

Luis von Ahn gave an interesting TED talk about crowd-sourced translation [10]. He starts by describing CAPTCHAs and the way that his company ReCAPTCHA provides the CAPTCHA service while also using people’s time to digitise books. Then he describes his online translation service and language education system DuoLingo which allows people to learn a second language for free while translating text between languages [11]. One of the benefits of this is that people don’t have to pay to learn a new language and thus poor people can learn other languages – great for people in developing countries that want to learn first-world languages! DuoLingo is in a beta phase at the moment but they are taking some volunteers.

Cory Doctorow wrote an insightful article for the Publishers Weekly titles “Copyrights vs Human Rights” [12] which is primarily about SOPA.

Naomi Wolf wrote an insightful article for The Guardian about the “Occupy” movement, among other things the highest levels of the US government are using the DHS as part of the crackdown [13]. Naomi’s claim is that the right-wing and government attacks on the Occupy movement are due to the fact that they want to reform the political process and prevent corruption.

John Bohannon gave an interesting and entertaining TED talk about using dance as part of a presentation [14]. He gave an example of using dancerts to illustrate some concepts related to physics and then spoke about the waste of PowerPoint.

Joe Sabia gave an amusing and inspiring TED talk about the technology of storytelling [15]. He gave the presentation with live actions on his iPad to match his words, a difficult task to perform successfully.

Thomas Koch wrote an informative post about some of the issues related to binary distribution of software [16]. I think the problem is evenm worse than Thomas describes.

Related posts:

  1. Links January 2011 Halla Tomasdottir gave an interesting TED talk about her financial...
  2. Links January 2010 Magnus Larsson gave an interesting TED talk about using bacteria...
  3. Links January 2009 Jennifer 8 Lee gave an interesting TED talk about the...

Amateurish Astronomy

... in which I log my astronomickal adventures around linux.conf.au in Ballaarat. Verily.

Monday 16 January 2012

On monday evening I was invited to a property about an hour north of Ballarat to do some observing from a hill-top in an area without (much) light pollution. The evening was perfect and much was to be seen even without any optical augmentation. M45, the milky way, the coal sack & magellanic clouds were perfectly visible with the naked eye. Sadly the southern cross was pretty low on the horizon.

I'm happy to say the winning entry in the LCA photo competition was taken at this place :-)

Photo by David Basden

Photo by David Basden, showing the milky way, coal sack and small magellanic cloud.

I took a series of photos of the Orion nebula, but haven't had the time to try some of the photo stacking apps with them yet.

Thursday 19 January 2012

Weather and drinkingsocial engagements didn't really allow for much in the way of observation until thursday night, when I dragged my scope out onto the cricket field on campus. It was pretty late, so we didn't managed to see Venus before it disappeared behind the treeline.

Conditions were less than ideal, with flood lights on the other end of the field messing with night vision and a cold southwestern blowing in cloud and causing dew. I got a cold. (Achievement unlocked!)

Photo by David Basden

Photo by David Basden.

Still, Jupiter and the Orion nebula were blingy enough to be able to see through my scope and they duly impressed some of the gathered. One other person (self confessed noob, never used scope) brought a 6" motorised schmidt-cassegrain scope and we managed to successfully align it via Jupiter, Betelgeuse and Sirius then and found various things to look at through holes in the cloud cover.

We played a bit with the various eyepiece sizes when looking at Jupiter, but cloud made the image hazy-ish regardless.  I am now in envy of a motorised scope :-)  We also spotted about 5 sattelites zooming along, one of which passed directly in front of Jupiter and was observed through both scopes simultaneously, so I'll call that a confirmed sighting!

I became a lot more adept at aligning the scope and generally managed to get what it was I wanted to look at into view within a few minutes or so. Except the horse head nebula, that is. I could not for the life of me even find Alnitak that evening!

However, it turns out that with a Celestron 130 you can easily see Jupiter through a layer of cloud, though not in focus.

As we packed up and walked back, the cloud cover disappeared. (Achievement unlocked!)

Friday 20 January 2012

On Friday, an LCA organiser kindly offered me a reserved spot on the Ballarat Observatory tour, which I of course accepted. There was come scattered cloud when we arrived, but the observatory volunteers managed to get everyone to catch a glimpse of Jupiter and its moons through the Oddie and Federation telescopes.

As cloud rolled in (Achievement unlocked!) we were ushered inside for a few 3D videos, which worked about half the time for me but made me squint quite badly. They were a bit daggy for the LCA geek audience, but considering the goal of the observatory and the intended audience I think they're alright.

James Oddie telescope

Still, I didn't feel like squinting uncomfortably for an hour so I nicked off out the back and stood around outside for a little while, getting a short tour of the Jelbart telescope from one of the volunteers. As that concluded, I noticed the cloud cover had completely disappeared and everyone was still inside, watching the videos.

I snuck off to the Oddie telescope and had it to myself for a good 20 minutes. I now have giant 80" telescope envy as well :-)  I did some eyeballing of Jupiter and the Orion nebula and found that my own telescope is really badly in need of collimation. The Oddie, on the other hand was wonderfully sharp, and the bands on Jupiter were lovely and well-defined.

The other finished their video at that point and discovered the sky was clear, so they came up and had a sticky-beak around the sky, before being ushered back on the bus and driven back to the campus. I drove myself up, so hung around the observatory for a little while afterward, deflecting attempts by the volunteers to have me provide free IT support for Windows XP (Achievement unlocked!)

We had a quick look at the Rosette nebula before packing up the Oddie and closing the roof. On the way out, a member of the observatory beckoned us to have a look through his telescope, in which the two main components of α-Crucis were separately visible. Lovely :-)

What next?

There are a few things I would like to do to make observing with my own telescope a more enjoyable experience.

  • Collimate the telescope properly (well, need not want)
  • Obtain more varied eyepieces, as I currently only have two
  • Obtain some filters, as I have none
  • Obtain a motorised mount, to aid photographing faint objects
  • Obtain a higher magnification barlow lens for the camera adapter, but this is useless without the motorised mount or proper collimation
  • Obtain a spotting scope to aid aiming the telescope
  • Obtain a focuser with finer control

Advertising

If you're interested in amateur astronomy and don't want to go overboard on investing time and energy and obsessing over it, or use don't want to webforums to connect with others or use web forums at all, join the skyeballing mailing list.

On Bogans and Racism

So, last night I got into one of those twitter arguments that I tend to (I have a bad case of "Someones wrong on the Internet").

In this case it was triggered by the following tweet:

Redglitterx: how do you know if youre an elitist, smug, over-priviledged racist? you make bogan jokes

Now, if you're not sure what a bogan joke is, a bogan is a name for someone who is considered ill-educated, with low income and not only ignorant, but revelling in their ignorance. As you may have noticed, this is actually a stereotype, a cardboard cutout that people use to mentally slot a group of people into the social heirachy. In this way, calling someone a bogan is very much LIKE racism.

However I have never considered the term itself to be racist.

This may be one of those cases where a term is whatever anyone wants it to be, but I really think that it doesn't help anyone to try and conflate anti-boganism with racism. If nothing else, "bogans" don't face the same extreme issues as those who suffer from racism. Bogans are not likely to be physically attacked because they are bogans, there is not a continual low level campaign by large sections of the media to seperate them out from the rest of the population, and let's be honest, there is a hell of a lot of government policy that is aimed at placating and winning the votes of those who might be called bogan.

As I said at the beginning, calling someone a bogan is like racism in that you are using a stereotype to mentally sort that person, just like calling someone an eastern sububs nancy boy, or a north shore soccer mum.

January 25, 2012

Twitter Weekly Updates for 2012-01-25

LeoStick with seven-segment display

In my LCA2012 schwag bag I got a LeoStick from Freetronics.

I’m hopelessly bad at electronics, but I managed to wire it up to a seven-segment LED display from an electronics kit I used to use as a kid.

Using just a few lines of C++ (or whatever the heck the language is that Arduino uses — I haven’t fully worked it out ;-) ) results in this video (and scroll down for the code):

Watch video

/*
LED segment mappings
Note that this is specific to the way I wired it
This will be different for you!
 
Top          = 0x01
Top left     = 0x02
Bottom left  = 0x04
Bottom       = 0x08
Top right    = 0x10
Middle       = 0x20
Bottom right = 0x40
Dot          = 0x80
*/
 
/* here we combine the above segments to produce a complete digit */
unsigned char digits[] = {
  0x5F, /* zero  */
  0x50, /* one   */
  0x3D, /* two   */
  0x79, /* three */
  0x72, /* four  */
  0x6B, /* five  */
  0x6F, /* six   */
  0x51, /* seven */
  0x7F, /* eight */
  0x7B  /* nine  */
};
 
void display_digit(unsigned char d)
{
  int i;
 
  /* examine each bit and determine if segment needs to be lit */
  for (i=0; i<8; i++) {
    digitalWrite(i, (d >> i & 0x01 == 0x01) ? HIGH : LOW); 
  }
}
 
void setup()
{
  int i;
 
  /* set pins 0-7 to output */
  for (i=0; i<8; i++) {
    pinMode(i, OUTPUT);
  }
 
  /* show all segments on bootup */
  display_digit(0xFF);
  delay(1000);
}
 
void loop()
{
  int i;
 
  /* display each digit one by one */
  for (i=0; i<10; i++) {
    display_digit(digits[i]);
    delay(500);
  }
}

Any tips on optimising the above code? Being fairly crap at C–like languages, and with no knowledge of the ATmega32U4 processor, I have no idea how efficient the bit shifting is (in the display_digit() function).

SE Linux Status in Debian 2012-01

Since my last SE Linux in Debian status report [1] there have been some significant changes.

Policy

Last year I reported that the policy wasn’t very usable, on the 18th of January I uploaded version 2:2.20110726-2 of the policy packages that fixes many bugs. The policy should now be usable by most people for desktop operations and as a server. Part of the delay was that I wanted to include support for systemd, but as my work on systemd proceeded slowly and others didn’t contribute policy I could use I gave up and just released it. Systemd is still a priority for me and I plan to use it on all my systems when Wheezy is released.

Kernel

Some time between Debian kernel 3.0.0-2 and 3.1.0-1 support for an upstream change to the security module configuration was incorporated. Instead of using selinux=1 on the kernel command line to enable SE Linux support the kernel option is security=selinux. This change allows people to boot with security=tomoyo or security=apparmor if they wish. No support for Smack though.

As the kernel silently ignores command line parameters that it doesn’t understand so there is no harm in having both selinux=1 and security=selinux on both older and newer kernels. So version 0.5.0 of selinux-basics now adds both kernel command-line options to GRUB configuration when selinux-activate is run. Also when the package is upgraded it will search for selinux=1 in the GRUB configuration and if it’s there it will add security=selinux. This will give users the functionality that they expect, systems which have SE Linux activated will keep running SE Linux after a kernel upgrade or downgrade! Prior to updating selinux-basics systems running Debian/Unstable won’t work with SE Linux.

As an aside the postinst file for selinux-basics was last changed in 2006 (thanks Erich Schubert). This package is part of the new design of SE Linux in Debian and some bits of it haven’t needed to be changed for 6 years! SE Linux isn’t a new thing, it’s been in production for a long time.

Audit

While the audit daemon isn’t strictly a part of SE Linux (each can be used without the other) it seems that most of the time they are used together (in Debian at least). I have prepared a NMU of the new upstream version of audit and uploaded it to delayed/7. I want to get everything related to SE Linux up to date or at least with comparable versions to Fedora. Also I sent some of the Debian patches for the auditd upstream which should reduce the maintenance effort in future.

Libraries

There have been some NMUs of libraries that are part of SE Linux. Due to a combination of having confidence in the people doing the NMUs and not having much spare time I have let them go through without review. I’m sure that I will notice soon enough if they don’t work, my test systems exercise enough SE Linux functionality that it would be difficult to break things without me noticing.

Play Machine

I am now preparing a new SE Linux “Play Machine” running Debian/Unstable. I wore my Play Machine shirt at LCA so I’ve got to get one going again soon. This is a good exercise of the strict features of SE Linux policy, I’ve found some bugs which need to be fixed. Running Play Machines really helps improve the overall quality of SE Linux.

Related posts:

  1. Status of SE Linux in Debian LCA 2009 This morning I gave a talk at the Security mini-conf...
  2. SE Linux in Debian I have now got a Debian Xen domU running the...
  3. Debian SE Linux Status At the moment I’ve got more time to work on...

Are you in a LUG? Do you want some promotional materials for LCA 2013?

Canberra was announced as the host for LCA 2013 at the close of LCA 2012. As part of that closing, we handed out postcards and laptop stickers to delegates. However, we deliberately had extra printed on the theory that groups like LUGs, university computer societies and so forth would be interested in having promotional materials for their groups. For those of you not lucky enough to attend the excellent LCA2012, the stickers looked like this:







And the postcards look like this:







All credit for the excellent art should go to the very capable Jenny Cox. So, if you're interested in having some stuff to hand out at your next LUG or computer society meeting, please drop us a line at contact@lca2013.linux.org.au. Don't forget to include the name of the group and a mailing address.



Tags for this post: conference lca2013 canberra promotion postcard sticker

Related posts: Scoble, I'll buy the damn book, just put your clothes back on; Two more weeks to go; In Canberra; Mont 24 hour race; Most novel traffic jam cause goes to... Canberra!; So, what on earth was I doing up at 4:30 am anyways?; What are we doing with the pets?; Electric shadows has a RSS feed!; Travel details so far; Frank Arrigo discovers Steve Walsh's free wireless; On a bunker kick; linux.conf.au Returns to Canberra in 2013; Calling Tate Needham, or, Hiring in Canberra; LCA weather; Back in Canberra again



Comment

Moving on – the journey continues

Today is a complicated day. I’m both sad and excited in equal measure about what this year may hold in store.

I’m sad because this is my last day working for Senator Kate Lundy as her IT Policy Advisor and inhouse geek. Kate headhunted me almost 3 years ago at BarCamp Canberra, though we had known each other for a few years beforehand from when she was the Shadow Minister for IT. I was quite wary of going to work in a political office, but my curiosity about how the machine works combined with a desire to help make good tech policy and an immense amount of respect for Kate brought me into one of the most interesting, fun and challenging jobs I’ve ever had.

I particularly wanted to better understand the legislative and executive arms of government. How ideas turn into policy and policy into implementation. As a result, along with doing my job I’ve spent time researching the history of democracy, of Australian politics, of the ideological and historical premise of all the major Australian parties and the interaction between party politics and democracy over the years.  I’ve also spent time coming to understand some of the layout, responsibilities and challenges of a multi-tiered system of government.

I have learnt a great deal in this job about government, but also about human nature. Working in an electoral office gives one some insight to the difficulties faced by many, but also some insight to the challenge in maintaining a constructive and respectful dialogue. I think it is human nature to try to boil issues down to black and white. But we are essentially grey creatures with enormous complexity, and I think democracy is about finding ways to have a transparent, informed, respectful and constructive dialogue with all the people on complex policies and implementation, so governments can best implement the best policies for the communities they serve.

I have been lucky to work for a politician who is passionate and knowledgeable about technology and good policy. She has been a valuable teacher and mentor. I shall always be thankful for the wisdom, patience, compassion, critical thinking, strategy and policy development I have learnt in this role and from Kate. I’m sure these skills will continue to serve me well.

My work on Kate’s website, the Public Spheres, Open Government, assisting Kate in linking together different tech policies across a variety of portfolios are all things I am proud of. I also feel very lucky to have met and worked with such inspirational people from many different walks of life through this role and in Kate’s office.

Meanwhile, having developed some understanding of the legislative and executive arms of government, I realised that I wanted to have more experience in the administrative arm of government. I had done some tech work in a previous life within departments but always as the outsourced person. I knew I wanted to really get in and contribute to the public service, as well as learn more about the implementation of policy and the delivery of government services to citizens.

As such, I’m excited to say I am hopefully moving into a role in the APS in the coming weeks and I hope my efforts there will be broadly useful to others in the APS. I can’t say more at this stage as it is being finalised at the moment, but I’ll update this post in the weeks ahead with more information.

By working within the APS, I hope to get a better personal understanding of the specific challenges facing the APS with regards to technology, and hopefully assist in developing strategies to be a more agile, responsive and citizen-centric public service. I will also continue helping to move the Open Government agenda ahead both in my own time and, where appropriate, within my new role. My commitment to Open Government (and Gov 2.0) lies in my understanding that it provides a path to a public service and democracy that is most relevant to, engaged with, responsive to, representative of and accountable to its citizens.

I’ll finish by saying that after three years in her office, my respect for Kate has only grown. She is a person who has engaged fully in her role with integrity, responsibility, grace and a firm grip on her own principles. She is a politician that makes me believe politics isn’t just a dirty word and I wish we had more like her. Even in spite of the fact the last time I socialised with her, I ended up with a fractured scaphoid! I have learnt a keen respect for the torque of a 2 stroke, especially on a motocross track.

My shiny black carbon fibre cast. Shiny!My shiny black carbon fibre cast. Shiny!

So, I’m diving into the deep end and I look forward to seeing how well I swim. Wish me luck :)

TextSecure – secure SMS for Android

So… having secure SMS really isn’t hard. Onec upon a time you may have been forgiven to think that your SMS messages weren’t recorded forever by telecommunications companies or various government agencies, but those times have long passed. At the very least you should be concerned about somebody getting hold of your phone and going through all your SMSs (phones no longer just store 20 messages).

TextSecure (Free and Open Source Software up on github) does both local encryption (messages are encrypted on your phone) and over the wire encryption. That’s right kids – you can send encrypted text messages to each other.

It’s a drop-in replacement for the built in Android text messages application, so it all “just works”.

Go install it now.

This is the app that Jacob Appelbaum mentioned in is Keynote at lca2012.

January 24, 2012

LCA2012: “Android is Not vi – User Experience for Geeks”

Paris Buttfield-Addison and I co-presented a talk at Linux.conf.au in Ballarat recently. The topic was on designing mobile apps that don’t suck on Android. The talk was pretty well received, the audience attentive and engaged (as evidenced by the fact that they heckled), and it was probably one of the better talks that Paris and I have co-presented.

The video of the talk is available as an ogv movie file, alternatively, the YouTube version is embedded below.

sadlittlewebjournal 3.0 released

sadlittlewebjournal, the Perl-based software that powers this site (and a few others) celebrates is 10th anniversary this year! To celebrate, I'm finally bumping the major version up to 3.0, with a swag of new features...

A Week or so with the Samsung Galaxy Nexus

After a couple of good years with my Nokia N900 I’ve come to the sad conclusion that there’s no future for that platform due to the combined actions of Nokia and Intel – Nokia for dumping Linux and going with Windows Mobile for their smart phones after getting a new CEO (ex-Microsoft) and then Intel through dumping Meego and setting up a partnership with Samsung for yet another mobile Linux platform called Tizen (which at least went for the code first, hype second path, unlike Meego). Intel are now on their third mobile Linux project as there was their Moblin project which was merged with Nokia’s Maemo to form Meego (announced less than 2 years ago) so they have form here as a serial abandoner.

Looking at what is left in the mobile space it was really a no-brainer as neither Windows Mobile nor Apple’s iOS appealed at all, so it had to be an Android phone. The timing was pretty good as Samsung and Google had just started shipping their jointly designed Galaxy Nexus with Android 4 (aka Ice Cream Sandwich or ICS). It has the advantage of apparently being a phone recommended for the AOSP (Android Open Source Program) should I feel the need once my warranty expires – though I can’t find a reference to that now! I ordered an unlocked Galaxy Nexus with 2 year warranty from Mobicity as I didn’t fancy the rubbish that carriers tend to put onto their phones, nor get handcuffed into a contract I didn’t want. As an added bonus Mobicity let you pick from 3 optional accessories for free – I picked the screen protector (the other were either a charger or a bluetooth headset from memory).

As an amusing aside I did try and see if Dick Smith Electronics would price match with Kogan for the Galaxy Nexus as Kogan was far cheaper and DSE was only selling them online, but with a manufacturers warranty (unlike Mobicity or Kogan). Unfortunately DSE declined to do so on the grounds that Kogan didn’t have a physical retail outlet which was a bit rich given that neither does DSE for these phones. But then I found out they are now owned by Woolworths and so I didn’t fancy supporting the largest owners of poker machines in Australia.

Despite the best efforts of UPS (who said it would take 6 days to cross Melbourne having taken 24 hours from Hong Kong – it actually arrived the following morning) I received it intact and on time.

Samsung / Google Galaxy Nexus

I’ve now been playing with it, er, using it in anger for over a week now and so far I’m very happy. I’d have to say the best description of the overall experience is “smooth”. Android 4 seems light years ahead of Android 2.3.3 on my wifes Huaewei Sonic, though part of that will be the fact that it’s just a much more capable phone with a larger screen and much more powerful processor.

Good bits:

  • Auto-language select – it started up in Chinese characters but before I could really wonder how I’d fix that it detected it had an Australian SIM in it and autoconfigured the locale to match.
  • No extra cruft – I’ve not spotted any “extras” from Samsung on the phone – the Market is the standard Android Market, etc.
  • Good size screen – the phone feels much smaller in the pocket than my old N900 due to its narrowness despite it having a much wider screen.
  • Android Market – heaps of apps, though the usual criticism of it not being easy to search for open source applications applies here.
  • Camera – it’s “only” 5 megapixels, but it’s still pretty good (though I’ve not yet figured out how to turn the flash off).
  • NFC – OK, a little bit of a toy at the moment, but there are a couple of apps that will read it and confirm that the reason my Myki and Uni ID card interfere is that they’re the same type of technology and so interfere with each other. As do my credit card and my bank card (same tech again).
  • Compass – my N900 had GPS and accelerometers (as does the Galaxy Nexus of course) but the compass allows neat things like Google Sky where you can just point your phone at the sky and have it show you a labelled view of planets stars and constellations.
  • IPv6 works on Wifi – I know people say IPv6 has worked on Wifi since Android 2.2, but it certainly doesn’t on my wifes Android 2.3 phone. But the Galaxy Nexus seems quite happy on my home network with native dual stack IPv6 courtesy of Internode.

Of course nothing is ever perfect, so here’s my feelings on the bad bits:

  • No real keyboard – I really miss the N900′s physical keyboard, it made typing easy. The on-screen keyboard that Android has is good, and quite usable for SMS, Twitter, etc, but for things like the Connectbot SSH client you can’t beat a real keyboard
  • No NTP synchronisation possible – you can get root on the phone (and void your warranty) but this *really* shouldn’t be necessary!
  • NITZ sucks – whilst it gets the time right the timezone is out by an hour. Probably a carrier issue but I don’t think phones should be relying on it. Had to set it by hand to fix it up.
  • Short notification sounds – a minor nit but the default notification sounds that are used for things like SMS, etc, are really short and quite easy to miss.
  • Not entirely open source – whilst the N900 wasn’t either it does seem to have been more open than Android, and it didn’t try and avoid GPL code at all costs like Android does.
  • No update to Android 4.0.2 available (yet) – so far it appears that Samsung haven’t pushed an Android 4.0.2 update to the region my phone was intended for – though other Galaxy Nexus owners around the world have reported getting updates at other times (including someone at Mobicity where I bought it). I suspect this is just an organisational delay and nothing more serious, but it is annoying. If it wasn’t for the warranty issue I’d consider reflashing the phone with the stock Google firmware for the Galaxy Nexus and pick the updates up directly from them in future.

To finish it off here are three images taken with the camera in the Samsung Galaxy Nexus (as I said I was happy with it), the first one was used on the weather slot as a background by the ABC News people last week!

Melbourne summer morning Swanston St Skyline The Light Side and the Dark Side

This item originally posted here:



A Week or so with the Samsung Galaxy Nexus

An Extensible Puppet ENC with class & parameter overrides (& more)

Puppet (http://www.puppetlabs.com/)'s a great tool for system automation - out of the box it comes with most of the functionality that you'll ever need, and a DSL that's easy enough to let junior sysadmins and developers maintain their own recipes. But, depending on the kind of workhorse you want puppet to be, it's possible to hit a wall. For my company, we had too many developers, many system administrators, multiple releases in production simultaneously, and a change control to rule them all (but managed nothing). We also had the odd typo in node definitions causing incidents.

One possible answer to this is to employ an External Node Classifier, which in simple terms means an external script that Puppet calls to get the node definition, or at least the classes and parameters (and the environment, but don't rely on that due to an outstanding bug). An ENC cannot call definitions or puppet functions directly however - it can just pass in classes, class parameters, and global parameters. It's therefore up to you to make your ENC more intelligent.

I think Dan Bode of Puppet Labs gave me the idea of using YAML at first, over a year ago, and I whipped up a basic version in a couple of hours.

Over my long weekend, I decided to write a new ENC library from scratch that would provide the basics that one might want in an ENC, but allows it to be extended with plugins. You instantiate it for a node with some seed data, pass in a loader object, and then include one or more input/override files. It will take care of class, parameter, and substitution-variable overrides, and will spit out a nice Puppet 2.6+ ENC-compatible hash.

The test_puppetenc.rb file is the best source of documentation.

The Code: https://github.com/rmt/puppetenc-ruby

Private Stupidity

Kate and I heard a talk on the radio the other day in which a German-sounding American professor, or pundit, or something, tried to establish that actually we needed a society where everything is owned. Obviously it's hard for me to know his full thesis, because we started listening in the middle of the talk and switched off in disgust less than a minute later. But of the bit we heard all of it was, basically, complete rubbish.

His idea seemed to be that public property was a real problem - that it made things difficult because then you had to have laws and police and you had people sponging off the public good and abusing public property. Then you had to pay all these taxes to keep everything going and it was all very draining and stopped people just doing whatever they wanted with their stuff. Yes, a nice straw man argument, but then the alternative completely baffled me. According to him, in addition to owning your own property you'd also have to be a kind of shareholder in the road in front of your street, and the footpath, and the fences between your neighbours and yourself.

What baffles me here is that he clearly didn't see this going much further. Presumably he stays in his own street, grows all his own food, has an amazing naturally-occurring spring of fresh water in his back yard, and doesn't use electricity or the internet. Because as soon as you start looking at where all those things come from, you realise that they're all some kind of shared property. Once you drive outside your street, you need to be a shareholder in the company that owns that street, and so forth. I can only assume the pundit doesn't have any friends, because they moment they come and visit him they're going to have to pay a fee to his street-ownership-company to get there and park. He may well never use a public hospital, gone to a public school, gone to a public park, flown in public airspace, used the public radio spectrum, or have to claim unemployment benefits, but only because he's most probably a well-off white male.

I'll hopefully save my readers the tedium of reading through the first course in a standard lecture on Government and Democracy. It's just incredibly irritatingly bizarre to hear someone spout this kind of nonsense which almost naturally disproves itself. He probably even thinks the world will be a better place if they followed his philosophy. I'd like to invite him, publicly, to stop using all our public resources and only use the ones he actually privately owns. Then, when the oxygen starts running out in a couple of hours, he may like to reconsider. Meanwhile, get off my public broadcasting network and pay for your own publicity yourself.

Benchmarking and QuickChecking readInt.

I'm currently working on converting my http-proxy library from using the Data.Enumerator package to Data.Conduit (explanation of why in my last blog post).

During this conversion, I have been studying the sources of the Warp web server because my http-proxy was originally derived from the Enumerator version of Warp. While digging through the Warp code I found the following code (and comment) which is used to parse the number provided in the Content-Length field of a HTTP header:

  -- Note: This function produces garbage on invalid input. But serving an
  -- invalid content-length is a bad idea, mkay?
  readInt :: S.ByteString -> Integer
  readInt = S.foldl' (\x w -> x * 10 + fromIntegral w - 48) 0

The comment clearly states that that this function can produce garbage, specifically if the string contains anything other than ASCII digits. The comment is also correct that an invalid Content-Length is a bad idea. However, on seeing the above code, and remembering something I had seen recently in the standard library, I naively sent the Yesod project a patch replacing the above code with a version that uses the readDec function from the Numeric module:

  import Data.ByteString (ByteString)
  import qualified Data.ByteString.Char8 as B
  import qualified Numeric as N

  readInt :: ByteString -> Integer
  readInt s =
      case N.readDec (B.unpack s) of
          [] -> 0
          (x, _):_ -> x

About 3-4 hours after I submitted the patch I got an email from Michael Snoyman saying that parsing the Content-Length field is a hot spot for the performance of Warp and that I should benchmark it against the code I'm replacing to make sure there is no unacceptable performance penalty.

That's when I decided it was time to check out Bryan O'Sullivan's Criterion bench-marking library. A quick read of the docs and bit of messing around and I was able to prove to myself that using readDec was indeed much slower than the code I wanted to replace.

The initial disappointment of finding that a more correct implementation was significantly slower than the less correct version quickly turned to joy as I experimented with a couple of other implementations and eventually settled on this:

  import Data.ByteString (ByteString)
  import qualified Data.ByteString.Char8 as B
  import qualified Data.Char as C

  readIntTC :: Integral a => ByteString -> a
  readIntTC bs = fromIntegral
          $ B.foldl' (\i c -> i * 10 + C.digitToInt c) 0
          $ B.takeWhile C.isDigit bs

By using the Integral type class, this function converts the given ByteString to any integer type (ie any type belonging to the Integral type class). When used, this function will be specialized by the Haskell compiler at the call site to to produce code to read string values into Ints, Int64s or anything else that is a member of the Integral type class.

For a final sanity check I decided to use QuickCheck to make sure that the various versions of the generic function were correct for values of the type they returned. To do that I wrote a very simple QuickCheck property as follows:

  prop_read_show_idempotent :: Integral a => (ByteString -> a) -> a -> Bool
  prop_read_show_idempotent freader x =
      let posx = abs x
      in posx == freader (B.pack $ show posx)

This QuickCheck property takes the function under test freader and QuickCheck will then provide it values of the correct type. Since the function under test is designed to read Content-Length values which are always positive, we only test using the absolute value of the value randomly generated by QuickCheck.

The complete test program can be found on Github in this Gist and can be compiled and run as:

  ghc -Wall -O3 --make readInt.hs -o readInt && ./readInt

When run, the output of the program looks like this:

  Quickcheck tests.
  +++ OK, passed 100 tests.
  +++ OK, passed 100 tests.
  +++ OK, passed 100 tests.
  Criterion tests.
  warming up
  estimating clock resolution...
  mean is 3.109095 us (320001 iterations)
  found 27331 outliers among 319999 samples (8.5%)
    4477 (1.4%) low severe
    22854 (7.1%) high severe
  estimating cost of a clock call...
  mean is 719.4627 ns (22 iterations)

  benchmarking readIntOrig
  mean: 4.653041 us, lb 4.645949 us, ub 4.663823 us, ci 0.950
  std dev: 43.94805 ns, lb 31.52653 ns, ub 73.82125 ns, ci 0.950

  benchmarking readDec
  mean: 13.12692 us, lb 13.10881 us, ub 13.14411 us, ci 0.950
  std dev: 90.63362 ns, lb 77.52619 ns, ub 112.4304 ns, ci 0.950

  benchmarking readRaw
  mean: 591.8697 ns, lb 590.9466 ns, ub 594.1634 ns, ci 0.950
  std dev: 6.995869 ns, lb 3.557109 ns, ub 14.54708 ns, ci 0.950

  benchmarking readInt
  mean: 388.3835 ns, lb 387.9500 ns, ub 388.8342 ns, ci 0.950
  std dev: 2.261711 ns, lb 2.003214 ns, ub 2.585137 ns, ci 0.950

  benchmarking readInt64
  mean: 389.4380 ns, lb 388.9864 ns, ub 389.9312 ns, ci 0.950
  std dev: 2.399116 ns, lb 2.090363 ns, ub 2.865227 ns, ci 0.950

  benchmarking readInteger
  mean: 389.3450 ns, lb 388.8463 ns, ub 389.8626 ns, ci 0.950
  std dev: 2.599062 ns, lb 2.302428 ns, ub 2.963600 ns, ci 0.950

At the top of the output is proof that all three specializations of the generic function readIntTC satisfy the QuickCheck property. From the Criterion output its pretty obvious that the Numeric.readDec version is about 3 times slower that the original function. More importantly, all three version of this generic function are an order of magnitude faster than the original.

That's a win! I will be submitting my new function for inclusion in Warp.

Update : 14:13

At around the same time I submitted my latest version for readInt Vincent Hanquez posted a comment on the Github issue suggesting I look at the GHC MagicHash extension and pointed me to an example.

Sure enough, using the MagicHash technique resulted in something significantly faster again.

Render Single-Line Markdown Text with Redcarpet

At Icelab, we love Markdown. We use it wherever we can for text formatting. In a web app, the obvious place for it is in large text areas, where we can allow complete freedom of formatting. Headers, paragraphs, lists, it’s all good.

What about the formatting of text in single-line text fields? If our form entry is a single line, that’s usually how its text will be displayed in our interface. In this case, we probably want to avoid all the block-level elements that Markdown will let the user create.

This is easy to do using Redcarpet, a fantastic Markdown renderer for Ruby. It is fast and, importantly, it is modular: it allows us to define our own custom renderers. We can use this to create a renderer that ignores all the block-level Markdown elements. Put this somewhere in your Rails app (e.g., lib/redcarpet_renderers.rb):

module Redcarpet
  module Render
    class HTMLWithoutBlockElements < HTML
      include SmartyPants

      def initialize(opts = {})
        opts[:tables] = false
        super(opts)
      end

      # Regular markdown, just ignore all the block-level elements

      def block_code(code, language)
        code
      end

      def block_quote(quote)
        quote
      end

      def block_html(raw_html)
        raw_html
      end

      def header(text, header_level)
        "#{text} "
      end

      def hrule
        " "
      end

      def list(contents, list_type)
        " #{contents}"
      end

      def list_item(text, list_type)
        "* #{text}"
      end

      def paragraph(text)
        text
      end

      # Span-level calls

      def linebreak
        " "
      end

      # Postprocessing: strip the newlines

      def postprocess(document)
        document.gsub("\n", ' ').strip
      end
    end
  end
end

Now we can safely render the content from our single-line text fields using markdown. The user can still make things _emphasised_ or **bold** or even [linked](http://icelab.com.au/) and we don’t have to worry about unwanted block elements messing up our page layouts.

To use this renderer, just throw something like the following into your app/helpers/application_helper.rb file:

module ApplicationHelper
  def markdown(text)
    renderer = Redcarpet::Render::HTML.new({
      :filter_html => true,
      :hard_wrap => true
    })
    markdown = Redcarpet::Markdown.new(renderer, {
      :autolink => true,
      :no_intra_emphasis => true
    })

    markdown.render(text).html_safe
  end

  def markdown_line(text)
    renderer = Redcarpet::Render::HTMLWithoutBlockElements.new({
      :filter_html => true,
      :hard_wrap => true
    })
    markdown = Redcarpet::Markdown.new(renderer, {
      :autolink => true,
      :no_intra_emphasis => true
    })

    markdown.render(text).html_safe
  end
end

And it’s now a helper ready for use in our page templates. For example:

<div class="item">
  <h3 class="name"><%= markdown_line(item.name) %></h3>
  <div class="description">
    <%= markdown(item.description) %>
  </div>
</div>

The more Markdown we can use, the happier we are!

January 23, 2012

Flickr photos for LCA2012 now online

All my linux.conf.au 2012 photos have been uploaded to Flickr, within a week of the conference. (that's gotta be a record! :-)

MySQL and PostgreSQL Cloud Offerings – linux.conf.au 2012 miniconf talk by myself and Selena

Selena and I gave a talk on the various issues of running databases “in the cloud” at the recent linux.conf.au in Ballarat. Video is up, embedded below:

Identically partition disks.. the easy way!

Was just looking into a software RAID howto.. for no reason really, but kinda glad I did! When you set up software raid you want to make sure all disks are partitioned the same, right. so check this out:

3. Create partitions on /dev/sda identical to the partitions on /dev/sdb:

sfdisk -d /dev/sdb | sfdisk /dev/sda

That’s a much easier way ;)

This gem is thanks to: http://www.howtoforge.com/how-to-create-a-raid1-setup-on-an-existing-centos-redhat-6.0-system

NTLM Authentication in Squid using Winbind.

Some old windows servers require authentication through the old NTLM protocol, luckily with the help from squid, samba and winbind we can do this under Linux.

Some URLs a much of this information was gathered from are:

  • http://wiki.squid-cache.org/ConfigExamples/Authenticate/NtlmCentOS5
  • http://wiki.squid-cache.org/ConfigExamples/Authenticate/Ntlm

HOW TO

In order to authenticate through winbind we will be using that and samba to connect to a windows domain, so you will need to have a domain and the details for it or all this will be for naught. I’ll use some fake credentials for this post.

Required Packages

Let’s install all the required packages:



yum install squid krb5-workstation samba-common ntp samba-winbind authconfig

NTP (Network Time Protocol)

Kerberos and windbind can be a little thingy about date and time, so its a good idea to use NTP for your network, I’ll assume your domain controller (DC) will be also your NTP server in which case lets set it up.

Comment out any lines that begin with server and create only one that points to your Active Directory PDC.



# vim /etc/ntp.conf

server pdc.test.lan

Now add it to the default runlevels and start it.



chkconfig ntpd on

/etc/init.d/ntpd start

Samba, Winbind and Kerberos

We will the use the authconfig package/command we installed earlier to configure Samba, Winbind and perform the join in one step, this makes things _SO_ much

easier!!!

NOTE: If you don’t have DNS set up then you will need to add the DC to your hosts file, and it is important to use the name the DC machine knows itself as in AD.



authconfig --enableshadow --enablemd5 --passalgo=md5 --krb5kdc=pdc.test.lan \

--krb5realm=TEST.LAN --smbservers=pdc.test.lan --smbworkgroup=TESTLAN \

--enablewinbind --enablewinbindauth --smbsecurity=ads --smbrealm=TEST.LAN \

--smbidmapuid="16777216-33554431" --smbidmapgid="16777216-33554431" --winbindseparator="+" \

--winbindtemplateshell="/bin/false" --enablewinbindusedefaultdomain --disablewinbindoffline \

--winbindjoin=administrator --disablewins --disablecache --enablelocauthorize --updateall

NOTE: Replace pdc.test.lan with that of your FQDN of your DC server, TESTLAN with your domain, TEST.LAN with the full name of the domain/realm, and make sure you set ‘–winbindjoin’ with a domain admin.

If that succeeds lets test it:



# wbinfo -u

# wbinfo -g



If you are able to enumerate your Active Directory Groups and Users, everything is working.

Next lets test that we can authenticate with winbind:



# wbinfo -a



E.G:



# wbinfo -a testuser

Enter testuser's password:

plaintext password authentication succeeded

Enter testuser's password:

challenge/response password authentication succeeded

Great, we have been added to the domain, so now we can setup squid for NTLM authentication.

SQUID Configuration

Squid comes with its own ntlm authentication binary (/usr/lib64/squid/ntlm_smb_lm_auth) which uses winbind, but as of Samba 3.x, samba bundle their own which is the recommended binary to use (according to the squid and samba projects). So the binary we use comes from the samba-winbind package we installed earlier:



/usr/bin/ntlm_auth

Add the following configuration elements to the squid.conf to enable NTLM authentication:



#NTLM

auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp

auth_param ntlm children 5

auth_param ntlm keep_alive on

acl ntlm proxy_auth REQUIRED

http_access allow ntlm



NOTE: The above is allowing anyone access as long as they authenticate themselves via NTLM, you could use further acl's to restrict this more.

The ntlm_auth binary has other switches that might be of use, such as restricting users by group membership:



auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp --require-membership-of=EXAMPLE+ADGROUP

Before we are complete there is one more thing we need to do, for squid to be allowed to use winbind, the squid user (which was created when the squid package was installed) needs to be a member of a wbpriv group:



gpasswd -a squid wbpriv

IMPORTANT!

NTLM authentication WILL FAIL if you have "cache_effective_group squid" set, if you do then remove it! As this overrides the effective group and squid then isn't seen as part of the 'wbpriv' group which breaks authentication!!!

/IMPORTANT!

Add squid to the runlevels and start it:



# chkconfig squid on

# /etc/init.d/squid start

Trouble shooting

Make sure you open the port in iptables, if squid is listening on 3128 then:



# iptables -I INPUT 1 -p tcp --dport 3128 -j ACCEPT

# /etc/init.d/iptables save

NOTE: The '/etc/init.d/iptables save' command saves the current running configuration so the new rule will be applied on reboot.

Happy squid-ing.

Reverse proxy using squid + Redirection

Squid – Reverse Proxy

In computer networks, a reverse proxy is a type of proxy server that retrieves resources on behalf of a client from one or more servers. These resources are then returned to the client as though it originated from the reverse proxy itself. While a forward proxy is usually situated between the client application (such as a web browser) and the server(s) hosting the desired resources, a reverse proxy is usually situated closer to the server(s) and will only return a configured set of resources.

See: http://en.wikipedia.org/wiki/Reverse_proxy

Configuration

Squid should already be installed, if not then install it:



yum install squid

Then we edit squid config:



vim /etc/squid/squid.conf

Add we add the following to the top of the file:



http_port 80 vhost

https_port 443 cert=/etc/squid/localhost.crt key=/etc/squid/localhost.key vhost

cache_effective_user squid

cache_effective_group squid

cache_peer 1.2.3.4 parent 80 0 no-query originserver login=PASS name=site1-http

cache_peer 1.2.3.5 parent 443 0 no-query originserver login=PASS ssl sslflags=DONT_VERIFY_PEER name=site2-ssl

cache_peer_domain site1-http site1.example.lan

cache_peer_domain site2-ssl site2.anotherexample.lan

acl bad_requests urlpath_regex -i cmd.exe \/bin\/sh \/bin\/bash default\.ida?XXX insert update delete select

http_access deny bad_requests

Now I’ll walk us through the above configuration.



http_port 80 vhost

https_port 443 cert=/etc/squid/localhost.crt key=/etc/squid/localhost.key vhost

This sets the http and https ports squid is listening on. Note the cert options for https, we can get squid to use https up to the proxy and unencrytped link to the last hop if we want.. which is cool. If for some reason the server doesn’t support https.



cache_effective_user squid

cache_effective_group squid



Set the effective user and group for squid.. this may not be required, but doesn’t hurt.



cache_peer 1.2.3.4 parent 80 0 no-query originserver name=site1-http

cache_peer 1.2.3.5 parent 443 0 no-query originserver ssl sslflags=DONT_VERIFY_PEER name=site2-ssl

cache_peer_domain site1-http site1.example.lan

cache_peer_domain site2-ssl site2.anotherexample.lan

This is the magic, the first two lines, tell squid which peer to reverse proxy for and what port to use. Note if you use ssl the ’sslflags=DONT_VERIFY_PEER’ is useful otherwise if your using a self signed cert you’ll have certificate errors.

IMPORTANT: If you want to allow http authentication (auth handled by the web server, such as htaccess) then you need to add ‘login=PASS’ otherwise squid will try and authenticate to squid rather than the http server.

The last two lines, reference the first two and tell squid the domains to listen to, so if someone connects to squid looking for that domain it knows where to go/cache.



acl bad_requests urlpath_regex -i cmd.exe \/bin\/sh \/bin\/bash default\.ida?XXX insert update delete select

http_access deny bad_requests



NOTE: The acl line has been cut over two lines, this should be on one. There should be the ACL line and the http_access line.

These lines set up some bad requests to which we deny access to, this is to help prevent SQL injection, and other hack attempts, etc.

That’s it, after a (re)start to squid you it will be reverse proxying the domains.

Redirect to SSL

We had a requirement to automatically redirect to https if someone came in on http. Squid allows redirecting through a variety of ways, you can write a redirect script at get squid to use it, but there is a simpler way, using all squid internals and acls.

Add the following to the entries added in the last section:



acl port80 myport 80

acl site1 dstdomain site1.example.lan

http_access deny port80 site1

deny_info https://site1.example.lan/ site1

acl site2 dstdomain site2.anotherexample.lan

http_access deny port80 site2

deny_info https://site2.anotherexample.lan/ site2

We create an acl for the squids port 80 and then one for the domain we want to redirect. We then use "http_access deny" to cause squid to deny access to that domain coming in on port 80 (http). This causes a deny which is caught by the deny_info which redirects it to https.

The order used of the acl's in the http_access and the deny_info is important. Squid only remembers the last acl used by a http_access command and will look for a corresponding deny_info matched to that acl. So make sure the last acl matches the acl used in the deny_info statement!

NOTE: See http://www.squid-cache.org/Doc/config/deny_info/

Appendix

The following is the configuration all put together now.

Reverse proxy + redirection:



http_port 80 vhost

https_port 443 cert=/etc/squid/localhost.crt key=/etc/squid/localhost.key vhost

cache_effective_user squid

cache_effective_group squid

cache_peer 1.2.3.4 parent 80 0 no-query originserver login=PASS name=site1-http

cache_peer 1.2.3.5 parent 443 0 no-query originserver login=PASS ssl sslflags=DONT_VERIFY_PEER name=site2-ssl

cache_peer_domain site1-http site1.example.lan

cache_peer_domain site2-ssl site2.anotherexample.lan

acl bad_requests urlpath_regex -i cmd.exe \/bin\/sh \/bin\/bash default\.ida?XXX insert update delete select

http_access deny bad_requests

acl port80 myport 80

acl site1 dstdomain site1.example.lan

http_access deny port80 site1

deny_info https://site1.example.lan/ site1

acl site2 dstdomain site2.anotherexample.lan

http_access deny port80 site2

deny_info https://site2.anotherexample.lan/ site2

Posfix – Making sense of delays in mail

The maillog

The maillog is easy enough to follow, but when you understand what all the delay and delays numbers mean then this may help really understand what is going on!

A standard email entry in postfix looks like:



Jan 10 10:00:00 testmtr postfix/smtp[20123]: 34A1B160852B: to=, relay=mx1.example.lan[1.2.3.4]:25, delay=0.49, delays=0.2/0/0.04/0.25, dsn=2.0.0, status=sent

Pretty straight forward: date, email identifier in the mailq (34A1B160852B), recipient, which server the email is being sent to (relay). It is the delay and delays I’d like to talk about.

Delay and Delays

If we take a look at the example email from above:



Jan 10 10:00:00 testmtr postfix/smtp[20123]: 34A1B160852B: to=, relay=mx1.example.lan[1.2.3.4]:25, delay=0.49, delays=0.2/0/0.04/0.25, dsn=2.0.0, status=sent

The delay parameter (delay=0.49) is fairly self explanatory, it is the total amount of time this email (34A1B160852B) has been on this server. But what is the delays parameter all about?



delays=0.2/0/0.04/0.25



NOTE: Numbers smaller than 0.01 seconds are truncated to 0, to reduce the noise level in the logfile.

You might have guessed it is a break down of the total delay, but what do each number represent?

Well from the release notes we get:



delays=a/b/c/d:

a=time before queue manager, including message transmission;

b=time in queue manager;

c=connection setup time including DNS, HELO and TLS;

d=message transmission time.

There for looking at our example:

  • a (0.2): The time before getting to the queue manager, so the time it took to be transmitted onto the mail server and into postfix.
  • b (0): The time in queue manager, so this email didn’t hit the queues, so it was emailed straight away.
  • c (0.04): The time it took to set up a connection with the destination mail relay.
  • d (0.25): The time it took to transmit the email to the destination mail relay.

However if the email is deferred, then when the email is attempted to be sent again:



Jan 10 10:00:00 testmtr postfix/smtp[20123]: 34A1B160852B: to=, relay=mx1.example.lan[1.2.3.4]:25, delay=82, delays=0.25/0/0.5/81, dsn=4.4.2, status=deferred (lost connection with mx1.example.lan[1.2.3.4] while sending end of data -- message may be sent more than once)

Jan 10 testmtr postfix/smtp[20123]: 34A1B160852B: to=, relay=mx1.example.lan[1.2.3.4]:25, delay=1092, delays=1091/0.2/0.8/0.25, dsn=2.0.0, status=sent

This time the first entry shows how long it took before the destination mail relay took to time out and close the connection:



delays=0.25/0/0.5/81

Therefore: 81 seconds.

The email was deferred then about 15 minutes later (1009 seconds [delays - <total delay from last attempt> ]) another attempt is made.

This time the delay is a lot larger, as the total time this email has spent on the server is a lot longer.

delay=1092, delays=1091/0.2/0.8/0.25



What is interesting though is the value of ‘a’ is now 1091, which means when an email is resent the ‘a’ value in the breakdown also includes the amount of time this email has currently spend on the system (before this attempt).

So there you go, those delays values are rather interesting and can really help solve where bottlenecks lie on your system. In the above case we obviously had some problem communicating to the destination mail relay, but worked the second time, so isn’t a problem with our system… or so I’d like to think.

Use xmllint and vim to format xml documents

If you want vim to nicely format an XML file (and a xena file in this example, 2nd line) then add this to your ~/.vimrc file:

" Format *.xml and *.xena files by sending them to xmllint

au FileType xml exe ":silent 1,$!xmllint --format --recover - 2>/dev/null"

au FileType xena exe ":silent 1,$!xmllint --format --recover - 2>/dev/null"



This uses the xmllint command to format the xml file.. useful on xml docs that aren’t formatted in the file.

Debian 6 GNU/KFreeBSD Grub problems on VirtualBox

Debian 6 was released the other day, with this release they not only released a Linux kernel version but they now support a FreeBSD version as well!

So I decided to install it under VirtualBox and check it out…

The install process went smoothly until I got to the end when it was installing and setting up grub2. It installed ok on the MBR but got an error in the installer while trying to set it up. I jumped into the console to take a look around.

I started off trying to run the update-grub command which fails silently (checking $? shows the return code of 1). On closer inspection I noticed the command created an incomplete grub config named /boot/grub/grub.cfg.new

So all we need to do is finish off this config file. So jump back into the installer and select continue without boot loader, this will pop up a message about what you must set the root partition as when you do set up a boot loader, so take note of it.. mine was /dev/ad0s5.

OK, with that info we can finish off our config file. Firstly lets rename the incomplete one:

cp /boot/grub/grub.cfg.new /boot/grub/grub.cfg

Now my /boot/grub/grub.cfg ended like:

### BEGIN /etc/grub.d/10_kfreebsd ###

menuentry 'Debian GNU/kFreeBSD, with kFreeBSD 8.1-1-amd64' --class debian --class gnu-kfreebsd --class gnu --class os {

insmod part_msdos

insmod ext2




set root='(hd0,1)'

search --no-floppy --fs-uuid --set dac05f8a-2746-4feb-a29d-31baea1ce751

echo 'Loading kernel of FreeBSD 8.1-1-amd64 ...'

kfreebsd /kfreebsd-8.1-1-amd64.gz

So I needed to add the following to finish it off (note this I’ll repeat that last part):

### BEGIN /etc/grub.d/10_kfreebsd ###

menuentry 'Debian GNU/kFreeBSD, with kFreeBSD 8.1-1-amd64' --class debian --class gnu-kfreebsd --class gnu --class os {

insmod part_msdos

insmod ext2

insmod ufs2




set root='(hd0,1)'

search --no-floppy --fs-uuid --set dac05f8a-2746-4feb-a29d-31baea1ce751

echo 'Loading kernel of FreeBSD 8.1-1-amd64 ...'

kfreebsd /kfreebsd-8.1-1-amd64.gz

set kFreeBSD.vfs.root.mountfrom=ufs:/dev/ad0s5

set kFreeBSD.vfs.root.mountfrom.options=rw

}

Note: My root filesytem was UFS, thus the ‘ufs:/dev/ad0s5′ in the mountfrom option.

That’s it, you Debian GNU/kFreeBSD should now boot successfully :)

Fedora preupgrade from local mirror

If you have a local mirror and want to use it as the mirror for preupgrade then follow the these normal steps EXCEPT do the following BEFORE you run the preupgrade(-cli) command:

  1. Download the releases.txt file used:

    wget http://mirrors.fedoraproject.org/releases.txt
  2. Modify the releases.txt file, I changed the Fedora 14 (what I’m upgrading to) options to:

    [Fedora 14 (Laughlin)]

    stable=True

    preupgrade-ok=True

    version=14

    baseurl=http://localmirror/fedora/linux/releases/14/Fedora/$basearch/os/

    installurl=http://localmirror/fedora/linux/releases/14/Fedora/$basearch/os/

    #mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-14&arch=$basearch

    #installmirrorlist=


    Note: I commented out the ‘mirrorlist’ and ‘installmirrorlist’ options and added the ‘baseurl’ and ‘installurl’ options.
  3. Finally run the preupgrade command from this directory, as one of the locations it looks for the releases.txt file is ./ (current directory).

For more places you can put the releases.txt see here or see the same info at the end of this post.

Happy upgrading.

preupgrade - tool to help you update a fedora system from one distro to the

next. Pre-resolves dependencies and sets up the system to be

upgraded via anaconda

License: GPLv2 or above

URL: https://fedorahosted.org/preupgrade/

= NOTES =

== Cleanup ==

preupgrade modifies data in ~3 places:

/var/cache/yum/preupgrade*

/boot/upgrade

/etc/grub.conf

If you want to clean up manually, you can do:

preupgrade --clean

Or, if you really want to be sure, do it by hand:

grubby --remove-kernel=/boot/upgrade/vmlinuz

rm -rf /var/cache/yum/preupgrade* /boot/upgrade

== Remote Headless Upgrades ==

Use preupgrade-cli --vnc=VNCPASSWORD.

See preupgrade-cli --help for more info.

The upgrade will start a VNC server on port 5901, requiring the given password.

The upgrade will proceed whether you connect the VNC client or not.

--> IMPORTANT NOTE ABOUT VNC INSTALLS <--

If something goes wrong or the installer needs more info, it will hang forever,

waiting for you to tell it what to do. So you should probably connect a VNC

client and monitor its progress.

== Adding Custom Distributions ==

preupgrade searches the following locations for release data, in order:

./releases.txt

./data/releases.txt

http://mirrors.fedoraproject.org/releases.txt

If you want to add your own distribution to preupgrade: download releases.txt,

edit it to your liking, then run preupgrade from that dir - or save it to

~/releases.txt to make it work when preupgrade is run normally.

Please note that /usr/share/preupgrade/releases.list is ignored and is only

being shipped for compatibility reasons. Use ~/releases.txt for customization.

Bind the bash history methods

Bash is an awesome shell and also very configurable. Some of bash’s build-in functions are not binded to keys, some of which REALLY should be!

Bash has two very useful functions used for searching your bash history:

  • history-search-backward
  • history-search-forward

So the question is how do we bind them. I bound them to Control-<up> and Control-<down>. To do this add the following lines to your ~/.inputrc file:

"\e[1;5A": history-search-backward

"\e[1;5B": history-search-forward

Note: To test it out you can use the bind command for your current shell:

bind "\e[1;5A": history-search-backward

If you want to use a different key combination then you can use the ‘read’ command to print it out, for example runnding the command read, then hitting control-<up>:



matt@wks1005847 ~ $ read

^[[1;5A


When we bind the key we replace the ‘^[‘ with a ‘\e’ as ‘^[‘ can match the <Escape> key.

Git remote ssh syntax

Git makes it easy to add remote repositories to push to, best part is you can use ssh.

The ‘git remote add’ command takes in a html URL like parameter for even SSH like:

ssh://<user>@<host>/<path to git repo>/

NOT the standard ssh scp syntax:

ssh://<user>@<host>:<path to repo>/

Which allows you to base the location from the users home directory or specify a full path:

matt@notrealhost.com:<repo in home directory>/

matt@notrealhost.com:/home/matt/<repo in home directory>/

NOTE: Just put a full path after the ‘:’, or if you want something from the home directory the just assume your in the homedir with out a starting ‘/’.

OK so git doesn’t seem to like that syntax, which is a shame because those of us who use ssh are SO used to it.

The good news is you can specify from the home directory with the git URL style syntax as you do with the standard URL sytax and it looks like:

ssh://matt@notrealhost/~/<repo in home directory>/



Those Linux/Unix guys will recognise the ‘~’ as a shortcut to home in bash, which means the same thing here!

So putting it all together I can add a remote to a git repository which exists in my home directory on that server by:

git remote add notrealhost ssh://matt@notrealhost.com/~/code/myRepository/

This post was written because I keep trying to use the scp syntax when dealing with ssh + git.. So its being filled away here for my own reference.

Git clean

Sometimes I find myself saying WTF, why isn’t something behaving the way I expect it do, and then get frustrated. “This is not how I’d have designed it, if I was writing it”, I guess you can call this the mantra of the OSS developer :P

But as normal, when you blaming something like git or Linux, it just means your doing something wrong or you don’t have a complete understanding of the situation. A lesson I have learnt time and time again, you’d think I’d learn but I don’t.

He’s what happened, we use git at work. Git has some very useful commands.

To return ALL repo tracked files to the state they were in at last checkout:

git reset --hard

To remove all untracked files:

git clean -df

Usually running these two allows out to go back to the point you were at last checkout, removing all compiled files, logs, etc. This is extremely useful for testing.

Also as most revision control systems have, git allows you to create ignore files (.gitignore), so you can tell git try and add certain files or folders to the repository.

OK, so for most people who use git, you’d be saying yeah of course. Well some of my work colleagues noticed that the ‘git clean’ wasn’t actually cleaning all untracked files. It was ignoring the compiled .class files and a heap of other stuff. This seemed weird, we could go to the root directory of the repository create a file, and ‘git clean’ would remove it, put it a few subdirectories down and nope wouldn’t be removed.

In fact running a dry run would return nothing.. so why wasn’t git removing these untracked files.

Well it turns out, and if you haven’t guessed by the fact I mentioned ignore files in the lead up, git is smarter then we gave it credit for. We have ignore files, so what does git do? …it ignores them!

It turns out the ignore files don’t just stop git from adding or wanting to add (telling us about hundreds of untracked files) certain files to the repo, but also, and the file suggests, ignores them in other git commands. This behaviour actually makes sense, if you wanted to keep some notes, wanted to keep them with your code, but not delete them during a ‘git clean’, then just add your notes directory to  a .gitignore file.

Turns out ‘git clean’ has another switch, created to solve the “problem” we were having:

  • -x: Don’t use the ignore rules. This allows removing all untracked files, including build products. This can be used (possibly in conjunction with git reset) to create a pristine

    working directory to test a clean build.
  • -X: Remove only files ignored by git. This may be useful to rebuild everything from scratch, but keep manually created files.

So all we needed to do is run:

git clean -dfx

NOTE: git clean, cleans from the current directory, so if you want to clean the entire repo then make sure your in the root folder of it.

This is actually an awesome feature, so yup, lesson learned again. It wasn’t a problem with git, it was a problem with my understanding! Maybe this time I’ll remember :P

January 22, 2012

Twitter Weekly Updates for 2012-01-22

  • Melbourne summer morning – grass blowing in the wind at Lincoln Square in Carlton, on the way to work at VLSCI. http://t.co/fgC13ZcF #
  • Why can't Android 2.3.3 phone see the CFA FireReady app my Galaxy Nexus can? Link from CFA site gives not found error too #
  • Fact for the day: Papua New Guinea has 1 in 7 of the planets languages (@newscientist, 10th Dec 2011) #
  • Why doesn't the IBM XL C++ compiler for BlueGene/P obey the CPLUS_INCLUDE_PATH environment variable? It's meant to.. #hpc #
  • "Do not edit below this line" – I laugh in the face of your pitiful warning Makefile! (Just needed to change -O3 to -O2) #
  • anyone know if OVCC (Open Visual Communications Consortium – which @AARNet just joined) is pushing real open standards? #
  • Or is OVCC just another vendor club pushing encumbered "open" standards? Their website is content free.. :-( #
  • Rumour is that #VLSCI will be on a segment on the @abc730 program tonight.. #
  • Swanston St Skyline http://t.co/WA92agMK #
  • #Linux kernel 3.3-rc1 now tagged in git.. #
  • A shame that the @CFA_Updates FIreReady app for Android crashes whenever it gets a push notification /cc @naturallybeing #
  • Interesting @abcnews story about crackers using owned accounts to sell shares at high price to others http://t.co/siaAYWvV #
  • Apparently the @abcnews weather people used this photo yesterday – shame I missed it! :-) http://t.co/fgC13ZcF #
  • RT @ScienceCalendar Jan. 22, 1561-Birthday of philosopher Francis Bacon, formulated the principles of the scientific method. #
  • At Fed Square waiting for @mikekuiper to arrive to go see Star Voyager at ACMI #
  • The Light Side and the Dark Side http://t.co/UotAzd2l #
  • Is it just me or is Google's AGPS system being particularly random this arvo? Keep popping up in Europe! #
  • Played Quake at ACMI – did pretty well after not playing for many years + without keyboard config – fragged @mikekuiper #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-01-22

Twitter Weekly Updates for 2012-01-22

  • Melbourne summer morning – grass blowing in the wind at Lincoln Square in Carlton, on the way to work at VLSCI. http://t.co/fgC13ZcF #
  • Why can't Android 2.3.3 phone see the CFA FireReady app my Galaxy Nexus can? Link from CFA site gives not found error too #
  • Fact for the day: Papua New Guinea has 1 in 7 of the planets languages (@newscientist, 10th Dec 2011) #
  • Why doesn't the IBM XL C++ compiler for BlueGene/P obey the CPLUS_INCLUDE_PATH environment variable? It's meant to.. #hpc #
  • "Do not edit below this line" – I laugh in the face of your pitiful warning Makefile! (Just needed to change -O3 to -O2) #
  • anyone know if OVCC (Open Visual Communications Consortium – which @AARNet just joined) is pushing real open standards? #
  • Or is OVCC just another vendor club pushing encumbered "open" standards? Their website is content free.. :-( #
  • Rumour is that #VLSCI will be on a segment on the @abc730 program tonight.. #
  • Swanston St Skyline http://t.co/WA92agMK #
  • #Linux kernel 3.3-rc1 now tagged in git.. #
  • A shame that the @CFA_Updates FIreReady app for Android crashes whenever it gets a push notification /cc @naturallybeing #
  • Interesting @abcnews story about crackers using owned accounts to sell shares at high price to others http://t.co/siaAYWvV #
  • Apparently the @abcnews weather people used this photo yesterday – shame I missed it! :-) http://t.co/fgC13ZcF #
  • RT @ScienceCalendar Jan. 22, 1561-Birthday of philosopher Francis Bacon, formulated the principles of the scientific method. #
  • At Fed Square waiting for @mikekuiper to arrive to go see Star Voyager at ACMI #
  • The Light Side and the Dark Side http://t.co/UotAzd2l #
  • Is it just me or is Google's AGPS system being particularly random this arvo? Keep popping up in Europe! #
  • Played Quake at ACMI – did pretty well after not playing for many years + without keyboard config – fragged @mikekuiper #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-01-22

Twitter Weekly Updates for 2012-01-22

  • Saw the US Girl with the Dragon Tattoo movie. Apart from the gratuitous (but beautiful) intro, it was fantastic. Really well done :) #
  • Yays! :) RT @gavintapp: RT @maxious: Linux.conf.au 2013 Canberra (successful) bid website @ http://t.co/1xZykNtS #lca2012 #
  • #lca2012 was amazing. So many wonderful, inspiring, thought provoking, awesome ppl & discussions. Thx organisers, LA & every person there :) #
  • Yes. I wanna red one :) RT @emmajeans: @piawaugh This! http://t.co/rjCdKzUP #
  • Great point from @ioerror, encrypting data before you store it online helps with privacy/security issues of trusting the #cloud #lca2012 #
  • "I have nothing to hide" is a fallacy or a matter of privilege. — @ioerror #lca2012 #
  • .@ioerror talking about the panopticon and how people tend to self censor when they are under surveillance. See also Foucault. #lca2012 #
  • .@ioerror just took a photo of the crowd. Obviously to identify ALL THE PEOPLE. #lca2012 #gettingparanoid #
  • For those interested, the book ref'd last night is War on the Internet by @BernardKeane. I highly recommend. http://t.co/mltRTFp0 #lca2012 #
  • Last night I had a dream. A dream my cast was on the wrong hand. I woke up trying to pull it off *over* fractured scaphoid. Ouch :) #lca2012 #
  • .@nurhussein thanks, it's fantastic! Great content, ppl, discussions. It is wonderfully inspiring. #lca2012 #
  • I didn't mean it quite like how it came out :) RT @mibus: "With a name like Rusty, we had to get some type of tool…" — @piawaugh #lca2012 #
  • Cool RT @willozap: @piawaugh Ok, new way to start: Fun + Sarkozy: #society5 http://t.co/cVdh02nm http://t.co/t9d0dLbm http://t.co/sWaVur5L #
  • Interesting article by @dannolan on the scope creep (im)balance of policing http://t.co/QiUsXlmZ #
  • Elizabeth Garbee giving gr8 talk on astronomers trying to measure gravitational waves using millisecond pulsars. Ref'd Au SKA :) #lca2012 #
  • RT @kim_weatherall: Paper shows patents impact scientific research http://t.co/7Kb3xYGd (scientists avoid projects impacted) #lca2012 #
  • RT @kim_weatherall: This paper finds patents don't significantly help commercialization of inventions: http://t.co/Z5egwKiq #lca2012 #
  • In open data BoF @aimee_maree talked about Got Gastro, great website. http://t.co/sOtiQTK1 #lca2012 #gov2au #
  • Achieving open data: step 1) just publish, step 2) quality data (compliant systems, etc), step 3) collaborate #lca2012 http://t.co/x6EBDucM #
  • Hey @kim_weatherall, @felix42 said you have some research/stats around patents in Australia. Anything you can share? :) #lca2012 #
  • The Free Software Act, interesting idea. http://t.co/jJcvMe9U #lca2012 #
  • Tridge says independent invention as a defense & interoperability as fair use are two mods that would improve patents system #lca2012 #
  • Grin RT @patentology: FOSS headlines you will never see? What abt 'Blackburn decline Samba transfer request' http://t.co/2MIKrp2G #truestory #
  • Powerful statement on #SOPA from xkcd. Nice. http://t.co/YIglzRMQ #
  • Ta RT @trevclarke: @joshgnosis @j_hutch @gusworldau Slowest search ever, results for "open source" Herald Sun: http://t.co/UZ41UbkP #lca2012 #
  • Handy tip from @gusworldau: press releases are useful but personalise them, tell the journo why they'd be interested #lca2012 #
  • Handy tip for dealing with the press, press releases are useful but personalise them, tell the journo why they'd be interested #lca2012 #
  • Point from @gusworldau on falsity of broad assumption ads always dictate content in tech media. #lca2012 #
  • Linus' response to a media question from @gusworldau on how much it'd cost to get him to come to a geek party. #lca2012 http://t.co/Ypig34cD #
  • "journalists like free speech, they like free beer, but don't always want to get into the complexities in between." @gusworldau #lca2012 #
  • Fascinating, the top 10 tech media websites in Australia by @gusworldau #lca2012 http://t.co/9a7kxqIK #
  • I think we have a good community of tech journos in Australia. Doesn't always translate to coverage says @gusworldau #lca2012 #
  • Totally, voting now :) RT @chrisjrn: Hey, @gusworldau's talk on tech journalism is fantastic. #lca2012 #
  • Angus' talk about FOSS & media. Hilarious & informative. Gets my vote :) "The only way Linux would make the Sun Herald" http://t.co/Anqa5YCZ #
  • .@chrisjrn Hah! "With all due respect…" http://t.co/G4IEoSmp #talledaganights #lca2012 #
  • Whenever people start effectively with "no offense, but…" I am entertained and mildly annoyed. Thanks audience member :) #lca2012 #
  • Haha, jokes OH in @allisonrandal's talk: "man house", "apropos flush" #lca2012 #
  • Accidentally stumbled across this surprisingly interesting article on "Australian egalitarianism", starting w language http://t.co/anAwrkvn #
  • Karen's talk this morning has reminds me to turn off bluetooth/wireless on my phone now, and on my heart monitor later ;) #lca2012 #
  • Cool, RT to #lca2012 @lukeweston: @piawaugh: http://t.co/kAccnvxj #
  • .@Alegrya Big difference between striving for great user experience (similarity) & lock-in Apple imposes vs what FOSS tries to do #lca2012 #
  • .@ioerror Cool, thanks. Hey @supersat @aczeskis, is there a blog post or something about your car hacks? :) /cc @lukeweston #
  • Scary RT @ej_butler: @hackuador @piawaugh Did you see this story last year? Insulin Pump hacked http://t.co/if6ngxwD #medtronic #lca2012 #
  • Really enjoying the talk by Karen Sandler at #lca2012 Bring home the importance of software freedom to everyday life, esp health apps. #
  • Hand is aching today. Too much typing & Tweeting :( Retiring early to write blog & prepare for Martial Arts BoF tmrw morn at #lca2012 :) #fb #
  • Thanks @chrisjrn @kathyreid & @jaimekristene #
  • OK, I've forgotten and can't quickly find this on the wiki, how do I vote for talks for the "Best Of" slots at #lca2012 :) #
  • Fantastic talk by Bdale Garbee about FreedomBox, a great idea to make privacy easy for people online. This gets my vote :) #lca2012 #
  • Wikipedia has gone dark to protest SOPA. Fascinating. Be great to get stats after the blackout on how many click throughs & such #lca2012 #
  • For those I've chatted to about #society5 I've some early thoughts in my blog last year http://t.co/CWgkLcZt #lca2012 More coming v soon :) #
  • Hey @paulzee, you can see most tech related stuff from the last federal budget here http://t.co/HhJVwtrt #lca2012 #
  • OMG PWNIES! :) #lca2012 @caseopaya: @piawaugh link for you http://t.co/lEVUHaym #
  • If ppl want to read great book on critical thinking & failure (even in science!) read @tribalscientist's book http://t.co/q7sOuFAX #lca2012 #
  • Actually, bridge experiment isn't taking into account the variable of person type choosing to take a scary bridge vs safe bridge. #lca2012 #
  • Playing Tetris interferes with long term memory, scientifically proven apparently. Wow. That will be useful :) #lca2012 #
  • Listening to @pjf reminds me of Gladwell's Tipping point. Recognising different traits helps your project "connectors, mavens & salespeople" #
  • Hey @tribalscientist, @pjf is giving kenote (now) at #lca2012 & referencing the chicken and shovel split brain experiment :) You might enjoy #
  • Interesting analogy between taxes and club membership. Comments are quite diverse too. http://t.co/CPqmRWLz #
  • Also for @chrisjrn, other #lca2012 peeps may enjoy the Glenn Beck Conspiracy Theory Generator :) http://t.co/5d1UxIHM #
  • Yo @chrisjrn, you can find that song in my soundcloud favs http://t.co/Z5lhNzm3 "Burn (Angel of Destruction Mix) by TweakerRay" #
  • POLICY CIRCLES is interesting (made for Pacific consultation). Ppl might also be interested in #publicsphere http://t.co/zvI5NUPu #lca2012 #
  • Hearing about Pacific Institute of Public Policy http://t.co/V5XK93fL & Policy Circles http://t.co/Ky9UinMB Interesting projects #lca2012 #
  • Excited about the mix of martial arts lined up for the Martial Arts BoF. Come along if you are interested :) http://t.co/Ar5Y3aPi #lca2012 #
  • Interesting point from POLICY CIRCLES talk at #lca2012 "People are torn between the desire for peace and desire for justice". #
  • .@purserj Ah, BOFH culture, it is a dangerous and seductive path to darkness. Resist the user-hate people! ;) /cc @shorebuck #
  • I think there's something in that for all of us. RT @shorebuck as a user you can hate an editor. As tech support you can hate all of them (: #
  • Whenever I use OpenOffice I am reminded how annoying it can be. For the smug out there MSOffice is no better. I should stick to vim #lca2012 #
  • For @ajtowns, this is stunning! Manual 3D printing of fish -> painted layers between layers of resin. http://t.co/VVAXyEy5 #lca2012 #
  • Interesting read "APS employees have the same right to freedom of expression as other members of the community" http://t.co/aneyG1gd #gov2au #
  • Just linking to the Sarkozy speech for ppls interest, which I'm going over again for research. http://t.co/tA6y0EDM /cc @willozap #society5 #
  • Yes! RT @kattekrab: RT @rillian: #lca2012 remains an awesome group of people. #
  • Watching movie reader that describes video content as it plays. Demo is Elephants Dream which is hard to understand regardless ;) #lca2012 #
  • Really excited about @ioerror's talk Friday. Might feed into a project I'm working on atm so hoping to chat. Watch this space ;) #lca2012 #
  • Martial arts geeks at #lca2012 - Come to the Martial Arts BOF Thurs morn to train & share :) http://t.co/bjjOzUyY #
  • In multimedia miniconf. Jan makes a good point: DRM fundamentally flawed because limiting access to content you want ppl to access. #lca2012 #
  • Relevant to Haecksen #lca2012 RT @PennySharpemlc: From US: why women don't want to run for elected office. Worth a read http://t.co/5WjeEQmP #
  • Stunning RT @neerav: wow 1 of my photos (full moon last wk) http://t.co/ao3RR66Z was featured on the Flickr "Interesting" page. 1515 views #
  • Awesomes RT @BronyRT: RT @nekonoir: Linux + Bronies = all the squee #lca2012 #mylittlepony http://t.co/30RPmfV7 #
  • Tmrw I will bring my pwnies tshirt ;) RT @Tempestrix: Why are there ponies? #lca2012 #sysadminconf #
  • .@weezmgk ah, thanks :) and to the rest of you too :) #
  • In Haecksen miniconf at #lca2012 listening to stats on female representation at lca & of speakers. We have gone from 0% to 23% speakers :) #
  • Arrived at #lca2012 in time for the conf opening. Yays! I'll be tweeting this week about the conf, mostly just thoughts/links/blogs :) #
  • Interesting. Kind or kid? RT @bengrubb: Aussie wunderkind gets $US250k for technology that could revolutionise web http://t.co/1ds0Qtsr #
  • Hah! :) RT @linuxconfau: @piawaugh Safe travels Pia! We'd say break a leg, but we don't want to risk it :-) #
  • One for @johnf :) Moosli from the plane. http://t.co/O1ynQDPo #
  • One for @johnf :) Moosli from the plane. null #
  • Now just a bus ride away from #lca2012 Woot! #
  • On my way to #lca2012 Early start but it's a long way to Ballarat :) Will blog the week as usual. #

Twitter Weekly Updates for 2012-01-22

  • Saw the US Girl with the Dragon Tattoo movie. Apart from the gratuitous (but beautiful) intro, it was fantastic. Really well done :) #
  • Yays! :) RT @gavintapp: RT @maxious: Linux.conf.au 2013 Canberra (successful) bid website @ http://t.co/1xZykNtS #lca2012 #
  • #lca2012 was amazing. So many wonderful, inspiring, thought provoking, awesome ppl & discussions. Thx organisers, LA & every person there :) #
  • Yes. I wanna red one :) RT @emmajeans: @piawaugh This! http://t.co/rjCdKzUP #
  • Great point from @ioerror, encrypting data before you store it online helps with privacy/security issues of trusting the #cloud #lca2012 #
  • "I have nothing to hide" is a fallacy or a matter of privilege. — @ioerror #lca2012 #
  • .@ioerror talking about the panopticon and how people tend to self censor when they are under surveillance. See also Foucault. #lca2012 #
  • .@ioerror just took a photo of the crowd. Obviously to identify ALL THE PEOPLE. #lca2012 #gettingparanoid #
  • For those interested, the book ref'd last night is War on the Internet by @BernardKeane. I highly recommend. http://t.co/mltRTFp0 #lca2012 #
  • Last night I had a dream. A dream my cast was on the wrong hand. I woke up trying to pull it off *over* fractured scaphoid. Ouch :) #lca2012 #
  • .@nurhussein thanks, it's fantastic! Great content, ppl, discussions. It is wonderfully inspiring. #lca2012 #
  • I didn't mean it quite like how it came out :) RT @mibus: "With a name like Rusty, we had to get some type of tool…" — @piawaugh #lca2012 #
  • Cool RT @willozap: @piawaugh Ok, new way to start: Fun + Sarkozy: #society5 http://t.co/cVdh02nm http://t.co/t9d0dLbm http://t.co/sWaVur5L #
  • Interesting article by @dannolan on the scope creep (im)balance of policing http://t.co/QiUsXlmZ #
  • Elizabeth Garbee giving gr8 talk on astronomers trying to measure gravitational waves using millisecond pulsars. Ref'd Au SKA :) #lca2012 #
  • RT @kim_weatherall: Paper shows patents impact scientific research http://t.co/7Kb3xYGd (scientists avoid projects impacted) #lca2012 #
  • RT @kim_weatherall: This paper finds patents don't significantly help commercialization of inventions: http://t.co/Z5egwKiq #lca2012 #
  • In open data BoF @aimee_maree talked about Got Gastro, great website. http://t.co/sOtiQTK1 #lca2012 #gov2au #
  • Achieving open data: step 1) just publish, step 2) quality data (compliant systems, etc), step 3) collaborate #lca2012 http://t.co/x6EBDucM #
  • Hey @kim_weatherall, @felix42 said you have some research/stats around patents in Australia. Anything you can share? :) #lca2012 #
  • The Free Software Act, interesting idea. http://t.co/jJcvMe9U #lca2012 #
  • Tridge says independent invention as a defense & interoperability as fair use are two mods that would improve patents system #lca2012 #
  • Grin RT @patentology: FOSS headlines you will never see? What abt 'Blackburn decline Samba transfer request' http://t.co/2MIKrp2G #truestory #
  • Powerful statement on #SOPA from xkcd. Nice. http://t.co/YIglzRMQ #
  • Ta RT @trevclarke: @joshgnosis @j_hutch @gusworldau Slowest search ever, results for "open source" Herald Sun: http://t.co/UZ41UbkP #lca2012 #
  • Handy tip from @gusworldau: press releases are useful but personalise them, tell the journo why they'd be interested #lca2012 #
  • Handy tip for dealing with the press, press releases are useful but personalise them, tell the journo why they'd be interested #lca2012 #
  • Point from @gusworldau on falsity of broad assumption ads always dictate content in tech media. #lca2012 #
  • Linus' response to a media question from @gusworldau on how much it'd cost to get him to come to a geek party. #lca2012 http://t.co/Ypig34cD #
  • "journalists like free speech, they like free beer, but don't always want to get into the complexities in between." @gusworldau #lca2012 #
  • Fascinating, the top 10 tech media websites in Australia by @gusworldau #lca2012 http://t.co/9a7kxqIK #
  • I think we have a good community of tech journos in Australia. Doesn't always translate to coverage says @gusworldau #lca2012 #
  • Totally, voting now :) RT @chrisjrn: Hey, @gusworldau's talk on tech journalism is fantastic. #lca2012 #
  • Angus' talk about FOSS & media. Hilarious & informative. Gets my vote :) "The only way Linux would make the Sun Herald" http://t.co/Anqa5YCZ #
  • .@chrisjrn Hah! "With all due respect…" http://t.co/G4IEoSmp #talledaganights #lca2012 #
  • Whenever people start effectively with "no offense, but…" I am entertained and mildly annoyed. Thanks audience member :) #lca2012 #
  • Haha, jokes OH in @allisonrandal's talk: "man house", "apropos flush" #lca2012 #
  • Accidentally stumbled across this surprisingly interesting article on "Australian egalitarianism", starting w language http://t.co/anAwrkvn #
  • Karen's talk this morning has reminds me to turn off bluetooth/wireless on my phone now, and on my heart monitor later ;) #lca2012 #
  • Cool, RT to #lca2012 @lukeweston: @piawaugh: http://t.co/kAccnvxj #
  • .@Alegrya Big difference between striving for great user experience (similarity) & lock-in Apple imposes vs what FOSS tries to do #lca2012 #
  • .@ioerror Cool, thanks. Hey @supersat @aczeskis, is there a blog post or something about your car hacks? :) /cc @lukeweston #
  • Scary RT @ej_butler: @hackuador @piawaugh Did you see this story last year? Insulin Pump hacked http://t.co/if6ngxwD #medtronic #lca2012 #
  • Really enjoying the talk by Karen Sandler at #lca2012 Bring home the importance of software freedom to everyday life, esp health apps. #
  • Hand is aching today. Too much typing & Tweeting :( Retiring early to write blog & prepare for Martial Arts BoF tmrw morn at #lca2012 :) #fb #
  • Thanks @chrisjrn @kathyreid & @jaimekristene #
  • OK, I've forgotten and can't quickly find this on the wiki, how do I vote for talks for the "Best Of" slots at #lca2012 :) #
  • Fantastic talk by Bdale Garbee about FreedomBox, a great idea to make privacy easy for people online. This gets my vote :) #lca2012 #
  • Wikipedia has gone dark to protest SOPA. Fascinating. Be great to get stats after the blackout on how many click throughs & such #lca2012 #
  • For those I've chatted to about #society5 I've some early thoughts in my blog last year http://t.co/CWgkLcZt #lca2012 More coming v soon :) #
  • Hey @paulzee, you can see most tech related stuff from the last federal budget here http://t.co/HhJVwtrt #lca2012 #
  • OMG PWNIES! :) #lca2012 @caseopaya: @piawaugh link for you http://t.co/lEVUHaym #
  • If ppl want to read great book on critical thinking & failure (even in science!) read @tribalscientist's book http://t.co/q7sOuFAX #lca2012 #
  • Actually, bridge experiment isn't taking into account the variable of person type choosing to take a scary bridge vs safe bridge. #lca2012 #
  • Playing Tetris interferes with long term memory, scientifically proven apparently. Wow. That will be useful :) #lca2012 #
  • Listening to @pjf reminds me of Gladwell's Tipping point. Recognising different traits helps your project "connectors, mavens & salespeople" #
  • Hey @tribalscientist, @pjf is giving kenote (now) at #lca2012 & referencing the chicken and shovel split brain experiment :) You might enjoy #
  • Interesting analogy between taxes and club membership. Comments are quite diverse too. http://t.co/CPqmRWLz #
  • Also for @chrisjrn, other #lca2012 peeps may enjoy the Glenn Beck Conspiracy Theory Generator :) http://t.co/5d1UxIHM #
  • Yo @chrisjrn, you can find that song in my soundcloud favs http://t.co/Z5lhNzm3 "Burn (Angel of Destruction Mix) by TweakerRay" #
  • POLICY CIRCLES is interesting (made for Pacific consultation). Ppl might also be interested in #publicsphere http://t.co/zvI5NUPu #lca2012 #
  • Hearing about Pacific Institute of Public Policy http://t.co/V5XK93fL & Policy Circles http://t.co/Ky9UinMB Interesting projects #lca2012 #
  • Excited about the mix of martial arts lined up for the Martial Arts BoF. Come along if you are interested :) http://t.co/Ar5Y3aPi #lca2012 #
  • Interesting point from POLICY CIRCLES talk at #lca2012 "People are torn between the desire for peace and desire for justice". #
  • .@purserj Ah, BOFH culture, it is a dangerous and seductive path to darkness. Resist the user-hate people! ;) /cc @shorebuck #
  • I think there's something in that for all of us. RT @shorebuck as a user you can hate an editor. As tech support you can hate all of them (: #
  • Whenever I use OpenOffice I am reminded how annoying it can be. For the smug out there MSOffice is no better. I should stick to vim #lca2012 #
  • For @ajtowns, this is stunning! Manual 3D printing of fish -> painted layers between layers of resin. http://t.co/VVAXyEy5 #lca2012 #
  • Interesting read "APS employees have the same right to freedom of expression as other members of the community" http://t.co/aneyG1gd #gov2au #
  • Just linking to the Sarkozy speech for ppls interest, which I'm going over again for research. http://t.co/tA6y0EDM /cc @willozap #society5 #
  • Yes! RT @kattekrab: RT @rillian: #lca2012 remains an awesome group of people. #
  • Watching movie reader that describes video content as it plays. Demo is Elephants Dream which is hard to understand regardless ;) #lca2012 #
  • Really excited about @ioerror's talk Friday. Might feed into a project I'm working on atm so hoping to chat. Watch this space ;) #lca2012 #
  • Martial arts geeks at #lca2012 - Come to the Martial Arts BOF Thurs morn to train & share :) http://t.co/bjjOzUyY #
  • In multimedia miniconf. Jan makes a good point: DRM fundamentally flawed because limiting access to content you want ppl to access. #lca2012 #
  • Relevant to Haecksen #lca2012 RT @PennySharpemlc: From US: why women don't want to run for elected office. Worth a read http://t.co/5WjeEQmP #
  • Stunning RT @neerav: wow 1 of my photos (full moon last wk) http://t.co/ao3RR66Z was featured on the Flickr "Interesting" page. 1515 views #
  • Awesomes RT @BronyRT: RT @nekonoir: Linux + Bronies = all the squee #lca2012 #mylittlepony http://t.co/30RPmfV7 #
  • Tmrw I will bring my pwnies tshirt ;) RT @Tempestrix: Why are there ponies? #lca2012 #sysadminconf #
  • .@weezmgk ah, thanks :) and to the rest of you too :) #
  • In Haecksen miniconf at #lca2012 listening to stats on female representation at lca & of speakers. We have gone from 0% to 23% speakers :) #
  • Arrived at #lca2012 in time for the conf opening. Yays! I'll be tweeting this week about the conf, mostly just thoughts/links/blogs :) #
  • Interesting. Kind or kid? RT @bengrubb: Aussie wunderkind gets $US250k for technology that could revolutionise web http://t.co/1ds0Qtsr #
  • Hah! :) RT @linuxconfau: @piawaugh Safe travels Pia! We'd say break a leg, but we don't want to risk it :-) #
  • One for @johnf :) Moosli from the plane. http://t.co/O1ynQDPo #
  • One for @johnf :) Moosli from the plane. null #
  • Now just a bus ride away from #lca2012 Woot! #
  • On my way to #lca2012 Early start but it's a long way to Ballarat :) Will blog the week as usual. #

Internet Routers

It would appear our household really hammers the router, in fact so badly it doesn’t seem to cope after a few days.

I’ve got an old Pentium 4 here, so after running up pfsense on it for a 2nd time, it seems to handle things so much better. Plan is to use this machine for the next 6 or so months, while I continue to eye off a lovely ALIX 2-3 unit to take over from it. It’s just a matter of time before I buy one, as pfsense certainly seems to handle things nicely. I guess future purchases will be smart switch again to make use of the VLAN tagging (which will go nicely with our Ubiquiti UniFi Access Point too).

Will post picture of the ALIX 2-3 and other details when I do actually purchase one, it’s just a matter of timing on my part.

Talks to catchup on

Okay so I missed #lca2012 this year. However because the LCA team is awesome (continuing the grand tradition of awesome LCA teams), the talks have started appearing on the linuxconfau2012 youtube channel.

Squeeee.

Now I have a number of talks to watch on the googletv thing:

Tux in Space: High altitude ballooning - Joel Stanley,Mark Jessop

Lego + Kids + Arduino - James Muraca

Making video streaming interactive, heckling user groups from the clouds!

Android Accessories Made Easy With Arduino - Philip Lindsay

Desktop Home Hacks - Allison Randal

Now these are just the one's I want to watch immediately. If you have a love of hackery, FOSS and all things shiny then I think you need to go through those talks and queue them up on your device of choice.

Also remember, the videos will be made available on the Linux Australia Mirror shortly, so you will be able to grab all of the talks and watch them whether you're online or not.

As I said before: Squeeee

January 21, 2012

[debian] Bits from the ISC DHCP Maintainer

I really should write these a bit more often.

Wow, I can't believe it was over 4 years ago that I started having occasional face to face meetings with the ISC DHCP folks.

The entire ISC DHCP team (of 5) was in town for an all-hands meeting, and Larissa Shapiro, the Product Manager for DHCP (and BIND) suggested it would be a good opportunity for another catch up. Given the current (bad) state of DHCP 4.2 in unstable, I thought this was an excellent idea, and so we all had lunch on Tuesday.

I pretty much set the agenda, and it was

  • general state of 4.2.2 in Debian
  • situation with GNU/Hurd and their patch to fix an FTBFS (#616290)
  • the current FTBFS issues with kFreeBSD (#643569)
  • the embedded BIND sources in the DHCP source
  • removal of the RFCs from the embedded BIND source (#645760)

The general state of 4.2.2 in Debian

In a nutshell, it's a bit of a mess. We've got release critical bugs, build failures, the whole cat and kaboodle. It makes me very sad, because 4.2.2 was the first 4.2 series that I had a chance to upload, and I was very excited to do so, because it contains the hotly desired LDAP patches merged upstream. Unfortunately, it's also got the beginnings of the BIND/DHCP merger that's going to be BIND 10, and that is all a bit of a mess. It's directly responsible for the kFreeBSD FTBFS and the introduction of the RFCs, which are both keeping 4.2.2 out of testing.

I gave the ISC folks a high-level overview of how Debian development works, and the normal progression of packages from unstable to testing to stable, and the release process and whatnot, and impressed upon them the implications of the current release critical bugs. I also showed them how Ubuntu development fitted into the picture. Finally, I showed them the popcon statistics for DHCP. I think they found it useful.

FTBFS issues on kFreeBSD

This was a good segue to #643569. The issue is actually with the embedded BIND sources. I'd already forwarded this bug upstream when it first happened, but I don't know what had happened to it. They seemed to act as if this was the first they'd heard of it. I'm hoping that they can get this fixed in 4.2.3, which is due around the end of the quarter.

Embedded BIND sources

Since we were already talking about an issue caused by the embedded BIND sources, we moved on to talking about #645760 and the existence of the embedded BIND sources in general. It should be pretty straightforward for them to strip the RFCs out of the source. They've already done it in the past for the DHCP sources, so I'm also hopeful that this will get resolved in 4.2.3.

The issue of the embedded BIND sources is apparently a bit more complicated, although the day before our meeting, Michael Gilbert filed #643569 and #645760, so I hope that the ISC folks can take a look at these patches and see if it's feasible to adopt them.

Patches for GNU/Hurd

Finally we talked about #616290, which I know is near and dear to the GNU/Hurd porters' hearts.

We probably spent the most time talking about this. The DHCP developers have concerns about accepting a patch for an OS that they do absolutely no testing on, and also questioned the viability of the OS in general. They stressed that they're fairly thin in numbers relative to what they have on their plate to achieve this year, and so pushed back pretty firmly on accepting the current patch.

I relayed the frustration that the Hurd folks were having about a lack of dialogue around the patch (most of the interaction has been via an ISC support person). There was actually a bit of a split between the developers, with one of them appreciating that the Hurd was unlikely to go anywhere as a platform without a working DHCP client, so in some regards, they were condemning the platform by taking the position they were taking.

They're going to go away and take another look at the patch and try to come back with some actionable feedback on what needs to change to make it more acceptable to them, so we'll see what comes of this. I'm not particularly optimistic that anything acceptable to the GNU/Hurd folks is likely to happen any time soon, but maybe if the patch gets cleaned up a bit more, I'll just bite the bullet and start applying it to the Debian package.

BIND 10

One of the guys is more involved in BIND 10 than DHCP, and asked if I could help out with the packaging of a build dependency for BIND 10. It seemed like #578387 was languishing so I offered to pick it up. I've not packaged a library before, mainly because the library packaging guide has scared me off it (I feel I lack the deep C fu that seems necessary), but I figured that this would be a good learning opportunity, so I'm going to dive in.

linux.conf.au Returns to Canberra in 2013

I am incredibly pleased to announce that linux.conf.au 2013 will be hosted by Canberra, Australia between 28 January 2013 to 2 February 2013. As the director for 2013 I have been blessed with a simply incredible team who has done fantastic work during the bid process, and I am confident that we will pull off a fantastic event. 2013 is Canberra's centenary year, so I think its appropriate to have a conference with a bit of a party atmosphere. We're working hard already on making 2013 a conference to remember.



For those who were unable to see the announcement at the conference, you might find the following interesting:







linux.conf.au is one of the foremost open source conferences in the world, and is considered the most prestigious in the southern hemisphere. Many of the team that brought you linux.conf.au 2005 are coming back to help with the 2013 effort, and we're cognizant of the extremely high standard left by previous conferences, especially the astounding job that Josh's 2012 team did.



The web site for the conference http://lca2013.linux.org.au is already live, and we'll be keeping it up to date as details are locked in.



Tags for this post: conference lca2013 announcement canberra

Related posts: Two more weeks to go; In Canberra; Mont 24 hour race; Most novel traffic jam cause goes to... Canberra!; So, what on earth was I doing up at 4:30 am anyways?; Announcing early results of my survey of SMTP servers; What are we doing with the pets?; Electric shadows has a RSS feed!; Travel details so far; Frank Arrigo discovers Steve Walsh's free wireless; Are you in a LUG? Do you want some promotional materials for LCA 2013?; On a bunker kick; Calling Tate Needham, or, Hiring in Canberra; Historical revisionism; LCA weather; Back in Canberra again



Comment

LCA2012: Over for another year

I’m way too exhausted to write a full debriefing of linux.conf.au 2012, but I would like to share a few random thoughts, sorted equally randomly:

  • Pretty much all the talks I attended were great in their own way. David Rowe’s Codec 2 talk was one of my favourites, and Chris Neugebauer’s Open Programming miniconf was also educational. (Also, Joel Stanley mentioned me in his iviewiir talk!)
  • All the keynotes were great in their own way. In particular, Paul Fenwick proved that he can deliver a great talk on anything — his talk was topically psychological. All of the keynote speakers hung around for most of the conference, which was awesome! (Even got to meet Jacob Appelbaum, who is actually a surprisingly normal, cool, and friendly guy.)
  • The conference, overall, ran really smoothly. Most attendees I spoke to explicitly said they noticed no issues whatsoever all week. That, to me, says job well done. Of course, nothing is perfect, and when things did happen — particularly issues that were well handled and mitigated by the organisers behind the scenes — I was reminded of software engineering, in that you should judge quality based on how failures are handled, not just a measurement of success.
  • Really happy to hear that Canberra is hosting linux.conf.au 2013. I’ll probably not be volunteering in 2013 — doing it in 2011 and 2012 was quite enough. But never say never. :-)
  • Wish me well for the long drive home!

January 20, 2012

LCA2012 – Friday after lunch

Codec 2 – David Rowe

  • Open speech Codec. Low bitrate 2400 b/s down to 1400 b/s
  • Applcations for digital radio
  • Fills <5000 b/s gap
  • http://rowetel.com/codec2.html
  • Not a DSP talk
  • Can send 45 calls inside 64 kb/s chanel
  • Not useful for VOIP due to IP/UDP overhead of 8kb/s on 1400b/s data
  • Main use radio spectrum. Less data = less power required since your power gets concentrate on less bits
  • doesn’t matter too much if odd packet dropped
  • proprietary codecs slowing digital voice over radio
  • Proprietary codes: hardware or licensed software form, difficult to distribute, can’t modify
  • Example g729 license $40k. Doesn’t believe closed source codecs benefit society
  • Authors of propriety/patented codecs borrowed heavily from public domain. perhaps 5% is original. Good news is only 5% needs to be replaced
  • Speech coding: eg 16bit samples at 8kHz, comprss to 1400-2400 b/s . What can we thrown away, retain intelligible speech, retain natural speech. Use a model of speech, send model parameters, for effecient than coding waveform
  • Model: example is pitch, humans 50-500 Hz , can be represented with 7 bits, updated every 20ms 7/0.02 = 350b/s to represent pitch
  • Codec 2 uses Sinusoidal speech coding. Multiple Sine waves added togeather
  • Bit allocation: 56bits every 40ms. Of these: Amplitude 32 , Frame energy 10 , voicing 4, pitch 10
  • Developing Codecs: complex DSP algorithms, run codecs in non-realtime, dump values from codecs every “frame” ( 80 samples, 10 ms of speech) . Gnu Octave
  • Banned exports list includes ” Speech codecs below 2400 b/s ” . Have been advised by DECO that Codec 2 has “assessed as not controlled” but waiting for certificate
 

UEFI and Linux – Matthew Garrett

  • Replacement for PC BIOS
  • BSD licensed core
  • Adds standardized support for new hardware features
  • Platform init
  • EFI image load – loaded drivers
  • EFI OS loader load – oot from ordered list of EFIOS loaders
  • Boot services terminate -> OS handover
  • Boot services – memory allocation, timers, image loading, GUIDs.
  • Runtime services – non-volatile variable store, boot data, system information, crash dumps (already in Linux 3.2)
  • Able to update firmware by reset and grab new firmware out of variables on bootup
  • GPT – GUID partition table – no practical restrictions on size and number – more metadata about partition type and service
  • That all sounds good …. but ….
  • TianoCore – Open Intel reference UEFI reference implementation, 7061 files, >100MB of code, 10% of size of Linux kernel. Bigger than Linux core kernel
  • Large codebase, some bugs
  • UEFI is poorly tested in the real world. UEFI contains a lot of code. UEFI contains a lot of bugs
  • Some problems with secure boot :)

LCA2012 – Friday Morning

Bloat: How and Why UNIX Grew Up (and Out) – Matt Evans and Rusty Russell

  • Cool projects: spark, plover, Homebrew Cray-1A
  • Compare PDP-11 Unix vs Modern Ubuntu 11.10
  • Binary sizes: cat 152 bytes vs  531k KB
  • grep command: 2176 bytes vs 687 KB
  • ls command: 4904 bytes vs 628 KB
  • V6 cat command just 12 lines of assembler, 2 * 512bytes buffers, a.out 16 bytes overhead
  • Binaries 30% because we chose speed over size. ~9% speed gain
  • V6 Runtime coverage: cat 99% , grep 78%, ls 85%
  • V7 has reduced coverage. some commands converted from assembler to C
  • x86 runtime with dietlibc coverage: cat 11% , grep 23% , ls 39%
  • x86 static cat has 700k of libc dependencies, 17% of libc, 313 objects it depends on
  • libc 1.7M but widely shared among hundreds of processes
  • dynamic ls accesses 90k of libc but 476kB paged in.
  • For sample system. libwebkit 8.5MB , 5MB wasted, 33MB wasted real RAM
  • What about a 64Bit version of a PDP-11 – a PDP-11 . Various assumptions on how binary size would increase
  • PDP-44 – binaries around 50% larger
  • 32 bit ubuntu binaries are 9% smaller than 64 bit ubuntu
  • Forward port V6 binaries to x86 . V6 cat almost same size as dietlibc version
  • More work to forward port V6 ls, lots of assumptions not longer true. Code tricks no longer work. 20% larger cause of ELF and nmap. 120% penalty due to modern infrastructure (eg malloc realloc)
  • Backport x86 “ls” and “cat” to V6. Only backport some options
  • cat: remove old options and error reporting. Kept some features.
  • ls: remove lots of options.
  • Binaries 60% larger due to flexibility
  • 440% bloat due to new features
  • Asmutls – reimplementation of current Linux utils in x86 assembler.
  • The talk is online, hard to do notes since it jumped around a lot and graphs hard to read
 

Open vSwitch – Simon Horman

  • Switch contains ports, ports has one of more interface, packets are forwarded by flow
  • Flows may be identified by lots of combos, address, vlan, ports, TOS
  • 1st packet in flow gets sent to userspace controller, controller makes decision, tells datapath what to do with future packets, resends first packet back to datapath. Later packets the datapath knows what to do (from hashtable lookup) and handles itself
  • Configured by JSON database, persists across restarts
  • database controlled via Unix socket or via TCP. Change action won’t return until database update performed
  • cute ” –may-exist ” options when creating stuff that does nothing if what you are requested already exists
  • He did some demos of standard sort of stuff, truck interfaces, port mirroring, fairly simple commands to do
  • Does VXLAN and GRE tunnels
  • Oracle looking to put in Oracle Linux soon to replace current bridging code
  • Can do millions of packets per second. Some bottlenecks in tunneling code

New Relic, Heroku and Rails Streaming Responses

For Ticketscout’s payment processing, we use a custom Rails action with an HTTP streaming response. It shows the user a spinner and a “please wait” message, then periodically sends empty strings to the browser until the credit card transaction is completed. This streaming response keeps the request connection alive longer than the 30 second timeout period that is the default on the Heroku Cedar stack. The technique works well and ensures we don’t ever lose track of any slow-to-process credit card payments.

During development, we found that the streaming responses on Heroku did not work if we had the New Relic add-on enabled. Instead of seeing the page content incrementally delivered while the request ran, we’d see nothing until the request completed, then all the content delivered at once.

To deliver the feature on time, we simply disabled the add-on and released. Everything worked smoothly, but it was unnerving to run the app “blind” without New Relic’s useful metrics. This became especially problematic now that we’re wanting to pay some more attention to improving the app’s performance. I finally contacted New Relic support, and it turns out that a working solution is fairly simple: disabling the real user monitoring.

Turns out that the real user monitoring is currently incompatible with streaming responses. I disabled the automatic instrumentation in my newrelic.yml config file:

browser_monitoring:
  auto_instrument: false

I also switched off the “Enable end user monitoring” setting in the “Settings > Application” page. Then, after fresh deploy to Heroku, I re-enabled the add-on and we had both the metrics collection and streaming responses working together!

Granted, we’ve lost access to some useful real user metrics (like network load, page rendering and DOM processing times), but it’s better to have some metrics than none at all!

Down the track, we can probably get the best of both world’s by manually enabling the real user monitoring instrumentation in all the pages layouts except those used for the streaming responses. Hopefully further down the track, the automatic instrumentation will be compatible with streaming responses out of the box. In the meantime, it’s good to have our charts back!

January 19, 2012

LCA2012 – Thursday last session

Challenges for the Linux plumbing community – Jonathan Corbet

  • Good news is boring, so how about some “high quality problems”
  • Security
  • Stuxnet , kernel.org , RSA hack , DigiNotar
  • Scary ones are there must be others we haven’t heard about
  • The bad guys are: motivated, capable, well funded. Not just script kiddies
  • Not just about money anymore, with governments hacking lives are at stake
  • We are on the front line. Not just security software, all code security critical
  • Is your code secure? Who reviews it? What sort of testing? Plans for dealing with vulnerabilities?
  • Is your infrastructure secure? – Who has access, who can change files? Are security updates applied? What are your plans in case of a breach?
  • Are your processes secure? Who can commit? What can sign releases? Can you detect tampering? What do they know about the codes provenance?
  • Tools
  • Lockdep, valgrind, fault injection, sparse, smatch
  • GCC python plugin, MELT, LLVM static analyzer
  •  and need to actually use the tools that exist
  • Hardware
  • hardware complexity leads to software complexity
  • Complex interfaces: example V4L3 media controller interface.
  • Control over our hardware
  • Life is okay (could be better, could be worse)
  • What is our influence over manufacturer?
  • Example: Chasing tablet manufactures , no influence on design, have to port after device launched
  • Example: By the time “Rock Box” runs on a device device is obsolete and not in shops
  • How can we be more involved in conception and design of hardware in the first place?
  • Linux Only
  • Once upon a time we depended heavily on portability
  • The DRM tree deemphasized BSD support, This hurt BSD but… would we rather do without kernel mode settings
  • Might be inevitable but try not to be too arrogant
  • The platform problem
  • Code you control vs Black box
  • The kernel’s ARM subtree (re-implements stuff from elsewhere in kernel)
  • XFree86 (tried to keep everything in user space)
  • Opportunistic suspend (Andriod decided “too hard” to fix rest of kernel)
  • Async I/O (implemented multiple times, no comprehensive implementation)
  • Example: wireless devices had own 80211 implementation. replaced with max80211
  • Example: PowerTop used to find wide range a random things causing high power usage in laptops
  • Ongoing examples: Bufferbloat, marvell-cam drivers, User-Space TCP, Control groups, Andriod
 

 

What is in a tiny Linux installation? by Malcolm Tredinnick

  • Skipping bootloader portion
  • Kernel is big – 9.6M lines of C, 250k lines of assembler
  • Booting the kernel
  • “make allnoconfig” , smallish, 222 “y” ‘s. 842KB bzimage, build time under 15s, no file systems, no fancy hardware, ISA, no PCI
  • “make allyesconfig” , 5177 y options. 39MB bzimage, over 1h to build, includes drivers/staging
  • booting allnoconfig via qemu-kvm . Gets to “unable to mount root file system”
  • Kernel components – hardware arch, drivers, subsystems, others
  • need roto filesystem in memory, initrd / initramfs . init process just in cpio archive, can just be hello world
  • need initrd, initramfs , RAM disk block device, ELF binary support. 889KB bzimage (up 50kb)
  • Now boots, use “rdinit=/hello” option in qemu , just prints out hello world
  • Transition to userspace
  • initrd loads some modules etc, runs pivot_root , run startup scripts
  • Userspace
  • Why are you doing this? Single purpose system, usb stick (rescue, puppy linix, Damn Small Linux) , tiny memory, tiny storage usage, fast power on. Trade-off of options
  • We have to run something, need some binaries, shared libraries, large binaries with multiple purposes (busybox)
  • Busybox – one binary – acts differently depending on calling name, installed as symlinks
  • Busybox: fairly small, default utilities, 2MB without networking, easy to test
  • C libraries – glibc (probably not a good idea), eglibc (easier to build, binary compatible with glibc, can take things out), uClibc (alternative, very small, some overlap with busybox, source code compatible with glibc)
  • Device and Proc mngt will need: procfs, sysfs, tmpfs, udev, cgroups
  • Build environments: you are cross compiling (build root), binutils, C libraries & cross compiling, Test, x86 instead x86 is harder
  • See links in slides for some help
  • mdebian is something to look at

LCA2012 – Thursday after lunch

Women in Open Technology & Culture – Valerie Aurora and Mary Gardiner

  • Very umbrella term including fan fiction, open data, wikipedia, open access
  • Why – important areas – women’s participation (especially in charge) very low
  • Important for women to be in charge, creating, designing, building, not just as users
  • 5 kinds of groups – project specific (debian women), feminist activism, teaching technical skill, networking, majority women projects.
  • Community / project specific
  • Linuxchix, owoot, pyladies, wikichix, etc ( linuxchix spawned several)
  • low participation, poor replacement rate of leaders (often after they get FT jobs), low communication between, sometimes tension between.
  • Feminist advocacy
  • geek feminism, ada initiative, mind the gap
  • growing and active – the new hotness, sharing best practices, paid work more common, some conferences
  • Teaching women technical skills
  • usually one day or evening courses.
  • Growing hugely, vary widely in topics and skills, sharing best practises
  • In person networking socialization
  • Women in code, girl geek coffees, girl geek dinner
  • try not to be dominated by marketing women ( use of “geek” term helps)
  • Growing, easy to start local chapters
  • Majority Women Groups
  • Dreamwidth, Organisation for transformative works
  • Often fan-fiction support, protect against takedown, let author control commercialisation
  • Survey
  • In person vs Online
  • Activist vs non-activist
  • Community vs technical
  • Focussed vs broad topics
  • Projects with broad focus within a narrow group seem not to work
  • Projects with very technical focus but accoess different technologies seem not to work either (lack common language)
  • Why Start – recruit and retain, networking, role models, safe space, feel normal
  • Lessons on starting
  • Don’t – join an existing one
  • If you are a man, don’t do on behalf – “Nothing about us without us”
  • Don’t expect women to start a group
  • Find 3 or more women to start a group
  • Don’t use girl/chix/ladies – use women
  • Go broad instead of narrow on topic
  • have clear defined goals and scope
  • Start small, be realistic about work
  • Consider one-off event rather than group
  • Avoid NIH , reuse best practices
  • be prepared to moderate any public forums you create
  • Failure modes
  • Become “the nice place” that everybody goes to
  • Loses focus on women
  • Safe Space moderation too many hours
  • Ran low on time, slides will be online
 

Hacking Everything – Matt Evans

  • Reuse things , not just hacking things like audrino that are supposed to be hacked
  • reuse, need, art & design
  • Gambiarra – brazilian art of an improvised fix
  • 1940s radios and TV owners could fix their gear. today people are more passive
  • wants people to tinker with things.
  • Save resources
  • Save money
  • take apart things, learn by example
  • Low cost manufacturing makes hacking hard ( solid state everything )
  • Cheap development makes hacking easier ( reuses common technology, extra bits on devices unused )
  • Some products are open hardware designs
  • Things to look for – similar to ref design, debug code left in, unused features, factory test points/ports
  • Ports that are wired up but unused often serial ports
  • “My CD player has a serial port” , common on many devices
  • Acquire a “logic level”USB-serial cable
  • Other ports – JTAB , In-System programming
  • Example: Picture frame, derived from sample board for camera, serial interface, built in CLI
  • Old Wifi, ADSL boxes good with OpenWRT
  • Don’t just consume – re-consume
  • Teach others and tell the world
  • Collaborate at a local hackerspace
  • support companies that make things hackable

brendanscott

Stop Taking Our Rights And Calling Us Thieves Act

STORACUTA

There are manifold problems with the way the legalization of monopolies operates around the world.  One of those problems is the continual pushing of extremist positions on legal monopolies, in order for a not quite as extreme “compromise” position to be reached, which is just pushed further out in the next round of lobbying.  Instead of just opposing the SOPA, it would be more appropriate to propose alternative legislation as a counter balance to attempt to achieve a moderate position.

Provisions which could be included might be:

* express limitation of damages in monopoly infringement cases to damages actually incurred, abolition of presumed or statutory damages – per Tim O’Reilly (and many others, but Mr O’Reilly has been noteworthy recently);

* repeal of any provision which enables geographical market segmentation;

* prohibition on the use of monopolies to restrict speech;

* repeal of any provision which enables aftermarket control of goods – eg rental rights and DRM;

* an offence for a public official to call the infringement of any monopoly “theft”

* prohibition on claiming lack of sale as losses

* express application of anti-trust law to the exercise of any monopoly?

Ideas?  What would you like to see included in STORACUTA?

Note:

Tim OReilly, is The Man, by the way.   A few years ago I figured I wanted to learn Python.  I googled around for what resources were out there and, in the course of so doing, I stumbled across “free” downloads of some of the O’Reilly Python books.  I didn’t actually use the “free” versions – I bought them from O’Reilly anyway, largely because O’Reilly e-books were DRM free. Strangely, buying those first ebooks from O’Reilly was a thoroughly enjoyable experience.  I have since almost gone out of my way to buy stuff from them (Algorithms in a Nutshell? WtH?), this, despite being aware I could probably pick up a “free” copy somewhere.



LCA2012 – Thursday Morning

Desktop Home hacks – Allison Randal

  • Just a hobby product, must be fun, open,
  • Wanted computer to be available when away from desktop, but not be disruptive and uncomfortable
  • Affordable, approachable to hobbyist, there are more expensive alternatives
  • nodes – jeenode, audrino clone, tiny, easy to hid, cheap ( $20 including wireless RFM12B vs $20-30 + 30 for wifi for more conventional audrino )
  • RFM12B – 66 bytes
  • Server – pandaboard ~$200 . ARM processor with full Ubuntu install
  • Server – aggregates data from all notes and sends commands to all it’s nodes. Contains services for system, json data feeds from nodes, web client interface, interface for sending commands to system
  • Client(s) – several machines in house used from
  • client – jquery mobile , small applet , on desktop , chromium app mode
  • Inputs: Temperature ,  Humidity , motion sensor (lights on/off)
  • Input: RFID reader (disappointed at short range, <1m , one was in doorways to trace path of wallet)
  • Input: touch sensor ( simple controls, in pillow ) , small keyboard (bluetooth, usb)
  • Output: Glowing egg ( multi colour, hand sized ) , Power Tail (power extension, turns on/off)
  • Output: Hollowed out candle with LEDs inside , Instamorph & super Sculpey to create “solid” objects
  • Tools: Soldering iron, misc tools, wax carving kits
  • Lesson: Need better camera with macro lens and better light to document :)
  • software not currently release, not really in releasable format, thinking of making some as audrino shields
  • Resources: adafruit.com , sparkfun , parallax.com , digikey.com (bad interface) , optopart.com , makershed.com , freetronics.com (Australia) ,
  • Tools: Talk in inkscape/sozi  , Also use: vala (webserver) , jquery mobile
 

Cheap Tabloid tricks – Angus Kidman

  • Journalists have secret lawyer fantasies, they have wrung the information out of the sources
  • Is the IT media biased?
  • no coverage of lca2012 in main Aus IT press
  • more coverage in 2nd tier
  • Only 3% pageviews on lifehacker use Linux
  • Not many IT journalists these days, One on FT on lifehacker, more to do (websites, blogs, video, podcasts)
  • Freelancers – can be an option. Freelance rates not good, getting worse, competition from bloggers, etc. Hard to convince editor that story is worthwhile.
  • IT news driven by fashion. In early 2000s belief among publications that Linux stories would drive traffic from slashdot etc
  • Current fashion is facebook and Apple. Stories about them in demand
  • Eg Lifehacker did apple angle on LCA keynote
  • Open source Projects lack definitive spokesperson
  • Media obsessed with cult of Trivia, Celebrity . eg Linus
  • Media not influenced by advertisers
  • Does Linux need media? Yes if want to reach more than the 3%
  • Identify the Audience. right publication
  • Be Fashionable. eg mention facebook or apple
  • Be Concise.
  • Be contactable. email, phone if in a hurry

January 18, 2012

Twitter Weekly Updates for 2012-01-18

SCALE 10x – there’s lots of MySQL there!

I’m just about to get on a plane to head to my inaugural SCALE event. It’s their tenth year running!

In a world filled with NoSQL related media, its kind of nice to see that on Friday January 20 2012, we have a MySQL room right next to the PostgreSQL room (schedule). It is awesome to see that the track will have participation from Oracle, Monty Program Ab, and SkySQL Ab.

On Saturday for the main tracks, I’ve got a talk about the growing MySQL diaspora (just got larger this year in case you haven’t paid attention to the packaged up Galera product!). This one is a constant work in progress and I’m hoping to complete research closer towards March ’12.

Monty Program and SkySQL are also sharing a booth in the expo hall, so come by booth #65 for some interesting schwag (t-shirts, poppers, etc.). Looking at the schedule lineup, I’m surprised I’ve never ever been to a SCALE before – looks totally awesome. See you in LAX (well, we’re so close-by the Los Angeles Airport :P)

Related posts:

  1. The SkySQL Reference Architecture
  2. Our booth is yours… Sun at OSCON
  3. Lots of database talk at Sun Tech Days

An open letter to Peter Ryan regarding police treatment of cyclists

Hon Peter Ryan,



I am writing because I am concerned at the number of recent incidents where a driver has collided with a cyclist, and the case hasn't been followed up by the police. Such incidents and the publicity surrounding them does nothing to encourage road users to obey the law when they realise that they will most likely get away with not doing so.



A week ago in Ballarat, a 13 year old boy was hit by a car, and the police said the boy had the right of way[1]. Despite this, the article linked states that the police will not charge the driver. This, despite her having broken Australian Road Rule 67 to 72, 84 or 86 depending on circumstances at the stated intersection, or perhaps 140 to 144 if travelling in the same direction. She was likely negligent in allowing the collision to happen in the first place, which, by my understanding, is a criminal offence, especially since there was serious injury involved. If she used the usual excuse that "she didn't see him", then that's an admission of guilt in failing to obey ARR 297 - driver having proper control of vehicle.



Also recently, there was a highly publicised case where Shane Warne had an altercation with a bicycle rider. In that case, the fact that Warne hit the cyclist from behind (ARR 126) after overtaking unsafely (ARR 144) is undisputed[2]. The fact that details were not exchanged following the collision is also undisputed (ARR 287). It is also well established that Warne was stopped unnecessarily in a bike lane (ARR 125; 153)[3]. And yet the police will not investigate[4].



Going back a number of years, I also have not had good experiences getting the police to follow up on cases. In my most recent case (11/10/2005; I do not know the case number sorry, all I know was that I was attended to by Angove & Auchterlonie from Boroondara police), the driver also failed to obey ARR 287 (as well as a slew of other offences, such as ARR 46 and 148 - changing lanes without indicating sufficiently and without due care). The police refused to prosecute the driver, and also would not hand over the driver's details or insurer details, based on some misguided privacy policy, asking me instead to fork out for a freedom of information request. Given that I was a broke student at the time, this was not a feasible thing to do and I never did receive compensation from the driver for damage to my bicycle, clothes, and large out of pocket expenses for travel to medical care for several years that the TAC didn't cover. The police also displayed a lack of knowledge of the law, initially thinking that I had broken ARR 141.



I can't imagine why the police aren't investigating these cases, because in each case, clear evidence is at hand, and not disputed. The identities of all parties are known. It should be an open and shut case. Without the police making charges, the rider in each case will have a much harder time claiming from the driver's insurance (if the boy was not admitted overnight, his TAC excess will be an enormous burden to his family). The driver in each case will not be discouraged from driving in a similar fashion next time. And other drivers also know that they will most likely get away with any offences they commit if a bicycle is involved. This is a perverse reversal of the situation that we should have, in which drivers should be encouraged to take due diligence around cyclists. It almost seems that the police always assume a cyclist is at fault unless proven otherwise in Australia, whereas most other countries with an established bicycling culture assume that the driver is at fault unless proven otherwise as they hold the burden of driving the more deadly vehicle and so should be required to take due care.



If the laws weren't adequate enough to prosecute to the driver in the above cases, has your department been contacted to update the laws, and what is being done? Keep in mind that cyclists have no protection other than by the law, and as the more vulnerable road user, the laws should focus on their safety and ensuring that transgressions are dealt with effectively.



Can you please encourage the police in each of these cases to follow them up to the full extent that the law currently allows.





Sincerely,





[1]

http://www.theage.com.au/victoria/teen-cyclist-struck-by-car-20120110-1ps85.html



[2]

http://theage.drive.com.au/motor-news/warnes-tirade-triggers-bike-rego-call-20120118-1q5k0.html



[3]

http://www.cyclingtipsblog.com/2012/01/cyclist-versus-warnie-the-cyclists-story/



[4]

http://www.heraldsun.com.au/news/more-news/warne-blasts-cyclists-on-twittershane-warne-clashes-with-cyclist-on-way-home-from-training-session/story-fn7x8me2-1226246735306

Synology DS109+

So far I am very impressed with the Synology DS109+ that I purchased. It’s currently running with a 2Tb WD Green drive.

Seems to do exactly what it was advertised to do too.

LCA2012 – Wednesday last session

The Samba tour of scripting languages – Amitay Isaacs and Andrew Bartlett

  • Samba is C based
  • But seems to have a lot of scripting
  • Has to be portable no non-gnu systems like solaris, reply on POSIX sh, make (not gun make), awk, m4 and a c compiler
  • shell scripts for first testing. Over 10,000 lines if shell in building
  • Python , TCL and Lua bindings all added but unpopular and eventually removed
  • Perl over 20,00 lines
  • IDL build initially in awk. Switched to perl based PIDL
  • javascript before it was cool – embedded javascript engine
  • But the cool kids were using python so switched from Javascript to python
  • Exception based languages things cleaner
  • waf is python based build system
  • python bindings for most things, or via C hook
  • Lots of other stuff being written into python, called directly by samba for small tasks
  • 90,000 lines of python
  • Example: Samba3 upgrade -  python based tool in 3 weeks. Business login in python, exceptions test for bad input
  • At build time python checks to see if ABI has changed from previously and alert developer if it has.
  • Test frameworks – unit tests on standalone components. Environmental tests with everything running, different types on server setups, fake ips and tests made. All runnable as non-root
  • 9000 test in 1300 test suites. Mosts test in C, some in python or shell
 

 

Running Python in Grub – Josh Triplett

  •  ” I ported python to grub “
  • perception that Linux doesn’t need BIOS
  • Involves programming hardware in functional/safe/optismise configuration
  • Lots of stuff to support, a few decades of compatibility, very bare-metal programming, small number of people working on it
  • What can go wrong: Broken or disabled CPU features, missing or broken memory, sub-optimal power mngt, delays & latency, USB bios handoff. Undocumented customer interfaces
  • Why not test under Linux? Linux gets in the way, no direct BIOS access for tests
  • Grub2 – 32 bit addresses, written in C, can read files, menus, single thread, no OS to disturb, only uses a bit of BIOS itself
  • ORigin: Replacing DOS test programs, test for power mngt, new grub commands for command-line.
  • Grub script language – bashish, no expressions, can just glue things together in menus, have to write lots of C
  • Ported CPython 2.7 to GRUB
  • Wrote a C/Posix compatibility layer for GRUB, floating point functionality via fdlibm , ported much of python standard library
  • Build C extension modules, added “bits” module to access platform functionality
  • ACPICA already in Linux for parsing ACPI. ported as a grub module with python module to access it.
  • bits passes grub command line to python for python to parse
  • FUSE for Python and GRUB.  added a python device reading python/foo invokes a python callback.
  • logging to in-memory buffer and sends to log in FUSE via system. Save to ACPI table and then OS can grab it later
  • SMP support in grub, pyton scripting in ring0, python modules for platform interfaces CPU, PCI, PCI-e, ACPI including decoding and method evaluation, logging, test suite evaluation
  • BITS test: power mngt configuration, perf optimisation, CPU config registers, SMI frequency/latency , USB handoff from BIOS to OS including effects on SMI and C states.
  • Used by BIOS vendors before shipping boards, BIOS problems actually get fixed!
  • http://biosbits.org/
  • Freakiest stuff I’ve seen so far this week

brendanscott

I have long felt that the copyright industry are their own worst enemy, at least in relation to litigation.  Instead of treating with Napster to find some accommodation, they destroyed it.  What happened as a result? Decentralised peer to peer, that’s what. Instead of having hubs as in Napster that they might negotiate with, they eliminated that point of control from the system.  Ditto subsequent litigation.  Each time they have eliminated options for themselves to be able to negotiate terms with an aggregator.  This is probably why they are so keen to make ISPs liable for their customer’s infringements.  Of course whatever ISPs do, customers will route around that as well.

At every stage, the industry has simply evolved a better infringer.  SOPA, should it be passed, will be no different.  Other countries or private ventures will set up their own DNS to route around those controlled by the US.



LCA2012 – Wednesday after Lunch

Ubuntu ARM – David Mandala

  • Targeted ARM v7 – first release April 2009
  • 7 different kernels in 10.10 . Improving with Linaro
  • OMAP 3 100% in main Linux kernel so easy to support.
  • Someday unified kernel onto ARM
  • Toshiba AC-100 netbook
  • 11.10 preview release of ARM server
  • Lots of work to get SMP and now 64 bit to work. Some code assumed they would never exists
  • Virtualisation support soon for server space
  • Main sense of ARM in server space is 10x saving in power eg ~50W vs ~5W
  • Lots of other stuff this guy was going too fast for me to keep up
 

 Helping your Audience learn – Jacinta Richardson

  • Conferences let you vary your level of intensity according to your energy
  • Conferences – no assessment
  • Training is different, all day, 6-8 hours, several days in row, builds on previous days
  • can’t afford to get lost.
  • Cognitive load – how much effort somebody has to apply to learn a new thing.
  • Intrinsic – how hard actually thing is.
  • Extraneous – how harder trainer makes it than it needs to be
  • Germaine – how well concepts build on what we already understand
  • Building framework takes time. Scaffolding has to be well designed. Lots of simply examples
  • Mind map of material -> what you are teaching to build foundation
  • Be realistic what you can fit into day (including breaks, people late)
  • 6 – 6.5 hours optimistic. Seems to be max most people can handle
  • 90 minutes then break ( eg 90min on, 30min off repeat 4 times )
  • Documents – short prose sections, short examples, short chapters (work though in 90 minutes)
  • Key info at start of the day. Dont do lots of extra stuff like class rules
  • Easy stuff, unimportant stuff at the end of the day.
  • Essentials at the start of the course
  • First 90 minutes or first day is most important
  • Options extras at end of course or end of each day
  • If you have lots of stuff -> create another course, make more money
  • Use diagrams, code, pictures, comics
  • reduce germane cognitive load:
  • Order carefully
  • group similar concepts
  • Put import stuff in bold
  • 10:10 – 10 minutes instructions, 10 minutes of student exercise. Sometimes 10:20 . Occasionally 10:30
  • 1-3 concepts in that 10 minutes. But try to balnce
  • Spare time = more examples
  • 90 minutes = 3 x 10:20 + 4.5 x 10:10
  • Target exercises at each key point. Doesn’t have to be real-world
  • 1 point = 1 exercise
  • Easy to advanced exercises. Additional exercises to really advanced people
  • NO answer files. Cause everybody will cheat
  • Minimise cross-chapter reliance
  • Sometimes you have to rely on previous stuff (should have been at start on day one). Try and avoid since people will have missed or not picked up previous concepts
  • New topic = clean slate
  • Good, through course notes
  • Not slides, write a book, should be readable later, good advertising
  • A few other ideas:
  • Keep room cold, keep it fresh. 21-23 degrees
  • Bell curse applies to student ability.  Students not slow, but have less foundation or experience in topics
  • Target average student. Offer extra help for ones behind. Don’t slow down for slowest student.

Save the date: 2012 Linux Security Summit, 30-31 August, San Diego

This is a pre-announcement so people can start planning travel for the year.

The Linux Security Summit for 2012 will be held on the 30th and 31st of August in San Diego, CA, USA.  It will be co-located with LinuxCon North America, plumbers and the kernel summit.

More details to follow.

January 17, 2012

LCA2012 – Wednesday Morning

A Tour of btrfs – Avi Miller

  • Now
  • All data and metadata is copy-on-write
  • CRC all metadata and data
  • Writable snapshots
  • multi-device support ( raid0 , raid1 , raid 10 )
  • online resize and defrag, online device replace
  • transparent compressions, efficient storage for small files
  • Soon
  • Fixes for perf and stability
  • background scrubbing, LZO compression, batched discard, file defrag options, per-inode flags
  • Larger block sizes for ( especially for metadata, to provide perf improvements in some cases)
  • Scrubbing uses CRC to varify data on disk, fixes bad ones with good copy on another disk which has okay CRC
  • “df ” gives completely wrong values on how full the disk is or it’s size
  • discard/trim supported both real time and batched
  • Drive Swapping – current raid rebuilds via balance code, can also restripe between RAID levels
  • btrs send/receive in development
  • Embedded – friendly to small machines, not as friendly to small disks (being worked on) Works well with low-end flash drives
  • RAID 5/6 – MErge pending completion of fsck work. will also add triple mirroring
  • beta read-only filesystem recovery tool – copies data out of corrupt FS
  • tree root-history log lets us recovery from many hardware errors. “mount -recover”
  • New fsck release on the way. May be announced very soon
  • yum-plugin-fs-snapshot . yum plug to trigger a snapshot on package install/upgrades
 

The Web as an application development platform – Shane Stephens and Mike Lawther

  • When and how to move from native to web/cloud apps
  • Examples
  • Text based like email (pine, mutt, outlook). 15 years ago web-based email started. access anywhere, no install required, easy to start using, use anywhere
  • Desktop publishing. Hard to collaborate with other people using desktop based software by emailing docs around to other people. On the web it’s often native multi-user. Send Links rather than emailing whole doc.
  • On web everybody always running latest version
  • Github gives you one-stop shop for projects. wiki, forums etc. Native in app rather than bolt on to desktop version
  • Graphics
  • Flickr – has native sharing, backups etc
  • Online editing of images etc now possible
  • Games
  • Farmville – no install, easy to share links
  • Angry Birds – has web version, html5 , flash for sound, 60fps
  • Not at stage where First person shooters going to happen yet. Users have high end hardware
  • Overall Benefits – No install, universal access to data, always using latest version, collaboration and sharing built in. Simple text layout, Web as IDE, open and modular enviroment
  • Drawbacks – Layout more involved than desktops apps, distributed code makes debugging hard, cross browser compatibility, security limits flexibility.
  • Useful web technologies (see also HTML5Rocks website) :
  • Display / Rendering
  • HTML , SVG , Canvas – All fairly easy to combine
  • WebGL , flexbox and grid <- future
  • Communications
  • Standard http requests, AJAX / XHR , Websockets / Browser Channel , libraries like Faye etc
  • Storage
  • Traditionally just cookies, Session Storage, Local Storage, indexDb, AppCache
  • Environment Enhancements
  • jQuery , CoffeeScript , NaCl

LCA2012 – Tuesday after Lunch

Mistakes were made by Selena Deckelmann

  • Misc management
  • Prepare for failure, Failure is an option (it will happen)
  • Book: “Everything is Obvious”
  • 2 examples (NZ and Scotland) of rats gnawing through cables and taking out country
  • Document -> Test -> Verify
  • Failure to Document
  • Write Docs, Update Documentation, Make documentation a step with your written processes, assign some time to that step.
  • Doc Tools: Graphic Designers, wikis, sphinx, diagrams  – timelines – bug tracking – ordered todo lists
  • Failure to test
  • Verify success criteria – Write tests – test with buddy – have a plan
  • testing frameworks, staging environment, repeatable shell scripts
  • Failure to verify
  • Have a plan for things going wrong – have staging environment – test rollback plan, not just implementation plan
  • Tools – People, staging environment
  • Failure to imagine
  • share stories of failure – talk to people are different from yourself – act out implementation scenario
  • Failure to Implement
  • reflection ( post-mortem )
  • Plan to do  post-mortem, document the plan with numbered steps and a timeline – test plan & rollback plan – Identify point of no return
  • During – screen sharing – chatroom – Voice – Headsets – Designated time-keeper
Scaling Openstack by  James Blair and Monty Taylor
  • 6 projects in openstack.
  • collection of related repositories
  • Most contributors paid to work on it by their companies
  • number , quality and area or contributors varies
  • 6 monthly releases – design summits – continuously open truck – dev on master – Monthly milestones – stable branches post release
  • Vision – consistent tooling and process on all projects -> Consistent Product -> Multiplier effect.
  • Minimize meta-development, Standard tools
  • Gerrit – code review
  • Jenkins – Testing (pre and post merge)
  • Orchestra (bare metal deployment)
  • Lanchpad, documentation servers, planet, repos
  • Environment: Ubuntu, Everything in Python (pep8 standard, openstack.common ). virtualenv/pip
  • Gated truck – ensure quality – auto tests – means devs always start from working code – keeps bad code out of tree – process same for everybody, transparent, automated.
  • Gerrit – stand-alone patch review system – lots of integration hooks – lots of review categories
  • SSI using openid for all of project sites
  • Git review is implemented as git sub-command to submit things to gerrit. zero-config <- looks cute
  • Vendors can have labs and tests and code can be automatically submitted and tested on it

Walktime Blog #16: Arduino Miniconf 2012

A quick walk-around of the hardware assembly tutorial at the 3rd Arduino Miniconf, which took place on January 16, 2012 as part of linux.conf.au.

View or comment directly on YouTube: www.youtube.com/watch?v=xic4w5DL-tw

Links for this ep:

LCA2012 Tuesday – Before Lunch

I’ll be posting some updates from Linux.conf.au in Ballarat in the next few days. I arrived on Sunday but this is my first post. I’ll see if I do an out-of-order blog post on that later.

The keynote from Bruce Perens was really good. He talked a lot about the failure of open source to engage the public and policy makers. He also covered a bunch of cool hardware projects and cheap (and often open) small boards and other electronic products.

Smashing a square peg into a round hole – David Basden and Chris Collins

  • Automation your automation.
  • Anchor – Hosting provider, doing built solutions, non-standard requirements
  • Puppet is one step, PXE & dbootstrap just another, hundreds of others
  • A “simple” build used to take a day, down to 10 minutes
  • instead of defining all steps define dependencies to get a “partial ordering”
  • figure out what has to be done, in what steps, keep track of what has been done
  • Lack of security in many queuing systems, bad agents can grab tasks they aren’t supposed to etc
  • Outline of “Audience” job control system. Design goals and decisions
  • Code on github.net/anchor

 Extracting metrics from logs for realtime trending and alerting by Jamie Wilkinson

  • passive system, query application about it’s metrics. However sometimes hard to parse
  • However most apps log to system or other logs
  • emtail – exporting modular tail
  • plugins – on event X do Y – usually regex
  • Metrics are values, times ( name, value, when, type (counter, gauge) , string, tag
  • metrics are exported over a common protocol ( google protocol in G version, json in open source version )
  • exported over http using json or CSV,  sent to something slse to save and do something with
  • Written in Go. Old version in python but too google specific
  • 20 minutes talking about the source code (I browsed the web during this bit)
  • I and a few others in the audience seemed to think this duplicated a lot of other tools with no obvious huge advantage over them. Bit of google NIH.
 

 

AdaCamp and Haecksen

AdaCamp

Saturday was the very first AdaCamp, an unconference set up by the Ada Initiative to talk about issues facing women in open stuff, here in Melbourne.

The event was well attended by awesome feminists from a breadth of fields and some geographic diversity within Australia (including a surprising number of people from Perth). There were open source people, Wikimedia people, librarians and academics.

There were 21 sessions in total, in 3 streams, so I don’t know everything awesome that was discussed. I think the most important session I attended was the first one of the morning on Imposter syndrome. Also the session on getting women involved with open source, where I talked about the GNOME Outreach Programme for Women.

[Unfortunately I forgot to bring a notebook with me, and tiredness has caused my memory to let me down.]

Brianna and I supplied the morning and afternoon tea, which was various combinations of vegan, gluten-free and fructose-free. Some people asked for the recipes used, which I blogged here.

Haecksen

I drove to Ballarat yesterday to give my panel at the Haecksen miniconf at linux.conf.au1 on gender-focused outreach in open source.

It was, in my opinion, an extremely good panel. I was originally daunted by appearing on a panel with Pia Waugh, Leslie Hawthorn and Selena Deckmann (all of whom are amazingly talented), but I think I did okay. We covered a breadth of mentoring from school age girls, to open source mentoring with Summer of Code and specific programmes like GOPW with a little bit about mentoring in organisations. Unfortunately I feel like the panel ran out of time just when it was getting interesting, and I didn’t get to talk about optimising your mentoring and the differences I’ve found in mentoring people from high-context culture vs low-context cultures [1, 2].

We also talked a bit about improving your diversity (which was a bit off-topic, more of a carry-on from the talk preceding the panel). Key point for me was: people from minorities tend not to ask (or apply), if you want diversity, you have to seek it out. Furthermore, people tend to know people who are like them, to seek out diversity, ask for recommendations from people who are least like the people you already have.

~

Unfortunately, I’m not at linux.conf.au itself, so if you’re looking for me, I’m not there. If you want to talk to a Collaboran about what we can do for you or how awesome it is to work for us, you can find Daniel Stone or Dario Freddi at the conference.

  1. Many thanks to my employer Collabora for letting me be there.

January 16, 2012

New git repository for the Linux kernel security subsystem

I’ve set up a new git repository for the Linux kernel security subsystem on the new kernel.org server.

The URLs are:

git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security.git

http://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security.git

https://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security.git

Developers should work against the “next” branch.

A web-browsable interface via gitweb may be found at:

http://git.kernel.org/?p=linux/kernel/git/jmorris/linux-security.git;a=summary

The temporary repo on selinuxproject.org will go away soon, so please update your repositories.

January 15, 2012

Twitter Weekly Updates for 2012-01-15

  • Bright Red Eucalyptus Gum Flowers http://t.co/CI9J6LXK #
  • Reflections on Reading a Newspaper (Kofi Beans cafe, Croydon) http://t.co/uWaJE3XR #
  • Clouds and Sun at Cardinia Reservoir (HDR) http://t.co/2WYpM8Z1 #
  • This mornings earworm is "Down In The Park" by Gary Numan – thanks @abcdigmusic (Foo Fighters cover is the best tho) #
  • "Getting hit by a bus is not something we've developed an evolutionary response to" – Karim Brohi, trauma surgeon, RLH #
  • Does any one know if the Dick Smith unlocked Samsung Galaxy Nexus comes with an Aussie warranty or is it a parallel import? #
  • My friend @AlecMuffett is trying to find a good #Android cycling GPS app: suggestions? http://t.co/1MrHuxzy #
  • Making morning drink, looked over and realised I'd lit different burner to one kettle was on. Sigh. #BootstrapProblem #
  • return -EWRONGTEABAG; /* This is NOT meant to taste of Fennel */ #
  • Raindrop rose – a rose in our garden after overnight summer rain http://t.co/gQYyuwwh #
  • Fire hydrant gushing water on Swanston St south of Lincoln Square, any ideas who needs to know? #MFB #
  • OK – time for coffee and then off to the DC's again for another exciting adventure in "Chris versus Hardware" #hpc #
  • Dear #Drupal when we update a page please display it to all visitors, not just to people who are logged in. KTNXBAI #
  • Hooray, so finally we have all #VLSCI #HPC systems back online. Merri and Bruce yesterday and Tambo today (after HW fixes) #
  • Missed this when the last #Top500 supercomputer list came out – #Microsoft is down to 1 (from 4) Windows only #HPC system #
  • The deed is done, Samsung Galaxy Nexus ordered from @MobiCity in QLD with 24 month warranty.. Android here I come! #
  • Great picture of 3,300 year old bronze age wooden boat being excavated in Cambridgeshire on cover of @CurrentArchaeo 263 #
  • Sigh, I hate inconsistent network issues, if I bang my head against it an longer I swear it's going to bleed.. #
  • If you don't teach kids how computers work + how to program, then they are doomed to be controlled and not be in control #
  • Another great piece of cappuccino art from Cafe Have Ya Bean in Upwey http://t.co/cEW0IaGF #
  • A new update on our year old RedHat RHEL5 Mellanox driver bug! "We are still waiting from Engineering.". Oh well.. #
  • Barcelona Supercomputer Center – where architecture is about buildings as well as #HPC systems http://t.co/MPck6v55 #
  • Very sad, a lead #SKA scientist possibly murdered in the UK http://t.co/B9kIG6bz (via @rcrain_astro) #
  • UPS reckon my Galaxy Nexus will take 6 days to get from Melbourne to Melbourne – it only took 24 hours from Hong Kong! #
  • Pondering realisation that my two great fascinations, archaeology and astronomy, are inherently about looking back in time #
  • Dick Smith won't price match a phone that DSE themselves *only* sell online because others don't have retail store – FAIL! #
  • FAWN: A Fast Array of Wimpy Nodes – http://t.co/qOUqgtAM (via Jim Lux of NASA on the Beowulf list) #hpc #
  • #VLSCI SGI Altix XE cluster "Bruce" should clock over it's 1,000,000th job this weekend! Not bad for less than 2 years. #
  • My Samsung Galaxy Nexus has arrived! #android #google http://t.co/bnDB2iCy #
  • Finally had a chance to turn on IPv6 via @Internode at home – hoorah! #
  • .@twitter mobile web interface is actually quite usable on Android ICS, no need for an app #
  • Android 4 ICS has native IPv6 support for Wifi, works like a charm. #
  • Odd, only the official Samsung cable for my Galaxy Nexus works for it to charge or be recognised in a PC. Need more tests. #
  • Aha, my trusty microUSB cable from Jaycar works in my Galaxy Nexus, phew! /cc @MobiCity #
  • "All code in Linux will evolve until the point where it's implemented as a filesystem." – @mjg59 http://t.co/KtONtiXV #
  • Lovely Melbourne day with @donna_williams lounging at friends place in Brunswick after long lunch. #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-01-15

Twitter Weekly Updates for 2012-01-15

  • Bright Red Eucalyptus Gum Flowers http://t.co/CI9J6LXK #
  • Reflections on Reading a Newspaper (Kofi Beans cafe, Croydon) http://t.co/uWaJE3XR #
  • Clouds and Sun at Cardinia Reservoir (HDR) http://t.co/2WYpM8Z1 #
  • This mornings earworm is "Down In The Park" by Gary Numan – thanks @abcdigmusic (Foo Fighters cover is the best tho) #
  • "Getting hit by a bus is not something we've developed an evolutionary response to" – Karim Brohi, trauma surgeon, RLH #
  • Does any one know if the Dick Smith unlocked Samsung Galaxy Nexus comes with an Aussie warranty or is it a parallel import? #
  • My friend @AlecMuffett is trying to find a good #Android cycling GPS app: suggestions? http://t.co/1MrHuxzy #
  • Making morning drink, looked over and realised I'd lit different burner to one kettle was on. Sigh. #BootstrapProblem #
  • return -EWRONGTEABAG; /* This is NOT meant to taste of Fennel */ #
  • Raindrop rose – a rose in our garden after overnight summer rain http://t.co/gQYyuwwh #
  • Fire hydrant gushing water on Swanston St south of Lincoln Square, any ideas who needs to know? #MFB #
  • OK – time for coffee and then off to the DC's again for another exciting adventure in "Chris versus Hardware" #hpc #
  • Dear #Drupal when we update a page please display it to all visitors, not just to people who are logged in. KTNXBAI #
  • Hooray, so finally we have all #VLSCI #HPC systems back online. Merri and Bruce yesterday and Tambo today (after HW fixes) #
  • Missed this when the last #Top500 supercomputer list came out – #Microsoft is down to 1 (from 4) Windows only #HPC system #
  • The deed is done, Samsung Galaxy Nexus ordered from @MobiCity in QLD with 24 month warranty.. Android here I come! #
  • Great picture of 3,300 year old bronze age wooden boat being excavated in Cambridgeshire on cover of @CurrentArchaeo 263 #
  • Sigh, I hate inconsistent network issues, if I bang my head against it an longer I swear it's going to bleed.. #
  • If you don't teach kids how computers work + how to program, then they are doomed to be controlled and not be in control #
  • Another great piece of cappuccino art from Cafe Have Ya Bean in Upwey http://t.co/cEW0IaGF #
  • A new update on our year old RedHat RHEL5 Mellanox driver bug! "We are still waiting from Engineering.". Oh well.. #
  • Barcelona Supercomputer Center – where architecture is about buildings as well as #HPC systems http://t.co/MPck6v55 #
  • Very sad, a lead #SKA scientist possibly murdered in the UK http://t.co/B9kIG6bz (via @rcrain_astro) #
  • UPS reckon my Galaxy Nexus will take 6 days to get from Melbourne to Melbourne – it only took 24 hours from Hong Kong! #
  • Pondering realisation that my two great fascinations, archaeology and astronomy, are inherently about looking back in time #
  • Dick Smith won't price match a phone that DSE themselves *only* sell online because others don't have retail store – FAIL! #
  • FAWN: A Fast Array of Wimpy Nodes – http://t.co/qOUqgtAM (via Jim Lux of NASA on the Beowulf list) #hpc #
  • #VLSCI SGI Altix XE cluster "Bruce" should clock over it's 1,000,000th job this weekend! Not bad for less than 2 years. #
  • My Samsung Galaxy Nexus has arrived! #android #google http://t.co/bnDB2iCy #
  • Finally had a chance to turn on IPv6 via @Internode at home – hoorah! #
  • .@twitter mobile web interface is actually quite usable on Android ICS, no need for an app #
  • Android 4 ICS has native IPv6 support for Wifi, works like a charm. #
  • Odd, only the official Samsung cable for my Galaxy Nexus works for it to charge or be recognised in a PC. Need more tests. #
  • Aha, my trusty microUSB cable from Jaycar works in my Galaxy Nexus, phew! /cc @MobiCity #
  • "All code in Linux will evolve until the point where it's implemented as a filesystem." – @mjg59 http://t.co/KtONtiXV #
  • Lovely Melbourne day with @donna_williams lounging at friends place in Brunswick after long lunch. #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-01-15

Twitter Weekly Updates for 2012-01-15

  • Sharing again for back to work crowd :) BEST XMAS POPPER THING EVER! Why did the chicken cross the road, Machiavelli. http://t.co/tgGyZzUy #
  • Field Of Dreams mushroom burger at Grill'd (Belconnen). Oh wow. #
  • Listening to cover of Tomorrow ny Killing Heidi in a cafe. I prefer the original, but it is quite sweet. #
  • .@graceless_me that's right, follow us to land of awesome! Hopefully we'll find it soon ;) /cc @ScottRhodie @techAU @Asher_Wolf @nanopunk #
  • This will get annoying pretty fast :/ #fracturedscaphoid #firstworldproblems I still loved dirt bike riding/jumping :) http://t.co/s5KIllmN #
  • In xray, likelihood of fracture = high. Wrist feels heaps better than after dirt bike accident, but still tender bone. #firstworldproblems #
  • Wow RT @Liberationtech: Combating Net Copyright Infringement While Protecting Open & Innovative Net http://t.co/7c2CIvWT #
  • Very excited about #lca2012 :) I had to delay coming early (house painting) but I arrive first thing Monday morning. Yays! #getyourgeekon #
  • Packing up ones house you find the strangest things :) #lca2012 #lca2003 http://t.co/Cukpw8MA #
  • .@JoAllebone Heh, I hear ya! In exactly the same boat. Trying to find somewhere to live in Canberra whilst doing 100 other things = hard! :/ #
  • Painting my house today, am EXHAUSTED and it is only day 1. Ceiling looks a million times better though :) Tomorrow, WALLS! :) #
  • Fascinating RT @nigroeneveld: How China Thinks About the Future of Cyberspace Conflict http://t.co/0UiKudjH #china #cyberwar #infosec #
  • .@patentology heh, yeah :) Be interesting to get actual gender stats on diff games, but AFAIK women are roughly half of gamers /cc @IDEALAW #
  • .@reijin64 My mum was a computer technician. Jokes about women/mothers & tech have always seemed rly dumb to me :) /cc @IDEALAW @patentology #
  • Implying? #gamer RT @IDEALAW: LOL RT @patentology: 'Angry Birds' more popular with women over 30 than 'Call of Duty'? Who would've thought?! #
  • SM still maturing RT @peterjblack: hmm this is interesting "Study: Social networkers have more ethics problems at work" http://t.co/R970OfGj #
  • Always awesome to meet inspiring clever ppl. Thx @mia_will for the catch up and the intro :) #
  • Interesting RT @sanchezjb: Hacked memo leaked: Apple, Nokia, RIM supply backdoors for gov intercepts? http://t.co/sNBchf23 #netfreedom #

Twitter Weekly Updates for 2012-01-15

  • Sharing again for back to work crowd :) BEST XMAS POPPER THING EVER! Why did the chicken cross the road, Machiavelli. http://t.co/tgGyZzUy #
  • Field Of Dreams mushroom burger at Grill'd (Belconnen). Oh wow. #
  • Listening to cover of Tomorrow ny Killing Heidi in a cafe. I prefer the original, but it is quite sweet. #
  • .@graceless_me that's right, follow us to land of awesome! Hopefully we'll find it soon ;) /cc @ScottRhodie @techAU @Asher_Wolf @nanopunk #
  • This will get annoying pretty fast :/ #fracturedscaphoid #firstworldproblems I still loved dirt bike riding/jumping :) http://t.co/s5KIllmN #
  • In xray, likelihood of fracture = high. Wrist feels heaps better than after dirt bike accident, but still tender bone. #firstworldproblems #
  • Wow RT @Liberationtech: Combating Net Copyright Infringement While Protecting Open & Innovative Net http://t.co/7c2CIvWT #
  • Very excited about #lca2012 :) I had to delay coming early (house painting) but I arrive first thing Monday morning. Yays! #getyourgeekon #
  • Packing up ones house you find the strangest things :) #lca2012 #lca2003 http://t.co/Cukpw8MA #
  • .@JoAllebone Heh, I hear ya! In exactly the same boat. Trying to find somewhere to live in Canberra whilst doing 100 other things = hard! :/ #
  • Painting my house today, am EXHAUSTED and it is only day 1. Ceiling looks a million times better though :) Tomorrow, WALLS! :) #
  • Fascinating RT @nigroeneveld: How China Thinks About the Future of Cyberspace Conflict http://t.co/0UiKudjH #china #cyberwar #infosec #
  • .@patentology heh, yeah :) Be interesting to get actual gender stats on diff games, but AFAIK women are roughly half of gamers /cc @IDEALAW #
  • .@reijin64 My mum was a computer technician. Jokes about women/mothers & tech have always seemed rly dumb to me :) /cc @IDEALAW @patentology #
  • Implying? #gamer RT @IDEALAW: LOL RT @patentology: 'Angry Birds' more popular with women over 30 than 'Call of Duty'? Who would've thought?! #
  • SM still maturing RT @peterjblack: hmm this is interesting "Study: Social networkers have more ethics problems at work" http://t.co/R970OfGj #
  • Always awesome to meet inspiring clever ppl. Thx @mia_will for the catch up and the intro :) #
  • Interesting RT @sanchezjb: Hacked memo leaked: Apple, Nokia, RIM supply backdoors for gov intercepts? http://t.co/sNBchf23 #netfreedom #

New Digital Camera Canon DSLR 600D

Wife and I have been reviewing DSLR’s for a while now. We settled on a Canon 600D twin lens kit. As this was the model that had the features we wanted.

Took a few photos with it and very impressed so far. Now all I need to do is to understand all the features. Might have to do a bit of a course on how to best use it.

January 14, 2012

ODF/C++ Job seeking; br; see you at LCA...

If anyone knows of ODF(+RDF?), Qt, KDE, C++ jobs floating around I'm headed back into that market. It would be fabulous to continue to hack on FOSS code ;)



In other news, if you are at LCA this week then you might like to drop by to see Martin and my talk on Abiword and RDF stuff.

Lake Wendouree

I recommend visiting Lake Wendouree at least once during your LCA2012 free time.

IMG_3204 IMG_3205 IMG_3206 IMG_3208 IMG_3209 IMG_3210 IMG_3212 IMG_3213 IMG_3215 IMG_3217 IMG_3218 IMG_3219 IMG_3220

Debugging OpenWRT routers by shipping logs to a remote syslog server

Trying to debug problems with consumer-grade routers is notoriously difficult due to a lack of decent debugging information. It's quite hard to know what's going on without at least a few good error messages.



Here is how I made my OpenWRT-based Gargoyle router send its log messages to a network server running rsyslog.



Server Configuration

Given that the router (192.168.1.1) will be sending its log messages on UDP port 514, I started by opening that port in my firewall:

iptables -A INPUT -s 192.168.1.1 -p udp --dport 514 -j ACCEPT
Then I enabled the UDP module for rsyslog and redirected messages to a separate log file (so that it doesn't fill up /var/log/syslog) by putting the following (a modified version of these instructions) in /etc/rsyslog.d/10-gargoyle-router.conf:

$ModLoad imudp

$UDPServerRun 514

:fromhost-ip, isequal, "192.168.1.1" /var/log/gargoyle-router.log

& ~
The name of the file is important because this configuration snipet needs to be loaded before the directive which writes to /var/log/syslog for the discard statement (the "& ~" line) to work correctly.



Router Configuration

Finally, I followed the instructions on the Gargoyle wiki to get the router to forward its log messages to my server (192.168.1.2).



After logging into the router via ssh, I ran the following commands:

uci set system.@system[0].log_ip=192.168.1.2

uci set system.@system[0].conloglevel=7

uci commit
before rebooting the router.





Now whenever I have to troubleshoot network problems, I can keep a terminal open on my server and get some visibility on what the router is doing:

tail -f /var/log/gargoyle-router.log

A Simple Telnet Client Using Data.Conduit.

Below is a simple telnet client written using Haskell's new Conduit library. This library was written by Michael Snoyman the man behind the Yesod Web Framework for Haskell.

The Conduit library is a second generation approach to the problem of guaranteeing bounded memory usage in the presence of lazy evaluation. The first generation of these ideas were libraries like Iteratee, Enumerator, and IterIO. All of these first generation libraries use the the term enumerator for data producers and iteratee for data consumers. The new Conduit library calls data producers "sources" and data consumers "sinks" to make them a little more approachable.

The other big difference between Conduit and the early libraries in this space is to do with guaranteeing early clean up of potentially scarce resources like sockets. Although I have not looked in any detail at the IterIO library, both Iteratee and Enumerator simply rely on Haskell's garbage collector to clean up resources when they are no longer required. The Conduit library on the other hand uses Resource transformers to guarantee release of these resources as soon as possible.

The client looks like this (latest available here):

  import Control.Concurrent (forkIO, killThread)
  import Control.Monad.IO.Class (MonadIO, liftIO)
  import Control.Monad.Trans.Resource
  import Data.Conduit
  import Data.Conduit.Binary
  import Network (connectTo, PortID (..))
  import System.Environment (getArgs, getProgName)
  import System.IO


  main :: IO ()
  main = do
      args <- getArgs
      case args of
          [host, port] -> telnet host (read port :: Int)
          _ -> usageExit
    where
      usageExit = do
          name <- getProgName
          putStrLn $ "Usage : " ++ name ++ " host port"


  telnet :: String -> Int -> IO ()
  telnet host port = runResourceT $ do
      (releaseSock, hsock) <- with (connectTo host $ PortNumber $ fromIntegral port) hClose
      liftIO $ mapM_ (`hSetBuffering` LineBuffering) [ stdin, stdout, hsock ]
      (releaseThread, _) <- with (
                            forkIO $ runResourceT $ sourceHandle stdin $$ sinkHandle hsock
                            ) killThread
      sourceHandle hsock $$ sinkHandle stdout
      release releaseThread
      release releaseSock

There are basically three blocks, a bunch of imports at the top, the program's entry point main and the telnet function.

The telnet function is pretty simple. Most of the function runs inside a runResourceT resource transformer. The purpose of these resources transformers is to keep track of resources such as sockets, file handles, thread ids etc and make sure they get released in a timely manner. For example, in the telnet function, the connectTo function call opens a connection to the specified host and port number and returns a socket. By wrapping the connectTo in the call to with then the socket is registered with the resource transformer. The with function has the following prototype:

  with :: Resource m
       => Base m a             -- Base monad for the current monad stack
       -> (a -> Base m ())     -- Resource de-allocation function
       -> ResourceT m (ReleaseKey, a)

When the resource is registered, the user must also supply a function that will destroy and release the resource. The with function returns a ReleaseKey for the resource and the resource itself. Formulating the with function this way makes it hard to misuse.

The other thing of interest is that because a telnet client needs to send data in both directions, the server-to-client communication path and the client-to-server communication run in separate GHC runtime threads. The thread is spawned using forkIO and even though the thread identifier is thrown away, the resource transformer still records it and will later call killThread to clean up the thread.

The main core of the program are the two lines containing calls to sourceHandle and sinkHandle. The first of these lines pulls data from stdin and pushes it to the socket hsock while the second pulls from the socket and pushes it to stdout.

It should be noted that the final two calls to release are not strictly necessary since the resource transformer will clean up these resources automatically.

The experience of writing this telnet client suggests that the Conduit library is certainly easier to use than the Enumerator or Iteratee libraries.

Breaking windows

Another letter in The Age today. Unedited text below:





Ian Porter (Without car manufacturing, we are on the road to ruin, The Age, 13 Jan) believes that the government needs to keep throwing money at the car industry in order to support other industry in Australia. I'm surprised as an industry analyst, he hasn't heard of the broken window fallacy.



Throwing good money after a bad unsustainable industry that can't adapt is just a waste. It's exactly identical to sending soldiers to dig holes only to fill them back up again just to keep them employed and off the streets. The money could be better spent on doing useful things that will remain useful into the future. Yes, paying people to break windows and then paying the glazier to repair them will keep people employed, but couldn't the glazier be better employed building things that then keep other people employed into the future?



Why don't we do something useful with the money instead? Like built modern intra- and inter-city rail infrastructure? This won't become stranded assets when cheap oil becomes unavailable. We won't be left with vast tracts of useless motorways - we will continue to be able to use the rail infrastructure well past these boom times.

January 13, 2012

In Ballarat

I safely arrived in Ballarat for LCA2012 last night.

Was a long car trip — almost 12 hours from start to finish — and very stressful. After going through Melbourne (in which I took the wrong exit twice and nearly three times, got beeped at several times, and had some stressful merging experiences while going through roadworks) I was shaking nearly until Ballarat.

I must say, the bit where you go downhill on the Western Freeway where you get a glimpse of Bacchus Marsh gets the award for the most scenic shot of the trip.

Volunteer training begins this morning. Going to wolf down some breakfast and head over!

What a week

Been a busy week, and I am sure next week wont be no different.

Got Mac Mini back from Apple, and they replaced the logic board. In addition my 8gb of ram I ordered back in late November arrived from OWC, so I’ve since installed it in the Mac Mini. It hasn’t crashed or kernel panic since doing so. Guess the logic board might of been the cause of all the problems.

I said to my wife I wasn’t going to buy anything and well that lasted all of about 3 days, what can I say. Got a email from epowermac. Turns out they had some clearance stuff being sold. I browsed and found they had a Synology DS-109+ for $128 delivered (minus drive). Upon further research determined my 2Tb drive was on compatible list. Although this unit is discounted it has more ram than the model that replaced it. So purchased it. Was too good to pass up.

Also, the LPIC-2 book I purchased arrived. So now I have something to read over and revise. Although a friend mentioned that this Sybex LPIC-2 book I got is crap. He probably is correct too, as I saw an electronic copy which read oddly too. Oh well it’s a handy reference for the train trip (since I dont seem to get much chance to use the iPad). LPIC-2 Exam 201 target approx 5-6 weeks time.

January 12, 2012

Outstanding talks to take centre stage at linux.conf.au 2012

Ballarat, 11 January 2012

With only four days to go until the opening of Ballarat's linux.conf.au 2012, the conference is quickly becoming the talk of the town. Naturally it is the talks themselves that make the conference and 2012 gives delegates more chances to see more talks!

As Conference Director Josh Stewart explained, Delegates are sure to be impressed with the array and calibre of Speakers.

"We're really fortunate this year to be able to bring so many renowned community luminaries to Ballarat to share their experience and knowledge with delegates."

Tuesday will see Co-Founder of the Open Source Initiative (OSI) and creator of the Open Source Definition, Bruce Perens, given the first conference keynote. Other highlights from Tuesday include Jonathan Corbet's now-traditional kernel report, presentations on optimising web performance by Lenz Gschwendtner and even a talk on high altitude ballooning by Joel Stanley and Mark Jessop. Laura Thomson and her colleagues from Mozilla will also present on migrating enormous volumes of data.

Australia's favourite mad scientist, Perl developer Paul Fenwick, will keynote Wednesday's line up. He will be followed by, among others, Alice Boxhall and Silvia Pfeiffer presenting on developing accessible web applications, a shell tutorial by University of New South Wales computer scientist Peter Chubb and a talk on mentoring in free and open source (FOSS) communities by Open Source Outreach Manager for Oregon State University’s Open Source Lab, Leslie Hawthorn. Linux Australia's Annual General Meeting (AGM) will also be held as part of Wednesday's schedule.

On Thursday, open source luminary, ex-Software Freedom Law Centre counsel and current Executive Director of the GNOME project, Karen Sandler will keynote. Other highlights from Thursday's programme include a presentation on the Android open accessory development kit from open hardware guru Jon Oxer, talks on copyright and ending software patents by Ben Powell and Ben Sturmfels. Founders of the Ada Initiative - a group dedicated to furthering the representation of women in open technology and culture - Valerie Aurora and Mary Gardiner will also present around women in open source.

Thursday night's Penguin dinner will also play host to an as yet unannounced mystery speaker!

Significant interest surrounds Friday's keynote speaker, Jacob Appelbaum (better known as @ioerror on Twitter). Widely noted for his interest in computer security and stance against censorship, Jacob was one of the team behind exposing vulnerabilities in hardware after physical shutdown - the so called 'cold boot' attack.

Part of the final day of the conference, Friday 20th January, has been dedicated to showcasing 'best of' talks. A common lament of linux.conf.au delegates from years past has been the disappointment in missing 'the presentation' that everyone is talking about. The Best-Of slots allow for these amazing talks to be experienced by as many delegates as possible (or even experienced again for those who were blown away the first time around).

linux.conf.au will be held 16th-20th January 2012 in Ballarat, Victoria, Australia. Registrations are still open, but places are now limited.

Media enquiries are warmly welcomed to media@lcaunderthestars.org.au

January 11, 2012

Twitter Weekly Updates for 2012-01-11

New Purser Explores The World Episode is out

Man it's been a little while since I last posted here, need to fix that.

First up though, there is a new episode of Purser Explores The World out:

Ice Hockey in the land of Sun and Surf

Australia is famous for its sporting prowess. Everyone knows that when it comes to sports such as Cricket, Rugby (of either variety) and Swimming, Australia is up there with the best in the world.

However what people may not be aware of is that Australia, the land of Sun and Surf is home to a thriving and competitive Ice Hockey community.

I'm going to take you into the world of the Australian Womens Ice Hockey competition and we're going to find out what it takes to be a player on a team that can challenge European nations.

Links will be put up in the Angry Beanie Reddit as usual

Audio:

MP3

OGG

Torrent:

MP3

OGG

[life] Breaking and entering, with permission

I had a bit of an adventure yesterday, which would have taken some explaining if the police had gotten involved. It went a little something like this...

My friend and former co-worker Sara was in the US Virgin Islands for the holidays. Her boyfriend, Karl, flew there separately for the tail end of her time there.

Yesterday, I received a phone call from Sara, saying that Karl had managed to fly out to the Virgin Islands without his passport. Apparently you can get there without one, but to get back into the mainland US, you need one. She wanted to know if I could get one of my lock-picking co-workers to break into their apartment and retrieve Karl's passport and mail it them. Karl was supposed to fly out the next day. Attempts by Sara to contact her landlord had failed, so they didn't have many other options (apart from mailing me a key, which would have cost them another day).

I asked one of my co-workers, Jason, who I knew was into lock picking, if he was up for it, and he offered to put me in touch with another guy who had dominated the recent lock picking night that he'd run.

So now I'm talking to David, who's on board with the mission, but doesn't have his lock picking gear on him. No problem, Jason says he'll lend me his, which was at work with him. So we have a plan.

Our friends Ian and Melinda are currently in Australia. They've lent us their car because it's leased, and they have some minimum mileage they're supposed to do and they're under it, so I've been driving to work in their car some days. As it happens, I drove to work in it yesterday.

So now David and I set out in a car that neither of us own, with a lock picking set that belongs to another person, to break into an apartment of someone who's in the Virgin Islands. What could possibly go wrong?

I'm told that it's not illegal to own a lock picking set, but if you're caught with one on your person and you're not a locksmith, you can get into all sorts of trouble. On top of that I'd have a hard time explaining the car I'm driving.

We get to Sara and Karl's condo complex. It has a common gate that visitors would normal get buzzed through. Turns out it's not that hard to climb over. It's got some benign-looking spiky things on top, but I could get a leg over from the left hand side of the gate and jump over without impaling myself. Then I let David in and we proceeded upstairs to Sara and Karl's apartment door, where David set to work.

Sara said that just the dead bolt was locked. David started at it with Jason's tools, trying to be as discreet as possible. It was about 3:30pm and there was no one around, but we could hear some noises from the neighbouring apartment (the two front doors were right next to each other).

After what felt like about half an hour without success (the last pin of the lock was particularly tricky apparently) David was having to resort to more noisy techniques with the lock, so I decided to take the up-front approach and just inform the next door neighbour what we were doing in case he/she (I think it was a she) decided to call the cops on us. I told her through the door why we were there and what we were doing. She didn't seem to care too much.

David then proceeded to start "raking" the lock, essentially brute forcing the pins with a lot of jiggling, and finally managed to pick it and we were in. I quickly found Karl's passport where it was suspected to be, and then we pondered how we were going to lock the door again.

We could have just locked the door knob instead of the deadbolt and closed the door behind us, but we weren't sure if Sara and Karl had a key to the doorknob (Sara said they always just locked the deadbolt). Sara was fine with leaving the door unlocked until they got home, but weren't so keen on leaving our fingerprints all over the place and then leaving the door unlocked.

David tried to re-pick the deadbolt so that he could lock it via the same means as opened it, and I scouted around for a key. I managed to find a key that locked both the deadbolt and the doorknob, so I took that with us and locked up their apartment. In David's defense, the deadbolt was a bit stiff to lock even with the key.

I dropped David back at work, collected my stuff (it was now about 4:30pm) and headed to the UPS Store to ship Karl's passport to him as fast as humanly possible. I just made the 5pm pick up.

Today I received an SMS from Sara informing me that they had received the passport. I was very impressed with how fast it got to them.

So that was all a bit of an adventure. I'm not sure how much longer Karl is going to have to stay in the Virgin Islands as a result. I'm going to suggest that Sara and Karl leave a spare key with someone in future.

Podcasting at Conferences

For podcast hosts out there that attend conferences, especially those that only do Face-to-Face interviews like us, I’d so recommend making the most of these moments.

You get to speak to people you wouldn’t normally come in contact with, plus you get to ride on the coat-tails of the buzz that the conference creates. 

Craig and I have experienced this recently attending Open Source Developers Conference and YOW, recording for our podcast Coding By Numbers. We recorded 3 at OSDC and 2 at YOW, and so far they are own highest downloaded episodes. 

The 3 from OSDC are:

Just a few tips from me:

  1. Get the organiser’s permission. You probably don’t need it, but it’s always polite to do so. It also helps with number 2
  2. Ask the organisers if there is a quiet room you can use, they usually have a breakout rooms or unused conference rooms that you can loan for an hour
  3. Make contact with your interviewees early in the conference and try and do the interview after their talk so they are in that euphoric post talk mood!
  4. Make sure your interviewees understand when the podcast will go out and what the licence on it is. Also tell them if you are going to edit or not (we don’t)
  5. Most importantly - use social media and buzz to your advantage. If people are raving about a particular speaker that you are going to interview (in our case this was Mike Lee at Yow) then make sure people know about it and give them the link. When you finally post the show, make sure you use the conference hashtag and follow up with the people that you originally talked about. They will be your evangelists.

January 10, 2012

PyCon Australia 2012 – Call for Topics now open

In case you missed the news, PyCon Australia is being held on August 18th and 19th in Hobart, Tasmania. Like the first two PyCon Australia conferences, 2012 hopes to be full of presentations, tutorials and panel sessions from experts and core developers of the Python programming language, as well as the Python libraries and frameworks that you rely on for your work.

It’s time for us to start shaping the conference programme for 2012, and we need your help. We want to know what topics you want to see covered at PyCon Australia, or which presenters you think can help make our conference perfect for you.

If you’re already convinced, pop over to http://tinyurl.com/pyconau2012-cft and fill out our Call for Topics form. If not, read on!

Oh, still reading? Let see…

PyCon Australia is running a Call for Topics. This is like the reverse of a traditional Call for Proposals: instead of proposing a presentation, you can propose a topic that you’d like to see a presentation on at the conference, or possibly a presenter that you really want to see present. PyCon US have been doing this for a long time, it helps ensure that their conference attracts the best possible presenters. There’s a couple of reasons why you should help us out:

See the presenters you want to see

We’re planning on putting out our usual call for proposals in February 2012, but we need to make sure that the best possible presenters submit proposals to PyCon Australia. Our delegates, like you, want to enhance their skills in Python with every session that they attend. Our CfP can’t reach everyone, and even then not everyone who sees the CfP will think that they’re good enough to present at a conference — getting an invitation to present can be a pretty good motivator!

Learn about the tools that you want to use

One great reason to come to a Python conference is to increase your skillset in the tools and frameworks that you use in your day-to-day work. Perhaps there’s a new library that you’re considering using? Nominating it as a presentation topic for PyCon Australia will increase the chances it being covered in the conference. If you don’t know of an expert in the field, don’t worry. We can find one.

Heard enough?

Great! We can’t want to hear your suggestions. Just head over to our call for Topics form, and send in your ideas. Every idea can help make this conference perfect for you.

linux.conf.au 2012 set to shine in Ballarat

Ballarat, 11th January 2012

With only days to go until the covers come off 2012's linux.conf.au at the University of Ballarat, excitement is definitely mounting. The conference will welcome four stellar keynote speakers, including internet freedom and anti-censorship campaigner Jacob Appelbaum, GNOME Executive Director Karen Sandler, noted open source campaigner Bruce Perens and Australia's favourite mad scientist Paul Fenwick.

Preparations are in full swing. Conference Director Josh Stewart is particularly pleased with progress to date.

"The Organising Team are an amazing group of people - they're just so dedicated. Everything is coming together well, and we're looking forward to a hosting a conference that is thought provoking, entertaining and most importantly of all, great fun".

One of the challenges the conference team faced in Ballarat was ensuring that delegates had no troubles reaching the venue, with the city being located around an hour out of Melbourne. Stewart was confident that this focus paid off

"We've organised buses to ferry people both from Melbourne Airport and from Ballarat Train Station to the University and other accommodation venues. We have a conference Wiki page to allow people to arrange shared rides, hangout with other delegates on the train or find out the best way to get from A to B".

The first day of the conference, Monday 16th, will play host to eight Mini conferences - sessions focussed around specific topics. The 2012 Miniconferences include:

  • Sysadmin
  • Open Programming
  • High availability and distributed storage
  • Arduino open hardware
  • Business of Open Source
  • Music and Multimedia
  • Browser Miniconf
  • Business of Open Source
  • Haecksen / LinuxChix

The remaining four days of the conference will see over 70 presentations by notable speakers in the Linux and free and open source software (FOSS) community. Industry luminaries including Jonathan Corbett, Florian Haas and Jonathan Oxer are all confirmed.

"The Speaker line up is simply amazing. We've got something for everyone, ranging from deeply technical tutorials to presentations covering legal aspects of open source software and hardware" noted Stewart.

linux.conf.au will be held 16th-20th January 2012 in Ballarat, Victoria, Australia. Registrations are still open, but places are now limited.

Media enquiries are warmly welcomed to media@lcaunderthestars.org.au

PyCon Australia 2012 starts here

PyCon Australia

So one thing I forgot to mention on this blog is that I’ve taken over the reins of PyCon Australia for the 2012 and 2013 conferences. After spending two formative years in Sydney, under the direction of Tim Ansell, Richard Jones et al., we’re taking the conference south to Hobart, Tasmania. We’ve got a great team, consisting of myself, Joshua Hesketh and Matthew D’Orazio, and our papers committee is being led up once again by Richard Jones.

So, what can you look forward to? Well, here’s what we know so far.

Wrest Point

We’re holding PyCon Australia around the weekend of August 18 and 19 2012. Our venue is the Wrest Point Convention centre in Sandy Bay.  We’re really excited about our choice of venue — as well as offering us perfectly-sized rooms for our conference, the wide variety of spaces in the complex allow us to bring all of the traditional PyCon Australia events — CodeWars, the sprints and the conference itself — under the same roof for the first time.

Wrest Point is situated on the shoreline of the River Derwent, and this not only admits excellent views from the conference venue, but will also enable us to run some truly memorable social events, including the conference dinner, which we hope to share more details about shortly.

Our venue also lets us offer accommodation across a very wide range of budgets (starting around $124/room/night) to our delegates — this is not just a nominated conference hotel, it’s in the same building complex as the conference venue. This means that delegates can stay on-site for the entirety of the conference.  We think this will prove very popular, especially amongst delegates sticking around for the conference sprints.

Hobart & Wrest Point

For students and those travelling on a budget — we plan on keeping the conference affordable: there’ll still be heavily discounted student tickets, and we’ll announce budget accommodation options when registration opens.

Finally, you might be wondering how you can help make PyCon Australia the perfect conference for you? Well, in the coming week, we’ll be opening a Call for Topics.  This is an opportunity for you, as a potential PyCon Australia delegate, to nominate both topics and presenters that you’d like to see at the conference.  By nominating presentations, you can help ensure that PyCon Australia can help you enhance your skills and increase your knowledge of Python.

Of course, if you have something that you could present at PyCon Australia, we’d love to hear from you as well.  We’ll be opening a traditional call for presentations during February.

So, that’s it for now.  I’ll be sure to keep you up-to-date on our progress as we seek to put on the best Python Conference that Australia can offer. If you’ve got something to ask, feel free to drop a comment, either here, on Twitter, or on our Google+ page — we’ll get back to you as quickly as possible!

(Photos: “Wrest Point” by JJ Harrison, CC-BY-SA; “View of Hobart CBD” by Aaroncrick, CC-BY-SA)

January 09, 2012

RDF, Abiword and Relations

Abiword now has growing GUI support for editing RDF in ODF documents. Much of this support compliments what is available in Calligra for RDF handling. There are some areas where Calligra has more features and some areas where Abiword now does. Hopefully both will continue to have a large and growing shared feature base.



As some folks will know, ODF allows one or more RDF/XML files to be shipped in the ODF file, and for that RDF to be linking to the document content from the content.xml file. This means that you can explicitly say that a 1/2 inch bolt is from a particular maker and was procured on the 3rd of January 2012 by going to their office at geolocation ?x. Handy when you are reading the document a year down the line and want to know which office you bought the bolt from and the exact length of its thread.



Looking at the below image, one can see the purple underlined pieces of text. Each of these has some RDF associated with it. The citation to Dan Brickley has both contact and location RDF associated with it. Looking at the toolbar, towards the right side you see an "R" to start a change tracked document, and the "RF" button. Sorry about the images there, I draw at the 3 year old level so my icons are not quite polished shall we say. Anyway, the "RF" button selects this reference to an RDF item. So if you are between the "am" in the purple "James" then the whole word will be selected with this button. The ">" and "<" buttons then allow you to move to the next and previous reference to the selected RDF item. As you notice another purple James later in the document, this is the second and only other reference to him and you can move between them using these buttons.







These purple links I call "RDF links", which is a bit of a play on hyperlinks or bookmarks. Behind the scenes they are implemented using xml:id values and pkg:idrefs to those from RDF.



A feature added in the last days is the ability to capture and navigate by the relation between two RDF links. This is currently done by selecting a "source" link and then clicking another rdf link and setting the relation to the source. So in the below screenshot I am saying that Mark foaf:knows James. You will also notice the "Find by Relation" option which I can then use to see the people that mark foaf:knows. In the spirit of the foaf definition, I have made this a symmetric relation. So there is no stalking, or "following", if James knows Mark then Mark knows James. Asymmetric relationships are also possible, like son, child, or contains. I am hoping to add this feature to Calligra too in the future as the relations between RDF objects is one of the more powerful features that can be offered by using RDF in a document.







Note that the prev/next RDF item buttons work with relations. If I pick mark and navigate to the James he knows then I can "next" from that james to select the second reference to James. This is one recurring theme to RDF in ODF, that RDF objects like contacts can be cited or linked zero or more times in the text content of the document.



As mentioned above, the converse is also true, and the Dan Brickley text has two logical RDF objects linked to it; Dan's contact information and his location. Handling this multi-object for a single site is a little tricky and in this editing will create a window with both the semantic objects in it to let you edit the RDF abiword knows about. Note that this dialog is actually backed by two (or more) SPARQL queries;







If you are not squeamish about your triples then the "Show RDF" option for an RDF link will let you get right at them and edit away as shown below. There are a few technically cool things about this dialog: firstly the "Restrict to RDF Link" combo box lets you select one or more RDF links that the triples will have to be associated with, and secondly abiword makes sure any edits you make are properly linked to the RDF link you right clicked on. What I mean by this last bit is if you right click the RDF link "alice33" add a new triple "uri:alive myvocab:likes uri:bob" then abiword will add the triple "uri:alice pkg:idref alice33" for you. This is sort of having abiword do what you mean in that you want the new triple to be associated with the link but don't necessarily want to have to explicitly say it all the time. By choosing to edit the RDF for the link you have already explicitly said once that you want these things to be linked. This also applies if you change a subject, uri:alice to uri:amanda will update the pkg:idref values for you. Keep it linked, keep it valid.







Going one level deeper, the above dialog is actually a subclass of a restricted RDF model created using SPARQL. The SPARQL model is read only, and the subclass, RDFModel_XMLIDLimited handles mutations by creating a wrapper object which takes care of the automatic triple relinking mentioned above. Those still awake might like to see the abiword trunk code for src/text/ptbl/xp/pd_DocumentRDF.cpp.



This is part of an ongoing mad hacking sprint that is leading up to a talk at LCA 2012 which starts in a week. Many of the things I mention here are not in trunk yet, and only tested on Linux/GTK+3. Those in Ballarat in a week might like to pop in to the talk given my Martin and myself on Friday the 20th.

January 08, 2012

My First Cruise

A few weeks ago I went on my first cruise, from Sydney to Melbourne on the Dawn Princess. VacationsToGo.com (a discount cruise/resort web site) has a review of the Dawn Princess [1], they give it 4 stars out of a possible 6. The 6 star ships seem to have discount rates in excess of $500 per day per person, much more than I would pay.

The per-person rate is based on two people sharing a cabin, it seems that most cabins can be configured as a double bed or twin singles. If there is only one person in a cabin then they pay almost double the normal rate. It seems that most cruise ships have some support for cabins with more than two people (at a discount rate), but the cabins which support that apparently sell out early and don’t seem to be available when booking a cheap last-minute deal over the Internet. So if you want a cheap cruise then you need to have an even number of people in your party.

The cruise I took was two nights and cost $238 per person, it was advertised at something like $220 but then there are extra fees when you book (which seems to be the standard practice).

The Value of Cruises

To book a hotel room that is reasonably comfortable (4 star) in Melbourne or Sydney you need to spend more than $100 per night for a two person room if using Wotif.com. The list price of a 4 star hotel room for two people in a central city area can be well over $300 per night. So the cost for a cruise is in the range of city hotel prices.

The Main Dining Room (MDR) has a quality of food and service that compares well with city restaurants. The food and service in the Dawn Princess MDR wasn’t quite as good as Walter’s Wine Bar (one of my favorite restaurants). But Walter’s costs about $90 for a four course meal. The Dawn Princess MDR has a standard 5 course meal (with a small number of options for each course) and for no extra charge you can order extra serves. When you make it a 7 course meal the value increases. I really doubt that I could find any restaurant in Melbourne or Sydney that would serve a comparable meal for $119.

You could consider a cruise to be either paying for accommodation and getting everything else for free or to be paying for fine dining in the evening and getting everything else for free. Getting both for the price of one (along with entertainment etc) is a great deal!

I can recommend a cruise as a good holiday which is rather cheap if you do it right. That is if you want to spend lots of time swimming and eating quality food.

How Cruise Companies Make Money

There are economies of scale in running a restaurant, so having the MDR packed every night makes it a much more economic operation than a typical restaurant which has quiet nights. But the expenses in providing the services (which involves a crew that is usually almost half the number of passengers) are considerable. Paying $119 per night might cover half the wages of an average crew member but not much more.

The casino is one way that the cruise companies make money. I can understand that someone taking a luxury vacation might feel inclined to play blackjack or something else that seems sophisticated. But playing poker machines on a cruise ship is rather sad – not that I’m complaining, I’m happy for other people to subsidise my holidays!

Alcohol is rather expensive on board. Some cruise companies allow each passenger to take one bottle of wine and some passengers try to smuggle liquor on board. On the forums some passengers report that they budget to spend $1000 per week on alcohol! If I wanted a holiday that involved drinking that much I’d book a hotel at the beach, mix up a thermos full of a good cocktail in my hotel room, and then take my own deck-chair to the beach.

It seems that the cruise companies specialise in extracting extra money from passengers (I don’t think that my experience with the Dawn Princess is unusual in any way). Possibly the people who pay $1000 per night or more for a cruise don’t get the nickel-and-dime treatment, but for affordable cruises I think it’s standard. You have to be in the habit of asking the price whenever something is offered and be aware of social pressure to spend money.

When I boarded the Dawn Princess there was a queue, which I joined as everyone did. It turned out that the queue was to get a lanyard for holding the key-card (which opens the cabin door and is used for payment). After giving me the lanyard they then told me that it cost $7.95 – so I gave it back. Next time I’ll take a lanyard from some computer conference and use it to hold the key-card, it’s handy to have a lanyard but I don’t want to pay $7.95.

Finally some things are free at some times but not at others, fruit juice is free at the breakfast buffet but expensive at the lunch buffet. Coffee at the MDR is expensive but it was being served for free at a cafe on deck.

How to have a Cheap Cruise

VacationsToGo.com is the best discount cruise site I’ve found so far [2]. Unfortunately they don’t support searching on price, average daily price, or on a customised number of days (I can search for 7 days but not 7 or less). For one of the cheaper vessels it seems that anything less than $120 per night is a good deal and there are occasional deals as low as $70 per night.

Princess cruises allows each passenger to bring one bottle of wine on board. If you drink that in your cabin (to avoid corkage fees) then that can save some money on drinks. RumRunnerFlasks.com sells plastic vessels for smuggling liquor on board cruise ships [3]. I wouldn’t use one myself but many travelers recommend them highly.

Chocolate and other snack foods are quite expensive on board and there are no restrictions on bringing your own, so the cheap options are to bring your own snack food or to snack from the buffet (which is usually open 24*7). Non-alcoholic drinks can be expensive but you can bring your own and use the fridge in your cabin to store it, but you have to bring cans or pressurised bottles so it doesn’t look like you are smuggling liquor on board.

Generally try not to pay for anything on board, there’s enough free stuff if you make good choices.

Princess offers free on-board credit (money for buying various stuff on-board) for any cruise that you book while on a cruise. The OBC starts at $25 per person and goes as high as $150 per person depending on how expensive the cruise is. Generally booking cruises while on-board is a bad idea as you can’t do Internet searches. But as Princess apparently doesn’t allow people outside the US to book through a travel agent and as they only require a refundable deposit that is not specific to any particular cruise there seems no down-side. In retrospect I should have given them a $200 on the off chance that I’ll book another cruise with them some time in the next four years.

Princess provide a book of discount vouchers in every cabin, mostly this is a guide to what is most profitable for them – and thus what you should avoid if you want a cheap holiday. But there are some things that could be useful such as a free thermos cup with any cup of coffee – if you buy coffee then you might as well get the free cup. Also they have some free contests that might be worth entering.

Entertainment

It’s standard practice to have theatrical shows on board, some sort of musical is standard and common options include a magic show and comedy (it really depends on which cruise you take). On the Dawn Princess the second seating for dinner started at 8PM (the time apparently varies depending on the cruise schedule) which was the same time as the first show of the evening. I get the impression that this sort of schedule is common so if you want to see two shows in one night then you need to have the early seating for dinner. The cruise that I took lasted two nights and had two shows (a singing/dancing show and a magic show), so it was possible to have the late seating for dinner and still see all the main entertainment – unless you wanted to see one show twice.

From reading the CruiseCritic.com forum [4] I get the impression that the first seating for dinner is the most popular. On some cruises it’s easy to switch from first to second seating but not always possible to switch from second to first. Therefore the best strategy seems to be to book the first seating.

Things to do Before Booking a Cruise

Read the CruiseCritic.com forum for information about almost everything.

Compare prices for a wide variety of cruises to get a feel for what the best deals are. While $100 per night is a great deal for the type of cruise that interests me and is in my region it may not be a good match for the cruises that interest you.

Read overview summaries of cruise lines that operate in your area. Some cruise lines cater for particular age groups and interests and are thus unappealing to some people – EG anyone who doesn’t have children probably won’t be interested in Disney cruises.

Read reviews of the ships, there is usually a great variation between different ships run by one line. One factor is when the ships have been upgraded with recently developed luxury features.

Determine what things need to be booked in advance. Some entertainment options on board support a limited number of people and get booked out early. For example if you want to use the VR golf simulator on the Dawn Princess you should probably check in early and make a reservation as soon as you are on board. The forums are good for determining what needs to be booked early.

Also see my post about booking a cruise and some general discussion of cruise related things [5].

Related posts:

  1. Cruises It seems that in theory cruises can make for quite...
  2. Combat Wasps One of the many interesting ideas in Peter F. Hamilton’s...
  3. Victoria Hotel Melbourne I have just stayed at the Victoria Hotel Melbourne. I...

Twitter Weekly Updates for 2012-01-08

  • RT @CFA_Updates: Severe & Extreme Fire Danger Ratings and Total Fire Bans in areas of Victoria today. Stay informed at http://t.co/lHwDE#
  • in New York if you score too high on an "intelligence" test you cannot be a cop – says more about the folks in charge.. #
  • Ignoring CO2 because we don't fully understand climate is like jumping off a roof because we don't fully understand gravity #
  • Hopefully the UK policy to force publicly funded research to be published in open access journals will be carried through! #
  • Ordered VGA cable for @donna_williams Macbook, #Apple #039;s delivery website says it was delivered, but no sign of it! :-( #
  • Inflatable flying shark (needs helium) on Catch of the Day: http://t.co/l2zjmETb (via @fooishbar) #
  • Reminder all #VLSCI #HPC systems down: Bruce is being moved to new location, Merri and Tambo for electrical outage this w/e #
  • 9am-8pm day today at #VLSCI shutting down IBM gear ready for electrical work in the building over the weekend. Need sleep. #
  • Thanks @katharineannear and @Atlantean7001 for the #FF #
  • Other #Kubuntu 11.04 users waiting for LP bug 857828 (kontact migration to 11.10 fails) to be fixed before upgrading? #KDE #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-01-08

Twitter Weekly Updates for 2012-01-08

  • RT @CFA_Updates: Severe & Extreme Fire Danger Ratings and Total Fire Bans in areas of Victoria today. Stay informed at http://t.co/lHwDE#
  • in New York if you score too high on an "intelligence" test you cannot be a cop – says more about the folks in charge.. #
  • Ignoring CO2 because we don't fully understand climate is like jumping off a roof because we don't fully understand gravity #
  • Hopefully the UK policy to force publicly funded research to be published in open access journals will be carried through! #
  • Ordered VGA cable for @donna_williams Macbook, #Apple #039;s delivery website says it was delivered, but no sign of it! :-( #
  • Inflatable flying shark (needs helium) on Catch of the Day: http://t.co/l2zjmETb (via @fooishbar) #
  • Reminder all #VLSCI #HPC systems down: Bruce is being moved to new location, Merri and Tambo for electrical outage this w/e #
  • 9am-8pm day today at #VLSCI shutting down IBM gear ready for electrical work in the building over the weekend. Need sleep. #
  • Thanks @katharineannear and @Atlantean7001 for the #FF #
  • Other #Kubuntu 11.04 users waiting for LP bug 857828 (kontact migration to 11.10 fails) to be fixed before upgrading? #KDE #

Powered by Twitter Tools

This item originally posted here:



Twitter Weekly Updates for 2012-01-08

Twitter Weekly Updates for 2012-01-08

Twitter Weekly Updates for 2012-01-08

PyCon AU 2012 Report

PyCon AU 2012 is starting to come together. Here is a quick summary of how things are tracking

- Dates are set (18/19th of August),

- venues are booked + catering is arranged.

- A website and flyer is being prepared for distribution at LCA.

- We expect to open CFP's around Feb

If you have any questions please email contact@pycon-au.org

For subcommittee: 

Annual General Meeting & 2012 Council Election

Annual General Meeting

An agenda is now available for the AGM, to be held on Wednesday 18 January 2012 at 5:30pm in the Caro Theatre at LCA. You can view the agenda at http://tinyurl.com/6wgjr7d (will open in a new tab or window)

Election

Voting has commenced for the 2012 council. Please vote! You can do so via MemberDB, which lives at https://www.linux.org.au/membership/ or you can click the big button on the right :-)

January 07, 2012

A Tale of Two Conferences

Big BenLast week was a bumper week in London for MySQL users, DBAs & developers. We had the Oracle MySQL Developer Day and Percona Live London 2011. Both events were sold out, bringing in a good 300+ people to each event. From what I could tell the crowds were quite unique, so thats a good 600+ people interested in MySQL in London. The death and unpopularity of MySQL is greatly exaggerated.

At Oracle’s event, we naturally only had Oracle presenters. There was Simon Deighton (Sales Manager), Tony Holmes (Sales Consultant), Luca Olivari (Sales Consulting EMEA from the MySQL days), Andrew Morgan & Mat Keep for MySQL Cluster & High Availability. The event was actually pretty good if you were a MySQL beginner to intermediate user (that seemed to be the target audience — about 1 person was playing with 5.6, and about 1% of the audience was already using 5.5). The Q&A sessions were of high calibre, and answers obviously only pointed towards Oracle products.

At Percona’s event, we had wide and varied speakers, but an absence of Oracle. The crowd were already users of MySQL who wanted to get a lot more out of the database servers. It also served five tracks, so attendees had a lot of choice and value to choose from. There was an absence of beginner-centric talks, so one could get lost quite easily if you were sent there just for training. I already said I had an awesome time there.

The way I see it is Percona Live was meant for practitioners, while the Oracle MySQL Developer Day was meant for beginners to intermediate users of MySQL (they were probably already experienced Oracle DBAs). These kind of events are both important as you get a spread spectrum of people attending conferences. You can never really please all attendees at a large event, and in many ways it is always a balance you strike at large events like the O’Reilly MySQL Conference & Expo.

All in, London was abuzz with MySQL. Both events were out in the Tower Hill area. It is clear that MySQL and its diaspora are alive and kicking, and its quite possible the community of users are also growing.

Related posts:

  1. Percona Live London 2011
  2. Conferences selling out forget about the rest of the world
  3. qotd

Percona Live London 2011

I was at Percona Live London 2011 these past two days. Very interesting conference. Good work Peter & team — you’ve managed to gather a good 300+ people at one venue in London. So full was the venue, that during today morning’s keynote I had to sit in the spillover room and miss out on Peter calling out my name :-) (no, Stewart and I were not drinking at 9am!)

Gave my session titled Why MariaDB? (slides). Pleasantly realized that there were many new faces. Better still, everyone has heard of MariaDB in the room. More interestingly is that a bunch of people are now also using MariaDB in production!

Had to rush through the last few slides (about how open we are, the worklog, knowledgebase, etc.), but you don’t have much time in 30 minutes so you have to be succinct! The slides are attached.

 

Related posts:

  1. Using MariaDB in production?
  2. Google’s index is broken
  3. Opportunities to talk MariaDB/MySQL in Manila, Philippines