Planet linux.conf.au 2009

Sridhar Dhanapalan: Creating an Education Programme

Wed, 01 Feb 2012 12:34:34 +0000

OLPC Australia had a strong presence at linux.conf.au 2012 in Ballarat, two weeks ago.

I gave a talk in the main keynote room about our educational programme, in which I explained our mission and how we intend to achieve it.

Even if you saw my talk at OSDC 2011, I recommend that you watch this one. It is much improved and contains new and updated material. The YouTube version is above, but a higher quality version is available for download from Linux Australia.

The references for this talk are on our development wiki.

Here’s a better version of the video I played near the beginning of my talk:

I should start by pointing out out that OLPC is by no means a niche or minor project. XO laptops are in the hands of 8000 children in Australia, across 130 remote communities. Around the world, over 2.5 million children, across nearly 50 countries, have an XO.

Investment in our Children’s Future

The key point of my talk is that OLPC Australia have a comprehensive education programme that highly values teacher empowerment and community engagement.

The investment to provide a connected learning device to every one of the 300 000 children in remote Australia is less than 0.1% of the annual education and connectivity budgets.

For low socio-economic status schools, the cost is only $80 AUD per child. Sponsorships, primarily from corporates, allow us to subsidise most of the expense (you too can donate to make a difference). Also keep in mind that this is a total cost of ownership, covering the essentials like teacher training, support and spare parts, as well as the XO and charging rack.

While our principal focus is on remote, low socio-economic status schools, our programme is available to any school in Australia. Yes, that means schools in the cites as well. The investment for non-subsidised schools to join the same programme is only $380 AUD per child.

Comprehensive Education Programme

We have a responsibility to invest in our children’s education — it is not just another market. As a not-for-profit, we have the freedom and the desire to make this happen. We have no interest in vendor lock-in; building sustainability is an essential part of our mission. We have no incentive to build a dependency on us, and every incentive to ensure that schools and communities can help themselves and each other.

We only provide XOs to teachers who have been sufficiently enabled. Their training prepares them to constructively use XOs in their lessons, and is formally recognised as part of their professional development. Beyond the minimum 15-hour XO-certified course, a teacher may choose to undergo a further 5–10 hours to earn XO-expert status. This prepares them to be able to train other teachers, using OLPC Australia resources. Again, we are reducing dependency on us.

: Certifications

Training is conducted online, after the teacher signs up to our programme and they receive their XO. This scales well to let us effectively train many teachers spread across the country. Participants in our programme are encouraged to participate in our online community to share resources and assist one another.

: Online training process

We also want to recognise and encourage children who have shown enthusiasm and aptitude, with our XO-champion and XO-mechanic certifications. Not only does this promote sustainability in the school and give invaluable skills to the child, it reinforces our core principle of Child Ownership. Teacher aides, parents, elders and other non-teacher adults have the XO-basics (formerly known as XO-local) course designed for them. We want the child’s learning experience to extend to the home environment and beyond, and not be constrained by the walls of the classroom.

There’s a reason why I’m wearing a t-shirt that says “No, I won’t fix your computer.” We’re on a mission to develop a programme that is self-sustaining. We’ve set high goals for ourselves, and we are determined to meet them. We won’t get there overnight, but we’re well on our way. Sustainability is about respect. We are taking the time to show them the ropes, helping them to own it, and developing our technology to make it easy. We fundamentally disagree with the attitude that ordinary people are not capable enough to take control of their own futures. Vendor lock-in is completely contradictory to our mission. Our schools are not just consumers; they are producers too.

As explained by Jonathan Nalder (a highly recommended read!), there are two primary notions guiding our programme. The first is that the nominal $80 investment per child is just enough for a school to take the programme seriously and make them a stakeholder, greatly improving the chances for success. The second is that this is a schools-centric programme, driven from grassroots demand rather than being a regime imposed from above. Schools that participate genuinely want the programme to succeed.

: Programme cycle

Technology as an Enabler

Enabling this educational programme is the clever development and use of technology. That’s where I (as Engineering Manager at OLPC Australia) come in. For technology to be truly intrinsic to education, there must be no specialist expertise required. Teachers aren’t IT professionals, and nor should they be expected to be. In short, we are using computers to teach, not teaching computers.

The key principles of the Engineering Department are:

Technology is an integral and seamless part of the learning experience – the pen and paper of the 21st century.
To eliminate dependence on technical expertise, through the development and deployment of sustainable technologies.
Empowering children to be content producers and collaborators, not just content consumers.
Open platform to allow learning from mistakes… and easy recovery.

OLPC have done a marvellous job in their design of the XO laptop, giving us a fantastic platform to build upon. I think that our engineering projects in Australia have been quite innovative in helping to cover the ‘last mile’ to the school. One thing I’m especially proud of is our instance on openness. We turn traditional systems administration practice on its head to completely empower the end-user. Technology that is deployed in corporate or educational settings is typically locked down to make administration and support easier. This takes control completely away from the end-user. They are severely limited on what they can do, and if something doesn’t work as they expect then they are totally at the mercy of the admins to fix it.

In an educational setting this is disastrous — it severely limits what our children can learn. We learn most from our mistakes, so let’s provide an environment in which children are able to safely make mistakes and recover from them. The software is quite resistant to failure, both at the technical level (being based on Fedora Linux) and at the user interface level (Sugar). If all goes wrong, reinstalling the operating system and restoring a journal (Sugar user files) backup is a trivial endeavour. The XO hardware is also renowned for its ruggedness and repairability. Less well-known are the amazing diagnostics tools, providing quick and easy indication that a component should be repaired/replaced. We provide a completely unlocked environment, with full access to the root user and the firmware. Some may call that dangerous, but I call that empowerment. If a child starts hacking on an XO, we want to hire that kid

Evaluation

My talk features the case study of Doomadgee State School, in far-north Queensland. Doomadgee have very enthusiastically taken on board the OLPC Australia programme. Every one of the 350 children aged 4–14 have been issued with an XO, as part of a comprehensive professional development and support programme. Since commencing in late 2010, the percentage of Year 3 pupils at or above national minimum standards in numeracy has leapt from 31% in 2010 to 95% in 2011. Other scores have also increased. Think what you may about NAPLAN, but nevertheless that is a staggering improvement.

In federal parliament, Robert Oakeshott MP has been very supportive of our mission:

Most importantly of all, quite simply, One Laptop per Child Australia delivers results in learning from the 5,000 students already engaged, showing impressive improvements in closing the gap generally and lifting access and participation rates in particular.

We are also engaged in longitudinal research, working closely with respected researchers to have a comprehensive evaluation of our programme. We will release more information on this as the evaluation process matures.

Join our mission

Schools can register their interest in our programme on our Education site.

Our Prospectus provides a high-level overview.

For a detailed analysis, see our Policy Document.

If you would like to get involved in our technical development, visit our development site.

Credits

Many thanks to colleagues Rangan Srikhanta (CEO) and Tracy Richardson (Education Manager) for some of the information and graphics used in this article.

Colin Charles: MariaDB/MySQL users in Paris & Brussels

Tue, 31 Jan 2012 01:07:32 +0000

I’m about to head to Paris to present at the February meetup of the MySQL User Group in Paris, France. It happens 1st February from 6-8pm at the Patricks Irish Pub. Its free to attend, and I understand that SkySQL keeps this event afloat.

I’m also heading to my first FOSDEM right afterwards and will definitely hang out at the MySQL & Friends Devroom. There is an amazing lineup of speakers, with all talks being about 25-30 minutes, it looks like it is going to be a lot of fun. To boot, Michael “Monty” Widenius will also be there, so expect lots of Salmiakkikossu.

If you want to keep track of where Monty Program folk are going to be to talk about MariaDB, make sure you’re subscribed to our news page, which also includes important release information. Pretty much every conference that we plan to attend (and have attended) is at the conference page.

I am looking forward to meeting & learning from many MariaDB/MySQL users!

Colin Charles: Movies, January 2012

Tue, 31 Jan 2012 00:51:41 +0000

I’ve been on a plane quite a lot this month. Consequently caught quite a bunch of movies.

A Good Year – a romantic drama with Russell Crowe.
One Day – I like Anne Hathaway but this show is probably not one of her best. They pick moments from one day every year for some twenty years. Male star gets out of whack when she passes. Makes you always remember to seize the day. Carpe diem. Sometimes the best things in life are standing right in front of you.
What’s your number? – romantic comedy, just to pass the time over a meal
Killer Elite – most of Jason Statham’s movies tend to be action packed and this one is no exception. It’s based on a true story. I highly recommend watching this.

In non-movies, it’s worth noting that Californication is back! I’ve learned to watch Community, seems season 1 & 2 are complete so they are easy to watch back-to-back.

Gabriel Noronha: Getting ClearOS to work with Atheros Communications AR8151 v2.0 Gigabit Ethernet (rev c0)

Mon, 30 Jan 2012 12:17:07 +0000

ClearOS formally Clarkconnect based of Centos…

Start by enabling the Tim S repo

To install the repo first install the public key (yes all RPM’s will now be signed)

rpm --import ftp://timburgess.net/RPM-GPG-KEY-TimB.txt

Then install the release RPM (by default the ‘timb’ and ‘timb-testing’ repo’s will be disabled)

wget ftp://timburgess.net/repo/clearos/5.2/os/timb-release-1-0.noarch.rpm rpm -Kv timb-release-1-0.noarch.rpm rpm -Uvh timb-release-1-0.noarch.rpm

ref http://www.clearfoundation.com/docs/howtos/adding_tim_s_repo

yum --enablerepo=timb install kmod-atl1e

ref http://www.clearfoundation.com/component/option,com_kunena/Itemid,232/catid,28/func,view/id,24438/limit,10/limitstart,50/

last you need to edit the /etc/modprobe.conf

it need to contain an alias for every network card in my case I have a TP-link installed as well so

alias eth0 r8169 alias eth1 atl1e

if you fail to get this to work ifconfig -a will have a odd tmp interface.

leigh morresi: First android app released

Sun, 29 Jan 2012 21:59:14 +0000

Jonathan Oxer: Walktime Blog #19: No dominant Open Source home automation projects?

Sat, 28 Jan 2012 01:39:47 +0000

If you're just getting into home automation and go searching for Open Source HA projects, you'll discover there aren't any obviously dominant players - more a mix of partly-developed personal projects that aren't very portable. Why is that?

View or comment directly on YouTube: www.youtube.com/watch?v=Ywulcl_PCUM

Links for this ep:

* Allison Randal: www.twitter.com/allisonrandal

* Desktop Home Hacks: www.youtube.com/watch?v=a8asl5SsGy4

* Mister House: misterhouse.sourceforge.net

* Open Remote: www.openremote.org

* SuperHouseTV: www.superhouse.tv

Jonathan Oxer: Walktime Blog #18: Interview with Marco Ostini at LCA2012

Fri, 27 Jan 2012 07:52:33 +0000

Despite Australia's strong involvement in the early history of space exploration, we're now lagging way behind many far smaller nations in our commitment to the industry. At linux.conf.au 2009 in Hobart, Marco Ostini gathered together a group of like-minded people to create Lunar Numbat: a project to develop Open Source space technology as part of the White Label Space team competing for the Google Lunar X-Prize. At linux.conf.au 2012 in Ballarat I managed to pin him down long enough to film this interview.

View or comment directly on YouTube: youtu.be/6PQ6mIEmKfc

Links for this ep:

* Marco Ostini: www.twitter.com/marcoostini

* Lunar Numbat: www.lunarnumbat.org

* White Label Space: www.whitelabelspace.com

* Google Lunar X-Prize: www.googlelunarxprize.com

Jonathan Oxer: Walktime Blog #17: Interview with Matt Evans at LCA2012

Fri, 27 Jan 2012 03:32:46 +0000

At linux.conf.au 2012 I had the good fortune of meeting up with old friends and also making new ones, including the amazingly talented Matt Evans. Matt did a great talk called "Hack Everything: Re-Purposing Everyday Devices", so I caught up with him briefly afterwards to talk to him about it.

View or comment directly on YouTube: www.youtube.com/watch?v=BIQl0u-YZ5s

Jeff Waugh: QotD: Jon Corbet on linux.conf.au and Linux Australia

Thu, 26 Jan 2012 05:52:19 +0000

In summary, LCA remains unique in its combination of strongly technical talks, freedom-oriented and hands-on orientation, wide variety of topics covered, and infectious Australian humor. There is a reason some of us seem to end up there every year despite the painful air-travel experiences required. Linux Australia has put together a structure that allows the conference to be handed off to a new team in a new city every year, bringing a fresh view while upholding the standards set in the previous years.

– LWN’s Jon Corbet on linux.conf.au, An LCA 2012 Summary

Colin Charles: Popular Malaysian passwords (sample size=75,000)

Sun, 22 Jan 2012 15:58:31 +0000

I just read that about 100,000 most likely Malaysian Facebook accounts have been cracked. Well their passwords are available for all to see in pastebin. Upon checking, I only saw a little over 75,000 since the third file has been compromised. From that sample, I quickly derived that the most popular Malaysian passwords are:

123456
sayang
brokenheart
123456789
rozaliqa75
effaluve
akucintaallah
zzz999
pradeebkumar123$%
12345678

The least popular ones in that sample set include having spaces, hashes, brackets, and more. So there are some secure ones!

Using one’s phone number seems to also be popular. Sometimes appending or pre-pending a string (like a name) to it. Using birthdays seems to also be quite common, sometimes also appending or pre-pending a string like a name to it.

And for those wanting to “further analyse” the dataset yourself, I just quickly used standard Unix tools, and you can do it too.

grep "Password:" part_* | awk -F":" '{print $3}'| sort | uniq -c |sort -n

Pipe to less, use head/tail, etc.

Colin Charles: On killing Hollywood

Sun, 22 Jan 2012 14:13:00 +0000

Paul Graham recently published a new request for startups titled Kill Hollywood. It is definitely worth reading. The motivations behind such thoughts are clear. Filesharing is not killing the movie & TV industry.

“What’s going to kill movies and TV is what’s already killing them: better ways to entertain people.“

Better ways to entertain people. This thought has been sitting in my head for the last couple of days while I’m just a stones throw away from Hollywood & have a pretty good view of the Hollywood Hills from outside my window. The RFS goes into more detail about games, apps, the possibility that exercise might take over, but to think broadly and figure out where the entertainment of folk are going to in the next twenty years.

The studios are making less profits because the way Hollywood is structured. This is why Sarah Lacy says to kill Hollywood, you’ve got to learn their game. Someone like Ryan Kavanaugh is using math to beat Hollywood at their own game — you may have seen Relativity Media, and that’s the company who’s funding many successful movies today. Sarah Lacy sums up the content game that will help us win against Hollywood fairly well:

“The lesson: Eyeballs aren’t equivalent to one another. For Hollywood to be killed, the Internet needs to focus on a metric other than eyeballs. It’s not about mass, it’s about good. That’s absolutely anti-YouTube and anti-Farmville and any other content which we expect to be rapid, mass and disposable. Disposable content isn’t bad, it’s just not everything. And as long as that’s all that the Valley is putting out, we won’t kill Hollywood.”

There is an experience of going to the cinema in where I am happy to pay USD$12 or RM25 for a seat. In the USA I believe in the ratings system, but in Malaysia where I watch most of my movies I feel cheated by the censorship board. But I still go and spend cash because there’s an experience. However I’ve noticed my TV & movie watching habits have changed — I wrote about how I consume Hollywood in 2011. I believe that in Malaysia (and most of Asia), one is forced towards looking at content via filesharing. Because Hollywood hasn’t grown up and they believe in making money from regions, delaying releases by regions, etc. Traditional models.

Of late I’ve quite enjoyed watching the Sundance channel on cable. On Friday in the USA Today, Robert Redford, founder of the channel and the film festival had this to say: “With the new technology creating all the voices and noise from bloggers and tweeters, it’s chaos,” Redford says. “Where are you going to get the real truth with so many loud voices barking? I look to documentaries as almost investigative journalism.”

That covers a set of genres. But independent films rarely cover comedy, action, etc.

People get entertained by different things. At different times. Some days a romantic comedy makes sense. Some days a chick flick is all that gets you going. Then you’ve got days when action is all you crave. And the list can go on…

So what are better ways to entertain people? Games? Interactive movies? How does everyone get paid fairly when you get away from the big studios? Do production costs then go down when you bypass them?

This is why people love the Cheezeburger Network. Or 9gag. These are new ways for people to entertain themselves. However the metric there is eyeballs and the content is disposable. People need substance to entertain them. I once said that paying $10 for Plants vs Zombies provided me with a lot more entertainment on my iPad than going to maybe 2-3 feature length movies.

I’m still thinking about different ways for people to consume media. Different ways for people to sink their time in. And I presume I’ll be thinking about this for long.

As an aside, don’t assume that independent media folk get “new media” either. Classic examples in Malaysia would be Nasi Lemak 2.0 and Relationship Status. Nasi Lemak 2.0 stars the controversial Namewee, who not only made the movie on the cheap (independently), he went on to getting it in cinemas and also at the same time did the entrepreneurial thing of in tandem getting it showing on cable TV. This subsequently got his movie pulled from the cinemas in question, rather abruptly. He disrupted the cinemas and the cinemas reacted in their traditional methods to pull his movie. But even today, you can’t buy a DVD or download a digital version… Even if you’re willing to pay for it (I know I am). More recently, Khairil M. Bahar made Relationship Status; however still with the traditional model of going to the cinema. No DVDs. No downloadable digital version. Its worth noting that I’d pay RM35-40 for a digital download (though I don’t think that might be everyone’s price point – experimentation needs to happen clearly).

Its sad to see that even young independent film producers aren’t moving where their audience is moving to. They’re thinking like studios are thinking. They need to be disrupted. After all, these Malaysian producers are forgetting that there is such a large portion of the Malaysian diaspora spread across the world whom are unlikely to step into Malaysian cinemas anytime soon. Imagine a day when I can read a review about the show, then automatically click on a link that allows me to either stream the movie now or download a copy. If it is a service that has my credit card details on file, this is a seamless process; if its individuals, I just checkout via PayPal, and am either seeing the movie on my TV or waiting half an hour or so for the download so I can pop it on my iPad.

Back to the drawing board. There are better ways to entertain people. There are better ways for consumption of media & content.

Simon Lyall: LCA2012 – Friday after lunch

Fri, 20 Jan 2012 04:03:31 +0000

Codec 2 – David Rowe

Open speech Codec. Low bitrate 2400 b/s down to 1400 b/s
Applcations for digital radio
Fills <5000 b/s gap
http://rowetel.com/codec2.html
Not a DSP talk
Can send 45 calls inside 64 kb/s chanel
Not useful for VOIP due to IP/UDP overhead of 8kb/s on 1400b/s data
Main use radio spectrum. Less data = less power required since your power gets concentrate on less bits
doesn’t matter too much if odd packet dropped
proprietary codecs slowing digital voice over radio
Proprietary codes: hardware or licensed software form, difficult to distribute, can’t modify
Example g729 license $40k. Doesn’t believe closed source codecs benefit society
Authors of propriety/patented codecs borrowed heavily from public domain. perhaps 5% is original. Good news is only 5% needs to be replaced
Speech coding: eg 16bit samples at 8kHz, comprss to 1400-2400 b/s . What can we thrown away, retain intelligible speech, retain natural speech. Use a model of speech, send model parameters, for effecient than coding waveform
Model: example is pitch, humans 50-500 Hz , can be represented with 7 bits, updated every 20ms 7/0.02 = 350b/s to represent pitch
Codec 2 uses Sinusoidal speech coding. Multiple Sine waves added togeather
Bit allocation: 56bits every 40ms. Of these: Amplitude 32 , Frame energy 10 , voicing 4, pitch 10
Developing Codecs: complex DSP algorithms, run codecs in non-realtime, dump values from codecs every “frame” ( 80 samples, 10 ms of speech) . Gnu Octave
Banned exports list includes ” Speech codecs below 2400 b/s ” . Have been advised by DECO that Codec 2 has “assessed as not controlled” but waiting for certificate

UEFI and Linux – Matthew Garrett

Replacement for PC BIOS
BSD licensed core
Adds standardized support for new hardware features
Platform init
EFI image load – loaded drivers
EFI OS loader load – oot from ordered list of EFIOS loaders
Boot services terminate -> OS handover
Boot services – memory allocation, timers, image loading, GUIDs.
Runtime services – non-volatile variable store, boot data, system information, crash dumps (already in Linux 3.2)
Able to update firmware by reset and grab new firmware out of variables on bootup
GPT – GUID partition table – no practical restrictions on size and number – more metadata about partition type and service
That all sounds good …. but ….
TianoCore – Open Intel reference UEFI reference implementation, 7061 files, >100MB of code, 10% of size of Linux kernel. Bigger than Linux core kernel
Large codebase, some bugs
UEFI is poorly tested in the real world. UEFI contains a lot of code. UEFI contains a lot of bugs
Some problems with secure boot

Simon Lyall: LCA2012 – Friday Morning

Fri, 20 Jan 2012 01:01:09 +0000

Bloat: How and Why UNIX Grew Up (and Out) – Matt Evans and Rusty Russell

Cool projects: spark, plover, Homebrew Cray-1A
Compare PDP-11 Unix vs Modern Ubuntu 11.10
Binary sizes: cat 152 bytes vs 531k KB
grep command: 2176 bytes vs 687 KB
ls command: 4904 bytes vs 628 KB
V6 cat command just 12 lines of assembler, 2 * 512bytes buffers, a.out 16 bytes overhead
Binaries 30% because we chose speed over size. ~9% speed gain
V6 Runtime coverage: cat 99% , grep 78%, ls 85%
V7 has reduced coverage. some commands converted from assembler to C
x86 runtime with dietlibc coverage: cat 11% , grep 23% , ls 39%
x86 static cat has 700k of libc dependencies, 17% of libc, 313 objects it depends on
libc 1.7M but widely shared among hundreds of processes
dynamic ls accesses 90k of libc but 476kB paged in.
For sample system. libwebkit 8.5MB , 5MB wasted, 33MB wasted real RAM
What about a 64Bit version of a PDP-11 – a PDP-11 . Various assumptions on how binary size would increase
PDP-44 – binaries around 50% larger
32 bit ubuntu binaries are 9% smaller than 64 bit ubuntu
Forward port V6 binaries to x86 . V6 cat almost same size as dietlibc version
More work to forward port V6 ls, lots of assumptions not longer true. Code tricks no longer work. 20% larger cause of ELF and nmap. 120% penalty due to modern infrastructure (eg malloc realloc)
Backport x86 “ls” and “cat” to V6. Only backport some options
cat: remove old options and error reporting. Kept some features.
ls: remove lots of options.
Binaries 60% larger due to flexibility
440% bloat due to new features
Asmutls – reimplementation of current Linux utils in x86 assembler.
The talk is online, hard to do notes since it jumped around a lot and graphs hard to read

Open vSwitch – Simon Horman

Switch contains ports, ports has one of more interface, packets are forwarded by flow
Flows may be identified by lots of combos, address, vlan, ports, TOS
1st packet in flow gets sent to userspace controller, controller makes decision, tells datapath what to do with future packets, resends first packet back to datapath. Later packets the datapath knows what to do (from hashtable lookup) and handles itself
Configured by JSON database, persists across restarts
database controlled via Unix socket or via TCP. Change action won’t return until database update performed
cute ” –may-exist ” options when creating stuff that does nothing if what you are requested already exists
He did some demos of standard sort of stuff, truck interfaces, port mirroring, fairly simple commands to do
Does VXLAN and GRE tunnels
Oracle looking to put in Oracle Linux soon to replace current bridging code
Can do millions of packets per second. Some bottlenecks in tunneling code

Simon Lyall: LCA2012 – Thursday last session

Thu, 19 Jan 2012 06:20:38 +0000

Challenges for the Linux plumbing community – Jonathan Corbet

Good news is boring, so how about some “high quality problems”
Security
Stuxnet , kernel.org , RSA hack , DigiNotar
Scary ones are there must be others we haven’t heard about
The bad guys are: motivated, capable, well funded. Not just script kiddies
Not just about money anymore, with governments hacking lives are at stake
We are on the front line. Not just security software, all code security critical
Is your code secure? Who reviews it? What sort of testing? Plans for dealing with vulnerabilities?
Is your infrastructure secure? – Who has access, who can change files? Are security updates applied? What are your plans in case of a breach?
Are your processes secure? Who can commit? What can sign releases? Can you detect tampering? What do they know about the codes provenance?
Tools
Lockdep, valgrind, fault injection, sparse, smatch
GCC python plugin, MELT, LLVM static analyzer
and need to actually use the tools that exist
Hardware
hardware complexity leads to software complexity
Complex interfaces: example V4L3 media controller interface.
Control over our hardware
Life is okay (could be better, could be worse)
What is our influence over manufacturer?
Example: Chasing tablet manufactures , no influence on design, have to port after device launched
Example: By the time “Rock Box” runs on a device device is obsolete and not in shops
How can we be more involved in conception and design of hardware in the first place?
Linux Only
Once upon a time we depended heavily on portability
The DRM tree deemphasized BSD support, This hurt BSD but… would we rather do without kernel mode settings
Might be inevitable but try not to be too arrogant
The platform problem
Code you control vs Black box
The kernel’s ARM subtree (re-implements stuff from elsewhere in kernel)
XFree86 (tried to keep everything in user space)
Opportunistic suspend (Andriod decided “too hard” to fix rest of kernel)
Async I/O (implemented multiple times, no comprehensive implementation)
Example: wireless devices had own 80211 implementation. replaced with max80211
Example: PowerTop used to find wide range a random things causing high power usage in laptops
Ongoing examples: Bufferbloat, marvell-cam drivers, User-Space TCP, Control groups, Andriod

What is in a tiny Linux installation? by Malcolm Tredinnick

Skipping bootloader portion
Kernel is big – 9.6M lines of C, 250k lines of assembler
Booting the kernel
“make allnoconfig” , smallish, 222 “y” ‘s. 842KB bzimage, build time under 15s, no file systems, no fancy hardware, ISA, no PCI
“make allyesconfig” , 5177 y options. 39MB bzimage, over 1h to build, includes drivers/staging
booting allnoconfig via qemu-kvm . Gets to “unable to mount root file system”
Kernel components – hardware arch, drivers, subsystems, others
need roto filesystem in memory, initrd / initramfs . init process just in cpio archive, can just be hello world
need initrd, initramfs , RAM disk block device, ELF binary support. 889KB bzimage (up 50kb)
Now boots, use “rdinit=/hello” option in qemu , just prints out hello world
Transition to userspace
initrd loads some modules etc, runs pivot_root , run startup scripts
Userspace
Why are you doing this? Single purpose system, usb stick (rescue, puppy linix, Damn Small Linux) , tiny memory, tiny storage usage, fast power on. Trade-off of options
We have to run something, need some binaries, shared libraries, large binaries with multiple purposes (busybox)
Busybox – one binary – acts differently depending on calling name, installed as symlinks
Busybox: fairly small, default utilities, 2MB without networking, easy to test
C libraries – glibc (probably not a good idea), eglibc (easier to build, binary compatible with glibc, can take things out), uClibc (alternative, very small, some overlap with busybox, source code compatible with glibc)
Device and Proc mngt will need: procfs, sysfs, tmpfs, udev, cgroups
Build environments: you are cross compiling (build root), binutils, C libraries & cross compiling, Test, x86 instead x86 is harder
See links in slides for some help
mdebian is something to look at

Simon Lyall: LCA2012 – Thursday after lunch

Thu, 19 Jan 2012 03:56:21 +0000

Women in Open Technology & Culture – Valerie Aurora and Mary Gardiner

Very umbrella term including fan fiction, open data, wikipedia, open access
Why – important areas – women’s participation (especially in charge) very low
Important for women to be in charge, creating, designing, building, not just as users
5 kinds of groups – project specific (debian women), feminist activism, teaching technical skill, networking, majority women projects.
Community / project specific
Linuxchix, owoot, pyladies, wikichix, etc ( linuxchix spawned several)
low participation, poor replacement rate of leaders (often after they get FT jobs), low communication between, sometimes tension between.
Feminist advocacy
geek feminism, ada initiative, mind the gap
growing and active – the new hotness, sharing best practices, paid work more common, some conferences
Teaching women technical skills
usually one day or evening courses.
Growing hugely, vary widely in topics and skills, sharing best practises
In person networking socialization
Women in code, girl geek coffees, girl geek dinner
try not to be dominated by marketing women ( use of “geek” term helps)
Growing, easy to start local chapters
Majority Women Groups
Dreamwidth, Organisation for transformative works
Often fan-fiction support, protect against takedown, let author control commercialisation
Survey
In person vs Online
Activist vs non-activist
Community vs technical
Focussed vs broad topics
Projects with broad focus within a narrow group seem not to work
Projects with very technical focus but accoess different technologies seem not to work either (lack common language)
Why Start – recruit and retain, networking, role models, safe space, feel normal
Lessons on starting
Don’t – join an existing one
If you are a man, don’t do on behalf – “Nothing about us without us”
Don’t expect women to start a group
Find 3 or more women to start a group
Don’t use girl/chix/ladies – use women
Go broad instead of narrow on topic
have clear defined goals and scope
Start small, be realistic about work
Consider one-off event rather than group
Avoid NIH , reuse best practices
be prepared to moderate any public forums you create
Failure modes
Become “the nice place” that everybody goes to
Loses focus on women
Safe Space moderation too many hours
Ran low on time, slides will be online

Hacking Everything – Matt Evans

Reuse things , not just hacking things like audrino that are supposed to be hacked
reuse, need, art & design
Gambiarra – brazilian art of an improvised fix
1940s radios and TV owners could fix their gear. today people are more passive
wants people to tinker with things.
Save resources
Save money
take apart things, learn by example
Low cost manufacturing makes hacking hard ( solid state everything )
Cheap development makes hacking easier ( reuses common technology, extra bits on devices unused )
Some products are open hardware designs
Things to look for – similar to ref design, debug code left in, unused features, factory test points/ports
Ports that are wired up but unused often serial ports
“My CD player has a serial port” , common on many devices
Acquire a “logic level”USB-serial cable
Other ports – JTAB , In-System programming
Example: Picture frame, derived from sample board for camera, serial interface, built in CLI
Old Wifi, ADSL boxes good with OpenWRT
Don’t just consume – re-consume
Teach others and tell the world
Collaborate at a local hackerspace
support companies that make things hackable

Simon Lyall: LCA2012 – Thursday Morning

Thu, 19 Jan 2012 01:13:48 +0000

Desktop Home hacks – Allison Randal

Just a hobby product, must be fun, open,
Wanted computer to be available when away from desktop, but not be disruptive and uncomfortable
Affordable, approachable to hobbyist, there are more expensive alternatives
nodes – jeenode, audrino clone, tiny, easy to hid, cheap ( $20 including wireless RFM12B vs $20-30 + 30 for wifi for more conventional audrino )
RFM12B – 66 bytes
Server – pandaboard ~$200 . ARM processor with full Ubuntu install
Server – aggregates data from all notes and sends commands to all it’s nodes. Contains services for system, json data feeds from nodes, web client interface, interface for sending commands to system
Client(s) – several machines in house used from
client – jquery mobile , small applet , on desktop , chromium app mode
Inputs: Temperature , Humidity , motion sensor (lights on/off)
Input: RFID reader (disappointed at short range, <1m , one was in doorways to trace path of wallet)
Input: touch sensor ( simple controls, in pillow ) , small keyboard (bluetooth, usb)
Output: Glowing egg ( multi colour, hand sized ) , Power Tail (power extension, turns on/off)
Output: Hollowed out candle with LEDs inside , Instamorph & super Sculpey to create “solid” objects
Tools: Soldering iron, misc tools, wax carving kits
Lesson: Need better camera with macro lens and better light to document
software not currently release, not really in releasable format, thinking of making some as audrino shields
Resources: adafruit.com , sparkfun , parallax.com , digikey.com (bad interface) , optopart.com , makershed.com , freetronics.com (Australia) ,
Tools: Talk in inkscape/sozi , Also use: vala (webserver) , jquery mobile

Cheap Tabloid tricks – Angus Kidman

Journalists have secret lawyer fantasies, they have wrung the information out of the sources
Is the IT media biased?
no coverage of lca2012 in main Aus IT press
more coverage in 2nd tier
Only 3% pageviews on lifehacker use Linux
Not many IT journalists these days, One on FT on lifehacker, more to do (websites, blogs, video, podcasts)
Freelancers – can be an option. Freelance rates not good, getting worse, competition from bloggers, etc. Hard to convince editor that story is worthwhile.
IT news driven by fashion. In early 2000s belief among publications that Linux stories would drive traffic from slashdot etc
Current fashion is facebook and Apple. Stories about them in demand
Eg Lifehacker did apple angle on LCA keynote
Open source Projects lack definitive spokesperson
Media obsessed with cult of Trivia, Celebrity . eg Linus
Media not influenced by advertisers
Does Linux need media? Yes if want to reach more than the 3%
Identify the Audience. right publication
Be Fashionable. eg mention facebook or apple
Be Concise.
Be contactable. email, phone if in a hurry

Colin Charles: SCALE 10x – there’s lots of MySQL there!

Wed, 18 Jan 2012 13:08:58 +0000

I’m just about to get on a plane to head to my inaugural SCALE event. It’s their tenth year running!

In a world filled with NoSQL related media, its kind of nice to see that on Friday January 20 2012, we have a MySQL room right next to the PostgreSQL room (schedule). It is awesome to see that the track will have participation from Oracle, Monty Program Ab, and SkySQL Ab.

On Saturday for the main tracks, I’ve got a talk about the growing MySQL diaspora (just got larger this year in case you haven’t paid attention to the packaged up Galera product!). This one is a constant work in progress and I’m hoping to complete research closer towards March ’12.

Monty Program and SkySQL are also sharing a booth in the expo hall, so come by booth #65 for some interesting schwag (t-shirts, poppers, etc.). Looking at the schedule lineup, I’m surprised I’ve never ever been to a SCALE before – looks totally awesome. See you in LAX (well, we’re so close-by the Los Angeles Airport :P)

Tim Connors: An open letter to Peter Ryan regarding police treatment of cyclists

Wed, 18 Jan 2012 11:18:39 +0000

Hon Peter Ryan,

I am writing because I am concerned at the number of recent incidents where a driver has collided with a cyclist, and the case hasn't been followed up by the police. Such incidents and the publicity surrounding them does nothing to encourage road users to obey the law when they realise that they will most likely get away with not doing so.

A week ago in Ballarat, a 13 year old boy was hit by a car, and the police said the boy had the right of way[1]. Despite this, the article linked states that the police will not charge the driver. This, despite her having broken Australian Road Rule 67 to 72, 84 or 86 depending on circumstances at the stated intersection, or perhaps 140 to 144 if travelling in the same direction. She was likely negligent in allowing the collision to happen in the first place, which, by my understanding, is a criminal offence, especially since there was serious injury involved. If she used the usual excuse that "she didn't see him", then that's an admission of guilt in failing to obey ARR 297 - driver having proper control of vehicle.

Also recently, there was a highly publicised case where Shane Warne had an altercation with a bicycle rider. In that case, the fact that Warne hit the cyclist from behind (ARR 126) after overtaking unsafely (ARR 144) is undisputed[2]. The fact that details were not exchanged following the collision is also undisputed (ARR 287). It is also well established that Warne was stopped unnecessarily in a bike lane (ARR 125; 153)[3]. And yet the police will not investigate[4].

Going back a number of years, I also have not had good experiences getting the police to follow up on cases. In my most recent case (11/10/2005; I do not know the case number sorry, all I know was that I was attended to by Angove & Auchterlonie from Boroondara police), the driver also failed to obey ARR 287 (as well as a slew of other offences, such as ARR 46 and 148 - changing lanes without indicating sufficiently and without due care). The police refused to prosecute the driver, and also would not hand over the driver's details or insurer details, based on some misguided privacy policy, asking me instead to fork out for a freedom of information request. Given that I was a broke student at the time, this was not a feasible thing to do and I never did receive compensation from the driver for damage to my bicycle, clothes, and large out of pocket expenses for travel to medical care for several years that the TAC didn't cover. The police also displayed a lack of knowledge of the law, initially thinking that I had broken ARR 141.

I can't imagine why the police aren't investigating these cases, because in each case, clear evidence is at hand, and not disputed. The identities of all parties are known. It should be an open and shut case. Without the police making charges, the rider in each case will have a much harder time claiming from the driver's insurance (if the boy was not admitted overnight, his TAC excess will be an enormous burden to his family). The driver in each case will not be discouraged from driving in a similar fashion next time. And other drivers also know that they will most likely get away with any offences they commit if a bicycle is involved. This is a perverse reversal of the situation that we should have, in which drivers should be encouraged to take due diligence around cyclists. It almost seems that the police always assume a cyclist is at fault unless proven otherwise in Australia, whereas most other countries with an established bicycling culture assume that the driver is at fault unless proven otherwise as they hold the burden of driving the more deadly vehicle and so should be required to take due care.

If the laws weren't adequate enough to prosecute to the driver in the above cases, has your department been contacted to update the laws, and what is being done? Keep in mind that cyclists have no protection other than by the law, and as the more vulnerable road user, the laws should focus on their safety and ensuring that transgressions are dealt with effectively.

Can you please encourage the police in each of these cases to follow them up to the full extent that the law currently allows.

Sincerely,

[1]

http://www.theage.com.au/victoria/teen-cyclist-struck-by-car-20120110-1ps85.html

[2]

http://theage.drive.com.au/motor-news/warnes-tirade-triggers-bike-rego-call-20120118-1q5k0.html

[3]

http://www.cyclingtipsblog.com/2012/01/cyclist-versus-warnie-the-cyclists-story/

[4]

http://www.heraldsun.com.au/news/more-news/warne-blasts-cyclists-on-twittershane-warne-clashes-with-cyclist-on-way-home-from-training-session/story-fn7x8me2-1226246735306

Simon Lyall: LCA2012 – Wednesday last session

Wed, 18 Jan 2012 06:16:46 +0000

The Samba tour of scripting languages – Amitay Isaacs and Andrew Bartlett

Samba is C based
But seems to have a lot of scripting
Has to be portable no non-gnu systems like solaris, reply on POSIX sh, make (not gun make), awk, m4 and a c compiler
shell scripts for first testing. Over 10,000 lines if shell in building
Python , TCL and Lua bindings all added but unpopular and eventually removed
Perl over 20,00 lines
IDL build initially in awk. Switched to perl based PIDL
javascript before it was cool – embedded javascript engine
But the cool kids were using python so switched from Javascript to python
Exception based languages things cleaner
waf is python based build system
python bindings for most things, or via C hook
Lots of other stuff being written into python, called directly by samba for small tasks
90,000 lines of python
Example: Samba3 upgrade - python based tool in 3 weeks. Business login in python, exceptions test for bad input
At build time python checks to see if ABI has changed from previously and alert developer if it has.
Test frameworks – unit tests on standalone components. Environmental tests with everything running, different types on server setups, fake ips and tests made. All runnable as non-root
9000 test in 1300 test suites. Mosts test in C, some in python or shell

Running Python in Grub – Josh Triplett

” I ported python to grub “
perception that Linux doesn’t need BIOS
Involves programming hardware in functional/safe/optismise configuration
Lots of stuff to support, a few decades of compatibility, very bare-metal programming, small number of people working on it
What can go wrong: Broken or disabled CPU features, missing or broken memory, sub-optimal power mngt, delays & latency, USB bios handoff. Undocumented customer interfaces
Why not test under Linux? Linux gets in the way, no direct BIOS access for tests
Grub2 – 32 bit addresses, written in C, can read files, menus, single thread, no OS to disturb, only uses a bit of BIOS itself
ORigin: Replacing DOS test programs, test for power mngt, new grub commands for command-line.
Grub script language – bashish, no expressions, can just glue things together in menus, have to write lots of C
Ported CPython 2.7 to GRUB
Wrote a C/Posix compatibility layer for GRUB, floating point functionality via fdlibm , ported much of python standard library
Build C extension modules, added “bits” module to access platform functionality
ACPICA already in Linux for parsing ACPI. ported as a grub module with python module to access it.
bits passes grub command line to python for python to parse
FUSE for Python and GRUB. added a python device reading python/foo invokes a python callback.
logging to in-memory buffer and sends to log in FUSE via system. Save to ACPI table and then OS can grab it later
SMP support in grub, pyton scripting in ring0, python modules for platform interfaces CPU, PCI, PCI-e, ACPI including decoding and method evaluation, logging, test suite evaluation
BITS test: power mngt configuration, perf optimisation, CPU config registers, SMI frequency/latency , USB handoff from BIOS to OS including effects on SMI and C states.
Used by BIOS vendors before shipping boards, BIOS problems actually get fixed!
http://biosbits.org/
Freakiest stuff I’ve seen so far this week

Simon Lyall: LCA2012 – Wednesday after Lunch

Wed, 18 Jan 2012 03:53:09 +0000

Ubuntu ARM – David Mandala

Targeted ARM v7 – first release April 2009
7 different kernels in 10.10 . Improving with Linaro
OMAP 3 100% in main Linux kernel so easy to support.
Someday unified kernel onto ARM
Toshiba AC-100 netbook
11.10 preview release of ARM server
Lots of work to get SMP and now 64 bit to work. Some code assumed they would never exists
Virtualisation support soon for server space
Main sense of ARM in server space is 10x saving in power eg ~50W vs ~5W
Lots of other stuff this guy was going too fast for me to keep up

Helping your Audience learn – Jacinta Richardson

Conferences let you vary your level of intensity according to your energy
Conferences – no assessment
Training is different, all day, 6-8 hours, several days in row, builds on previous days
can’t afford to get lost.
Cognitive load – how much effort somebody has to apply to learn a new thing.
Intrinsic – how hard actually thing is.
Extraneous – how harder trainer makes it than it needs to be
Germaine – how well concepts build on what we already understand
Building framework takes time. Scaffolding has to be well designed. Lots of simply examples
Mind map of material -> what you are teaching to build foundation
Be realistic what you can fit into day (including breaks, people late)
6 – 6.5 hours optimistic. Seems to be max most people can handle
90 minutes then break ( eg 90min on, 30min off repeat 4 times )
Documents – short prose sections, short examples, short chapters (work though in 90 minutes)
Key info at start of the day. Dont do lots of extra stuff like class rules
Easy stuff, unimportant stuff at the end of the day.
Essentials at the start of the course
First 90 minutes or first day is most important
Options extras at end of course or end of each day
If you have lots of stuff -> create another course, make more money
Use diagrams, code, pictures, comics
reduce germane cognitive load:
Order carefully
group similar concepts
Put import stuff in bold
10:10 – 10 minutes instructions, 10 minutes of student exercise. Sometimes 10:20 . Occasionally 10:30
1-3 concepts in that 10 minutes. But try to balnce
Spare time = more examples
90 minutes = 3 x 10:20 + 4.5 x 10:10
Target exercises at each key point. Doesn’t have to be real-world
1 point = 1 exercise
Easy to advanced exercises. Additional exercises to really advanced people
NO answer files. Cause everybody will cheat
Minimise cross-chapter reliance
Sometimes you have to rely on previous stuff (should have been at start on day one). Try and avoid since people will have missed or not picked up previous concepts
New topic = clean slate
Good, through course notes
Not slides, write a book, should be readable later, good advertising
A few other ideas:
Keep room cold, keep it fresh. 21-23 degrees
Bell curse applies to student ability. Students not slow, but have less foundation or experience in topics
Target average student. Offer extra help for ones behind. Don’t slow down for slowest student.

Simon Lyall: LCA2012 – Wednesday Morning

Tue, 17 Jan 2012 06:09:38 +0000

A Tour of btrfs – Avi Miller

Now
All data and metadata is copy-on-write
CRC all metadata and data
Writable snapshots
multi-device support ( raid0 , raid1 , raid 10 )
online resize and defrag, online device replace
transparent compressions, efficient storage for small files
Soon
Fixes for perf and stability
background scrubbing, LZO compression, batched discard, file defrag options, per-inode flags
Larger block sizes for ( especially for metadata, to provide perf improvements in some cases)
Scrubbing uses CRC to varify data on disk, fixes bad ones with good copy on another disk which has okay CRC
“df ” gives completely wrong values on how full the disk is or it’s size
discard/trim supported both real time and batched
Drive Swapping – current raid rebuilds via balance code, can also restripe between RAID levels
btrs send/receive in development
Embedded – friendly to small machines, not as friendly to small disks (being worked on) Works well with low-end flash drives
RAID 5/6 – MErge pending completion of fsck work. will also add triple mirroring
beta read-only filesystem recovery tool – copies data out of corrupt FS
tree root-history log lets us recovery from many hardware errors. “mount -recover”
New fsck release on the way. May be announced very soon
yum-plugin-fs-snapshot . yum plug to trigger a snapshot on package install/upgrades

The Web as an application development platform – Shane Stephens and Mike Lawther

When and how to move from native to web/cloud apps
Examples
Text based like email (pine, mutt, outlook). 15 years ago web-based email started. access anywhere, no install required, easy to start using, use anywhere
Desktop publishing. Hard to collaborate with other people using desktop based software by emailing docs around to other people. On the web it’s often native multi-user. Send Links rather than emailing whole doc.
On web everybody always running latest version
Github gives you one-stop shop for projects. wiki, forums etc. Native in app rather than bolt on to desktop version
Graphics
Flickr – has native sharing, backups etc
Online editing of images etc now possible
Games
Farmville – no install, easy to share links
Angry Birds – has web version, html5 , flash for sound, 60fps
Not at stage where First person shooters going to happen yet. Users have high end hardware
Overall Benefits – No install, universal access to data, always using latest version, collaboration and sharing built in. Simple text layout, Web as IDE, open and modular enviroment
Drawbacks – Layout more involved than desktops apps, distributed code makes debugging hard, cross browser compatibility, security limits flexibility.
Useful web technologies (see also HTML5Rocks website) :
Display / Rendering
HTML , SVG , Canvas – All fairly easy to combine
WebGL , flexbox and grid <- future
Communications
Standard http requests, AJAX / XHR , Websockets / Browser Channel , libraries like Faye etc
Storage
Traditionally just cookies, Session Storage, Local Storage, indexDb, AppCache
Environment Enhancements
jQuery , CoffeeScript , NaCl

Simon Lyall: LCA2012 – Tuesday after Lunch

Tue, 17 Jan 2012 04:03:58 +0000

Mistakes were made by Selena Deckelmann

Misc management
Prepare for failure, Failure is an option (it will happen)
Book: “Everything is Obvious”
2 examples (NZ and Scotland) of rats gnawing through cables and taking out country
Document -> Test -> Verify
Failure to Document
Write Docs, Update Documentation, Make documentation a step with your written processes, assign some time to that step.
Doc Tools: Graphic Designers, wikis, sphinx, diagrams – timelines – bug tracking – ordered todo lists
Failure to test
Verify success criteria – Write tests – test with buddy – have a plan
testing frameworks, staging environment, repeatable shell scripts
Failure to verify
Have a plan for things going wrong – have staging environment – test rollback plan, not just implementation plan
Tools – People, staging environment
Failure to imagine
share stories of failure – talk to people are different from yourself – act out implementation scenario
Failure to Implement
reflection ( post-mortem )
Plan to do post-mortem, document the plan with numbered steps and a timeline – test plan & rollback plan – Identify point of no return
During – screen sharing – chatroom – Voice – Headsets – Designated time-keeper

Scaling Openstack by James Blair and Monty Taylor

6 projects in openstack.
collection of related repositories
Most contributors paid to work on it by their companies
number , quality and area or contributors varies
6 monthly releases – design summits – continuously open truck – dev on master – Monthly milestones – stable branches post release
Vision – consistent tooling and process on all projects -> Consistent Product -> Multiplier effect.
Minimize meta-development, Standard tools
Gerrit – code review
Jenkins – Testing (pre and post merge)
Orchestra (bare metal deployment)
Lanchpad, documentation servers, planet, repos
Environment: Ubuntu, Everything in Python (pep8 standard, openstack.common ). virtualenv/pip
Gated truck – ensure quality – auto tests – means devs always start from working code – keeps bad code out of tree – process same for everybody, transparent, automated.
Gerrit – stand-alone patch review system – lots of integration hooks – lots of review categories
SSI using openid for all of project sites
Git review is implemented as git sub-command to submit things to gerrit. zero-config <- looks cute
Vendors can have labs and tests and code can be automatically submitted and tested on it

Jonathan Oxer: Walktime Blog #16: Arduino Miniconf 2012

Tue, 17 Jan 2012 02:50:24 +0000

A quick walk-around of the hardware assembly tutorial at the 3rd Arduino Miniconf, which took place on January 16, 2012 as part of linux.conf.au.

View or comment directly on YouTube: www.youtube.com/watch?v=xic4w5DL-tw

Links for this ep:

Simon Lyall: LCA2012 Tuesday – Before Lunch

Tue, 17 Jan 2012 01:04:21 +0000

I’ll be posting some updates from Linux.conf.au in Ballarat in the next few days. I arrived on Sunday but this is my first post. I’ll see if I do an out-of-order blog post on that later.

The keynote from Bruce Perens was really good. He talked a lot about the failure of open source to engage the public and policy makers. He also covered a bunch of cool hardware projects and cheap (and often open) small boards and other electronic products.

Smashing a square peg into a round hole – David Basden and Chris Collins

Automation your automation.
Anchor – Hosting provider, doing built solutions, non-standard requirements
Puppet is one step, PXE & dbootstrap just another, hundreds of others
A “simple” build used to take a day, down to 10 minutes
instead of defining all steps define dependencies to get a “partial ordering”
figure out what has to be done, in what steps, keep track of what has been done
Lack of security in many queuing systems, bad agents can grab tasks they aren’t supposed to etc
Outline of “Audience” job control system. Design goals and decisions
Code on github.net/anchor

Extracting metrics from logs for realtime trending and alerting by Jamie Wilkinson

passive system, query application about it’s metrics. However sometimes hard to parse
However most apps log to system or other logs
emtail – exporting modular tail
plugins – on event X do Y – usually regex
Metrics are values, times ( name, value, when, type (counter, gauge) , string, tag
metrics are exported over a common protocol ( google protocol in G version, json in open source version )
exported over http using json or CSV, sent to something slse to save and do something with
Written in Go. Old version in python but too google specific
20 minutes talking about the source code (I browsed the web during this bit)
I and a few others in the audience seemed to think this duplicated a lot of other tools with no obvious huge advantage over them. Bit of google NIH.

leigh morresi: Drupal 7 and XMLRPC example

Tue, 17 Jan 2012 00:25:15 +0000

Tim Connors: Breaking windows

Sat, 14 Jan 2012 00:35:11 +0000

Another letter in The Age today. Unedited text below:

Ian Porter (Without car manufacturing, we are on the road to ruin, The Age, 13 Jan) believes that the government needs to keep throwing money at the car industry in order to support other industry in Australia. I'm surprised as an industry analyst, he hasn't heard of the broken window fallacy.

Throwing good money after a bad unsustainable industry that can't adapt is just a waste. It's exactly identical to sending soldiers to dig holes only to fill them back up again just to keep them employed and off the streets. The money could be better spent on doing useful things that will remain useful into the future. Yes, paying people to break windows and then paying the glazier to repair them will keep people employed, but couldn't the glazier be better employed building things that then keep other people employed into the future?

Why don't we do something useful with the money instead? Like built modern intra- and inter-city rail infrastructure? This won't become stranded assets when cheap oil becomes unavailable. We won't be left with vast tracts of useless motorways - we will continue to be able to use the rail infrastructure well past these boom times.

LCA2009 News: Outstanding talks to take centre stage at linux.conf.au 2012

Wed, 11 Jan 2012 05:58:15 +0000

Ballarat, 11 January 2012

With only four days to go until the opening of Ballarat's linux.conf.au 2012, the conference is quickly becoming the talk of the town. Naturally it is the talks themselves that make the conference and 2012 gives delegates more chances to see more talks!

As Conference Director Josh Stewart explained, Delegates are sure to be impressed with the array and calibre of Speakers.

"We're really fortunate this year to be able to bring so many renowned community luminaries to Ballarat to share their experience and knowledge with delegates."

Tuesday will see Co-Founder of the Open Source Initiative (OSI) and creator of the Open Source Definition, Bruce Perens, given the first conference keynote. Other highlights from Tuesday include Jonathan Corbet's now-traditional kernel report, presentations on optimising web performance by Lenz Gschwendtner and even a talk on high altitude ballooning by Joel Stanley and Mark Jessop. Laura Thomson and her colleagues from Mozilla will also present on migrating enormous volumes of data.

Australia's favourite mad scientist, Perl developer Paul Fenwick, will keynote Wednesday's line up. He will be followed by, among others, Alice Boxhall and Silvia Pfeiffer presenting on developing accessible web applications, a shell tutorial by University of New South Wales computer scientist Peter Chubb and a talk on mentoring in free and open source (FOSS) communities by Open Source Outreach Manager for Oregon State University’s Open Source Lab, Leslie Hawthorn. Linux Australia's Annual General Meeting (AGM) will also be held as part of Wednesday's schedule.

On Thursday, open source luminary, ex-Software Freedom Law Centre counsel and current Executive Director of the GNOME project, Karen Sandler will keynote. Other highlights from Thursday's programme include a presentation on the Android open accessory development kit from open hardware guru Jon Oxer, talks on copyright and ending software patents by Ben Powell and Ben Sturmfels. Founders of the Ada Initiative - a group dedicated to furthering the representation of women in open technology and culture - Valerie Aurora and Mary Gardiner will also present around women in open source.

Thursday night's Penguin dinner will also play host to an as yet unannounced mystery speaker!

Significant interest surrounds Friday's keynote speaker, Jacob Appelbaum (better known as @ioerror on Twitter). Widely noted for his interest in computer security and stance against censorship, Jacob was one of the team behind exposing vulnerabilities in hardware after physical shutdown - the so called 'cold boot' attack.

Part of the final day of the conference, Friday 20th January, has been dedicated to showcasing 'best of' talks. A common lament of linux.conf.au delegates from years past has been the disappointment in missing 'the presentation' that everyone is talking about. The Best-Of slots allow for these amazing talks to be experienced by as many delegates as possible (or even experienced again for those who were blown away the first time around).

linux.conf.au will be held 16th-20th January 2012 in Ballarat, Victoria, Australia. Registrations are still open, but places are now limited.

Web: http://lcaunderthestars.org.au
Twitter and Identi.ca: @linuxconfau
Registrations: http://linux.conf.au/register/prices
Tentative schedule: http://linux.conf.au/programme/schedule

Media enquiries are warmly welcomed to media@lcaunderthestars.org.au

LCA2009 News: linux.conf.au 2012 set to shine in Ballarat

Tue, 10 Jan 2012 11:27:20 +0000

Ballarat, 11th January 2012

With only days to go until the covers come off 2012's linux.conf.au at the University of Ballarat, excitement is definitely mounting. The conference will welcome four stellar keynote speakers, including internet freedom and anti-censorship campaigner Jacob Appelbaum, GNOME Executive Director Karen Sandler, noted open source campaigner Bruce Perens and Australia's favourite mad scientist Paul Fenwick.

Preparations are in full swing. Conference Director Josh Stewart is particularly pleased with progress to date.

"The Organising Team are an amazing group of people - they're just so dedicated. Everything is coming together well, and we're looking forward to a hosting a conference that is thought provoking, entertaining and most importantly of all, great fun".

One of the challenges the conference team faced in Ballarat was ensuring that delegates had no troubles reaching the venue, with the city being located around an hour out of Melbourne. Stewart was confident that this focus paid off

"We've organised buses to ferry people both from Melbourne Airport and from Ballarat Train Station to the University and other accommodation venues. We have a conference Wiki page to allow people to arrange shared rides, hangout with other delegates on the train or find out the best way to get from A to B".

The first day of the conference, Monday 16th, will play host to eight Mini conferences - sessions focussed around specific topics. The 2012 Miniconferences include:

Sysadmin
Open Programming
High availability and distributed storage
Arduino open hardware
Business of Open Source
Music and Multimedia
Browser Miniconf
Business of Open Source
Haecksen / LinuxChix

The remaining four days of the conference will see over 70 presentations by notable speakers in the Linux and free and open source software (FOSS) community. Industry luminaries including Jonathan Corbett, Florian Haas and Jonathan Oxer are all confirmed.

"The Speaker line up is simply amazing. We've got something for everyone, ranging from deeply technical tutorials to presentations covering legal aspects of open source software and hardware" noted Stewart.

linux.conf.au will be held 16th-20th January 2012 in Ballarat, Victoria, Australia. Registrations are still open, but places are now limited.

Web: http://lcaunderthestars.org.au
Twitter and Identi.ca: @linuxconfau
Registrations: http://linux.conf.au/register/prices
Tentative schedule: http://linux.conf.au/programme/schedule

Media enquiries are warmly welcomed to media@lcaunderthestars.org.au

Colin Charles: Growing out of Foursquare

Sat, 07 Jan 2012 13:04:45 +0000

It seems like after two years of using Foursquare, I’ve reached check-in fatigue. I’ve written my thoughts before after a year of usage.

Today I don’t turn the app on. I visit places but I can’t be bothered about checking in. If I’m visiting a new place, I might fire up the app to check on reviews. And I will continue using it for the Topguest integration. Beyond that, I can’t imagine why I’d be firing it up.

Some reasons to my lack of continued interest in Foursquare:

if I wanted to meet you, we’d be meeting. Coming to “catch” me at a place is not useful for me. I fixed this by checking into locations after I’d left.
many duplicate venues. Tips spread across all of them. No way to clean it up.
tips becoming increasingly less useful. People aren’t using it, ads are coming in, etc. frankly I’d like some way to not just see what my friends recommend, but friends whom have good taste/tastes similar to mine
hardly many establishments even care to offer check-in rewards, mayor discounts, etc. in fact the establishments in Malaysia/Singapore are easily countable. This I attribute to the company being disinterested in penetrating the market – I hope google places and Facebook check-ins become more useful as they both have local offices
after a while, the gamification, badges, etc just get boring
frequent roaming means I don’t frequently have a data connection enabled, which has helped me deplete my usage of Foursquare

In short, I don’t see the value from the application. I know that when I’m in San Francisco I can see value. Budapest surprised me with value again. But generally, it seems like value is tough to come by.

From a merchant standpoint, I’d like more control. If I know people frequent an area, I’d like to tell them about my establishment.

I do find the Explore function quite useful when I’m in a new city or I’m just looking for something to do. Sometimes I use it to gauge the parking situation at certain malls (a 10km radius from where I live covers some rather popular places :P). But it could be more useful again. Let’s say I want to search for the term “pork satay” in a 10km radius. Some people spell things like “sate”. Sometimes there’s no tips. Shops don’t normally label themselves after a food even though that might be their main pull.

Addresses are generally incomplete. Map locations can be wayward. Phone numbers and opening hours are non-existent.

The problem for me is largely dirty data. Foursquare would be a lot more useful if it were edited. And provided to me information, rather than data.

The idea is that dirty data gets fixed via crowd-sourcing. Get your users to do the work for you! Have you seen the lengthy application process to become a Superuser Level 2? Compare that to how easy it is to edit Wikipedia.

There used to be a movement to clean up Foursquare locations in KL. Eventually though, I think the users moved on, found busier jobs, and life took over. Crowd sourcing works; barriers matter.

Facebook and Google are at their primes here. They can win by providing information, rather than data. User generated comments are always useful, but building further filters with a wider network probably helps. Besides, I bet there are more connections on Facebook and more loose connections on twitter and google plus, in comparison to Foursquare for most. Google is already pushing getting merchants online, why not make them also focus on Places?

In the meantime, Foursquare is still on my phone. It’s usage is just severely reduced.

Colin Charles: New year, going on 8 years

Sat, 07 Jan 2012 11:40:46 +0000

It’s Week 1 of the New Year. I hit the ground running. No end of year celebrations as the start of the year brought setting up a retail presence at 7am!

This blog has been around since January 2004. Eight years of constant ramblings. I’ve written a lot less in recent times, preferring to go short form and tweet instead. But I keep coming back here due to the permanent nature of writing on something that I own and control.

I can now write on my phones, iPad, or laptop so I’m beginning to think I should focus on writing a lot more. Distilling thoughts.

2012 is expected to be exciting. If the first week has anything to show for it, it’s going to be very little rest, but lots of fun doing tonnes of amazing things. Opportunities for growth are endless.

2011 was awesome, in a nutshell. I should really look into writing a retrospective, but I’m too busy focusing on the present & future. I travelled a lot, saw many new places, made many new friends, and really enjoyed quality time spent with the people I care about.

As for this blog, it’s going to go back to becoming an activity log of sorts. Back to its roots 8 years ago. It will become more personal and dilution with posterous will stop (now to import all my previous content).

2012 however is going to be the year I do experiments in new content & media. So prior warning that if you’re on rss, there’s probably going to be more activity.

Here’s to an awesome 2012. Make the most of your remaining 51 weeks. Remember to just do awesome stuff.

Colin Charles: Piracy due to lack of legal options

Sat, 07 Jan 2012 10:58:11 +0000

“If you give people a legal way to consume the content they want, they will pay for it. But when you make it impossible to legally consume the content they want, they will pirate it.” – Fred Wilson #screwable

I’ve said this many times. The world is flat. There is no worldwide currency so you can’t use exchange rates to justify higher costs in different markets. People have friends around the world, and they’d like to talk about that episode of House with their friends worldwide after they’ve seen it.

Give people sensible legal options. Don’t censor content (really, rely on the ratings).

Markets in Asia today have no streaming tv/movie content. No streaming radio. No purchasing of legal music. There is so much growth potential. If only the industry sees it. The technology is already here. Profiteers are what’s blocking progress.

Sridhar Dhanapalan: OLPC Australia XO-AU OS 12 beta 1

Sat, 07 Jan 2012 07:37:16 +0000

The OLPC Australia XO-AU OS 12 has reached beta 1. This is based on OLPC OS 11.3.1 and Dextrose 3.

We’d really appreciate some testing. Please direct your feedback to the OLPC Australia mailing list.

Here is the notice I sent out to teachers:

———- Forwarded message ———-

From: Sridhar Dhanapalan

Date: 24 December 2011

Subject: Taking part in improving new XO software

Friends,

The 2012 OLPC Australia operating system, XO-AU OS 12, has reached a

beta stage of development. It has many improvements, and we looking

for feedback on how it works to help us create the final product. This

beta is suitable for testing, documentation and developing lesson

plans.

In early February, we will have a near-final release candidate,

suitable for trialling in classrooms. We are looking for clever

teachers to provide us with real-world feedback on how the software

works with their classes.

This is an opportunity for you to take part in XO development and

ensure that the device suits the needs of your classroom. We would be

especially interested to know how the connectivity and collaboration

works on your school’s networks.

To get started, visit our release notes page:

https://dev.laptop.org.au/projects/xo-au/wiki/120_release_notes

This page outlines the main changes in the new operating system. Go to

the “Beta 1″ part of the Installation section. Installing the beta is

no different from installing the XO-AU USB 3 stable release: extract

the zip file to a USB stick and you’re ready to go.

To provide feedback, join our technical mailing list:

https://dev.laptop.org.au/projects/general/wiki/Technical_mailing_list

Following this, you can send your comments or ask questions at

olpc-au at lists dot laptop dot org

The OLPC Australia Engineering team are active participants on this

list, and we will reply. Remember, the better you can help us with

quality information, the better we can make the product for you

Regards,

Sridhar

Jonathan Oxer: Walktime Blog #15: Arduino light switches

Fri, 30 Dec 2011 12:29:50 +0000

As part of my @SuperHouseTV home reno the entire house has been rewired, including replacing all the light switches with Freetronics Arduino-compatible control surfaces running on the LAN using Power-over-Ethernet. To save time assembling all the switches I designed a custom PCB for the control surface.

View or comment directly on YouTube: http://youtu.be/MW3Vs2mHies

Links for this ep:

leigh morresi: Easy win for lowering PHP memory usage (drupal)

Mon, 26 Dec 2011 16:13:29 +0000

Keith Packard: calypso

Fri, 23 Dec 2011 06:49:15 +0000

Calypso — CalDAV/CardDAV/WebDAV for Android and Evolution

Ever since I bought my first Palm Pilot in 1997, I’ve relied upon a pocket-able device to carry a copy of my calendar and contacts, and for that same database to be present on my laptop. I went through a long list of Palm-compatible devices, including both the Palm Treo and Palm Centro telephones. I even wrote a number of my own Palm applications. Years ago, it was pretty obvious that I’d have to find a new phone, but I was stuck looking for something that could provide the same ‘hot-sync’ functionality.

SyncML on the Series 40

I bought a Series 40 Nokia phone in Shanghai that promised ‘SyncML’ support. Given that I had seen numerous SyncML implementations for Linux, it seemed like I should be able to get something to work.

SyncML on the Series 40 is a disaster — the phone couldn’t actually store all of my contacts, and it couldn’t hold half of the data fields I used. So, that phone landed in the box and back to the Centro I went.

SyncML on the N900

Nokia kindly sent me an N900 at some point, so I gave SyncML another try. Given that the N900 runs evolution-data-server, and that I’ve had evolution-data-server running on my laptop, it seemed like I should be in business. Well, almost. It took several days of hacking to fix bugs in the evolution SyncML back end, then another several days fussing with opensync.

I ended up abandoning direct synchronization as unworkable — opensync would sit in an infinite loop, or worse, trash my database completely. I finally found ‘syncml-ds-tool’, which is a debugging tool that comes with opensync. This tool simply synchronizes a set of disk files, one per contact or calendar entry, with the phone. That worked for well over a year. And then, a few months ago, Bluetooth on the laptop stopped connecting with the phone’s SyncML server. I’d get ‘ECONNREFUSED’ every time I tried to use it. So much for the N900. DUN still worked, mostly, although it too would get ECONNREFUSED at times, but retrying seemed to make it work.

However, While the N900 SyncML solution worked, I discovered another thing I wanted—contacts and calendar entries stored in individual files and revision controlled with git. This makes it reasonable to delete stale calendar entries and know that they’re never really gone, just left behind in an older version of the calendar. And, if you mess up, you can recover by poking at the database with git directly.

I switched back to the venerable Palm Centro; it turns out that calendar and contacts are more important to me than being able to surf the web on my phone. Alas, my Centro went ‘swimming’ in August and has passed on to the great electronics recycling house in the sky. I pulled the SIM out and switched back to the N900.

I got my contacts imported on the N900 by copying files over the net work; not a long-term strategy, but at least I had phone numbers again. There was no hope for my calendar. I started looking for a solution in earnest.

How about Android?

At this point in my story, I’m sure you’re asking why I didn’t just use one of the numerous Android phones that came through my hands. The answer is simple — my calendar and contacts are probably some of my most personal data, and I’m not willing to store them outside of my direct control, for reasons similar to those which are driving the development of the FreedomBox.

When Android first came out, it could only talk to Google services, which didn’t meet my hard requirement for personal data storage.

One of my co-workers had his Google account suspended for violating the terms-of-service; he asked what he had done, but they wouldn’t say. He asked if he could get his data back, and they said “no”. They invited him to create a new account, but it would not ever get any of the old data. A few days later, he got a nice apologetic email letting him know that they’d made a mistake and that he hadn’t, in fact, violated any of the terms-of-service, and that his old account was restored with all of his data intact. Wasn’t that nice of Google?

WebOS?

I got a WebOS phone over the summer and discovered that while it had multiple contact/calendar back ends, none of them used standard protocols and so you only had the choice between multiple corporate data centers, which isn’t actually a choice at all.

Furthermore, the WebOS phone refused route PAN packets over the phone network, even though I have a data plan which allows this. It’s not that it couldn’t support it, it’s that it refused.

A couple of weeks after the WebOS phone arrived, HP canceled all of their WebOS hardware products, which made me less interested in trying to solve this problem.

Android recovers

About the same time the WebOS phone arrived, I discovered that Google had published enough information about the calendar/contacts internals for Marten Gajda to write CalDAV-Sync and CardDAV-Sync. And then, Andrew McMillan wrote aCal, which is a complete replacement for the built-in calendar and contacts applications and supports CalDAV and CardDAV.

With two different standards-compliant solutions available, it seemed like it might be time to try Android again.

I’d love for CardDAV-Sync and CalDAV-Sync to become free software like aCal is. Andrew makes money from aCal by offering it for sale via the Android Market, while still publishing the sources for those who want to build their own copy.

CalDAV/CardDAV on Linux

I think the most widely known CalDAV server for Linux is probably DAViCal, a huge pile of PHP and SQL sitting on top of Apache. I’m sure it’s suitable for running on a server and being accessed over the internet, but I’m not interested in that, nor am I interested in having my laptop run Apache and PostgreSQL.

I found a tiny little CalDAV server, Radicale, which seemed like a lot better fit. It’s written in Python and uses the usual Python HTTP server infrastructure, which provides SSL and authentication support along with some fairly convenient APIs for parsing and generating HTML.

Before long, I discovered that Radicale was actually too simple for my needs. It stores the whole calendar in a single file, re-parsing it whenever a request is made, so a calendar with just hundreds of entries caused the server to slow down enough that evolution would time-out when talking to it.

Also, Radicale doesn’t actually parse the calendar entries completely, it has some ad-hoc code that finds various pieces of data, but without dealing with the whole syntax.

I started hacking at Radicale to see how far I could get. I changed the storage code to store one event per file, then added hooks to use git for change management. Then, I found a full vcalendar/vcard parsing library in python, vobject, which I used to replace the ad-hoc parsing code. Finally, I added support for VCARD entries as well, allowing the system to store both calendar and contact information.

Introducing Calypso, a CardDAV/CalDAV server

With this much divergence from the original project, I’ve figured I’d best rename things to avoid confusion, so I decided to call it ‘calypso’, after a brief trip through the dictionary looking for names starting with ‘ca’.

Calypso works with evolution, iceowl and the Android CalDAV/CardDAV plugins. It does not yet work with aCal; for some reason aCal cannot find any calendars on the server.

Calypso also supports importing calendar changes from the command line, allowing you to integrate support into a text-based email application like notmuch or mutt.

Calypso is available via git from git://keithp.com/git/calypso and is distributed under the GPL (v3 or later). I still consider it a work derived from Radicale, and so the code retains all of the Radicale copyrights along with my own.

Using Calypso

Initial setup

Calypso runs as a regular user, all data are stored in ~/.config/calypso. To initialize calypso:

$ mkdir ~/.config/calypso ~/.config/calypso/calendars
$ cat > ~/.config/calypso/config << EOF
[server]
ssl=true
certificate=/home/keithp/.config/calypso/ssl/server.crt
key=/home/keithp/.config/calypso/ssl/server.key

[acl]
;type=htpasswd
type=fake
filename=/home/keithp/.config/calypso/passwd

Running calypso

Then run calypso:

$ python ./calypso.py

No, I haven’t figured out how to install it…

Creating new calendars

To add a new database:

$ mkdir -p ~/.config/calypso/calendars/private/my_calendar
$ cd ~/.config/calypso/calendars/private/my_calendar
$ git init
$ git commit --allow-empty -m'initialize new calendar'

The new calendar should now be visible as https://localhost:5233/private/my_calendar

You can add files to the directory at any time; calypso will check the directory mtime at each operation and update its internal state from that on disk automatically when the directory changes.

Importing files

Given a set of files with VCALENDAR or VCARD entries, you can import them with:

$ calypso --import private/my_calendar <filenames...>

This will update any changed entries and add any new ones.

ToDo list for calypso

Document the config file contents.
Make it installable
Figure out what aCal wants
Support calendar creation

More Android info

Synker. This provides a desktop widget which starts the sync process running on a list of accounts. This makes it easy to manually synchronize with the laptop when you are connected
Contact Editor. This lets you fully edit contacts synchronized over CardDAV. The built-in contact editor doesn’t let you change anything other than the name for some reason.

I’m running cyanogenmod on my Nexus S as that provides PAN support. With PAN, I can create a network link between laptop and phone which doesn’t depend on any local WiFi infrastructure and which gives both phone and laptop static IP addresses, allowing me to configure the sync URLs statically on the phone. I’d use mDNS, but Android doesn’t bother to support that.

Jonathan Oxer: Walktime Blog #14: Maker Faire Melbourne, Arduino Miniconf

Wed, 21 Dec 2011 13:18:19 +0000

Maker Faire is coming to Melbourne! Followed 2 days later by the Arduino Miniconf at linux.conf.au 2012 in Ballarat, it'll be a great time to be in Victoria if you're into Open Hardware.

Comment or view directly on YouTube: http://youtu.be/aiZCufdgc9c

Links for this ep:

Mini Maker Faire Melbourne: www.makerfairemelbourne.org
Arduino Miniconf: www.arduinominiconf.org

Sridhar Dhanapalan: OLPC Australia talk at OSDC 2011

Sun, 18 Dec 2011 05:17:15 +0000

Update: my talk has been covered by OLPC News.

Here’s the video of the talk I said I’d be giving at OSDC 2011, titled Australia’s Toughest Linux Deployment:

In it, I outline our educational programme and how the technology fits into it. Some key points:

we have a better version on YouTube of the video I show in the talk
we maintain a Policy Document, which provides an overview of our overall programme
OLPC Australia have two core principles in addition to OLPC’s original five
we have some support in government at different levels — for example, we were praised in federal parliament and the print media (paywall) by a prominent federal Member of Parliament
we have deployments across remote Australia — a feat that can only be managed through building self-sufficiency
our programme is showing beneficial results, and we are engaged in longitudinal and detailed evaluation
we have a comprehensive educational programme, with online training and certifications (such as our XO-cert course)
we are breaking dependence on special expertise and infrastructure — building sustainability and grass-roots support is key
deployments are made at the classroom level, which is more manageable than saturating a whole school at once
we don’t provide XOs without training — a teacher must earn a certification before they can receive XOs for their class
our support is focused on enabling schools and communities to help themselves, and each other
we have innovated in the technology space, with offerings such as the XO-AU OS, XO-AU USB, XOP and XS-AU
contextualising learning, for example through localisation, is a powerful tool to improve engagement from the child, school and community
we invite people to join our development efforts
there’s a nice surprise mentioned towards the end, which I shall elaborate upon in the near future

For those of you who have seen me speak about OLPC Australia at SLUG, this is a much more polished talk.

Tim Ansell: FastComplete, making bash completion fast on remote file systems

Fri, 16 Dec 2011 04:05:46 +0000

Here at Google we have quite a few remote file systems which contain various tools we use in our day-to-day work. As typing sucks we generally want the tools in our $PATH. When you try to tab complete Bash needs to stat a couple of thousand files and even on fast remote file systems this takes a drastically long time.

I wrote FastComplete as a solution to this problem. The tool creates a local cache of links on your hard drive to everything in your $PATH. It uses a couple of tricks to make sure all the stats remain locally, while still allowing the remote file to change without needing to update the cache. Linux should also keep this information in memory disk cache making tab completion almost instant again. Yay!

You can find FastComplete at https://github.com/mithro/rcfiles/blob/master/bin/fastcomplete It is a stand alone python program which shouldn’t have any non-core dependencies. The usage documentation is as follows;

Fast complete creates a local disk cache of your path.

It’s specifically designed to make bash tab complete run much faster. The correct fix would be to add caching to bash, but it was to hard to do so.

To find out what path fastcomplete is currently using:
> ~tansell/bin/fastcomplete
# Found 3977 commands
export PATH=/home/tansell/bin: ... :/home/build/google3/googledata/validators:/home/build/google3/ads/db
To get fastcomplete to rebuild it’s cache:
> ~tansell/bin/fastcomplete --rebuild
# Using path of '/home/tansell/bin: ... :/home/build/google3/googledata/validators:/home/build/google3/ads/db'
# Cache /usr/local/google/users//tansell/tabcache/d7e5fb63454ae33b4a171b6437be904a did not exist! Rebuilding....
# Looking in: /home/tansell/bin (execv)
...
# Looking in: /usr/bin (symlink)
...
# Looking in: /home/build/google3/ads/db (execv)
# Found 3977 commands
export PATH=/usr/local/google/users//tansell/tabcache/d7e5fb63454ae33b4a171b6437be904a
To use fastcomplete all the time add the following as the *LAST* line in your ~/.bashrc file. Fastcomplete will echo some output to stderr so you can see what is happening.
# Create a cache of the command
eval `~tansell/bin/fastcomplete $PATH`

leigh morresi: Some good simple Drupal SEO

Thu, 15 Dec 2011 14:47:03 +0000

Jonathan Oxer: Walktime Blog #13: Cool tools - SSH

Thu, 15 Dec 2011 10:42:19 +0000

Another cool tool that can help break you free of any particular computer is SSH. It's for more than just getting a shell on a remote machine though: it can also do X session forwarding, and local and remote port forwarding.

Note: Before anyone accuses me of doing anything nasty to Google's corporate network, I didn't do anything I shouldn't and I expect that Google's network is as tight as tight can be. My comments about networks with crispy shells and soft centers weren't meant to imply that's what Google's are like, but after listening to the video it sounded like it may have come across that way. Sorry!

Comment or view directly on YouTube: http://youtu.be/MzwevAJ0rwA

Links for this ep:

Mary Gardiner: An appeal for the Ada Initiative

Wed, 14 Dec 2011 21:00:13 +0000

When I was 15 I went on the web for the first time. A boy in my computing class went to Yahoo!, typed in “girls” and spent some time showing me porn.

I’ve programmed since I was a kid. I’ve loved the idea of open technology since I read a curious article in the 1990s about people all over the world, fixing complex bugs in an operating system that a university student had named after himself.

But every so often, I’m reminded how my Internet experience began. Women friends haven’t been safe on mailing lists, they haven’t been safe on Wikipedia’s talk pages, and they haven’t been safe at conferences. And even when they are safe, sometimes they’re lonely: estimates of women’s participation in open source run to about 2%, and as Wikipedia editors at 9%.

Thus, I’ve been a volunteer creating communities by and for women in open source since 2000. It’s been the equivalent of an unpaid part-time job for several of those years. But a year ago, Valerie Aurora became more ambitious, and proposed that since we were doing real work, we should do it as our real job. Together we created the Ada Initiative, a non-profit supporting women in open technology and culture. We rely on your support for our work:

Within a year we’ve organised our first AdaCamp, surveyed thousands of people about their perspective on women in open technology and culture, wrote and encouraged adoption of an anti-harassment policy by over 30 conferences and organizations in open tech/culture, and much more.

To continue our work in 2012, we need your help! Please donate to the Ada Initiative, and contribute to our planned work, including future AdaCamps, methodologically rigorous research into women in open source, and training for women contributors to open tech/culture projects and their allies.

Donate now: we can’t do it without you!

LCA2009 News: Humanitarian FOSS Miniconf cancelled

Tue, 13 Dec 2011 12:21:43 +0000

The core organising team of linux.conf.au 2012 wishes to advise participants that the Humanitarian Free and Open Source (HFOSS) Miniconf has been cancelled due to a lack of talk submissions.

However, this leaves eight Miniconfs going ahead, including;

Arduino
Sysadmin
Haecksen / LinuxChix
Music and Multimedia
Business of Open Source
High availability and storage
Open Programming
Browser

The team would like to send a special thanks to Tim McNamara for all of his efforts in attempting to bring the HFOSS Miniconf to fruition.

Jonathan Oxer: Walktime Blog #12: Cool tools - Git, Dropbox, Google Docs

Wed, 07 Dec 2011 04:18:10 +0000

I hate the idea that everything I work on is trapped inside a specific computer, and that if I don't have that computer with me I can't do anything - or worse still, if it's stolen or explodes I'm in big trouble. I've (mostly) managed to make myself non-computer-specific using a combination of Git, Dropbox, and Google Docs.

Comment or watch directly on YouTube: www.youtube.com/watch?v=7Y-iKRiVizQ

Links for this ep:

Mary Gardiner: Interested in women in open tech and culture? AdaCamp Melbourne wants you!

Wed, 07 Dec 2011 00:00:31 +0000

My non-profit organisation, the Ada Initiative, wants to go full steam ahead into 2012, and we’re holding an AdaCamp event in Melbourne to kick off the year!

The Ada Initiative supports women in open technology and culture, ranging from open source to free culture to grassroots community organising to makerspaces to remix and fandom culture to open government initiatives and more. This stuff is powerful: it’s already shaping society and is going to continue to do so more and more. The Ada Initiative is focussed on supporting women in becoming an integral part of these communities.

AdaCamp will be a one day “unconference” (that is, it will have free-form sessions scheduled by participants) focussed on furthering women’s work in open technology and culture. It will be held on Saturday January 14 in Melbourne, some travel funding is available.

AdaCamp places are by invitation, if you’re interested in coming along please apply today. Applications close December 14. Hoping to meet some readers and ‘net friends there!

Tim Ansell: A typewriter that types colors!

Mon, 05 Dec 2011 22:19:40 +0000

A typewriter that types colors!

Janet Hawtin originally shared this post:

http://tyreecallahan.tumblr.com/post/13602352689/this-is-the-chromatic-typewriter-my-entry-to-the

Tyree Callahan | This is the Chromatic Typewriter, my entry to the…

This is the Chromatic Typewriter, my entry to the 2012 West Prize competition. The prize is awarded via popular vote this year. Click the source to get to the West Collection and to download the West….

Planet linux.conf.au 2009

Sridhar Dhanapalan: Creating an Education Programme

Investment in our Children’s Future

Comprehensive Education Programme

Technology as an Enabler

Evaluation

Join our mission

Credits

Colin Charles: MariaDB/MySQL users in Paris & Brussels

Colin Charles: Movies, January 2012

Gabriel Noronha: Getting ClearOS to work with Atheros Communications AR8151 v2.0 Gigabit Ethernet (rev c0)

leigh morresi: First android app released

Jonathan Oxer: Walktime Blog #19: No dominant Open Source home automation projects?

Jonathan Oxer: Walktime Blog #18: Interview with Marco Ostini at LCA2012

Jonathan Oxer: Walktime Blog #17: Interview with Matt Evans at LCA2012

Jeff Waugh: QotD: Jon Corbet on linux.conf.au and Linux Australia

Colin Charles: Popular Malaysian passwords (sample size=75,000)

Colin Charles: On killing Hollywood

Simon Lyall: LCA2012 – Friday after lunch

Codec 2 – David Rowe

UEFI and Linux – Matthew Garrett

Simon Lyall: LCA2012 – Friday Morning

Bloat: How and Why UNIX Grew Up (and Out) – Matt Evans and Rusty Russell

Open vSwitch – Simon Horman

Simon Lyall: LCA2012 – Thursday last session

Challenges for the Linux plumbing community – Jonathan Corbet

What is in a tiny Linux installation? by Malcolm Tredinnick

Simon Lyall: LCA2012 – Thursday after lunch

Simon Lyall: LCA2012 – Thursday Morning

Cheap Tabloid tricks – Angus Kidman

Colin Charles: SCALE 10x – there’s lots of MySQL there!

Tim Connors: An open letter to Peter Ryan regarding police treatment of cyclists

Simon Lyall: LCA2012 – Wednesday last session

Simon Lyall: LCA2012 – Wednesday after Lunch

Simon Lyall: LCA2012 – Wednesday Morning

A Tour of btrfs – Avi Miller

The Web as an application development platform – Shane Stephens and Mike Lawther

Simon Lyall: LCA2012 – Tuesday after Lunch

Mistakes were made by Selena Deckelmann

Jonathan Oxer: Walktime Blog #16: Arduino Miniconf 2012

Simon Lyall: LCA2012 Tuesday – Before Lunch

Smashing a square peg into a round hole – David Basden and Chris Collins

Extracting metrics from logs for realtime trending and alerting by Jamie Wilkinson

leigh morresi: Drupal 7 and XMLRPC example

Tim Connors: Breaking windows

LCA2009 News: Outstanding talks to take centre stage at linux.conf.au 2012

LCA2009 News: linux.conf.au 2012 set to shine in Ballarat

Colin Charles: Growing out of Foursquare

Colin Charles: New year, going on 8 years

Colin Charles: Piracy due to lack of legal options

Sridhar Dhanapalan: OLPC Australia XO-AU OS 12 beta 1

Jonathan Oxer: Walktime Blog #15: Arduino light switches

leigh morresi: Easy win for lowering PHP memory usage (drupal)

Keith Packard: calypso

Calypso — CalDAV/CardDAV/WebDAV for Android and Evolution

SyncML on the Series 40

SyncML on the N900

How about Android?

WebOS?

Android recovers

CalDAV/CardDAV on Linux

Introducing Calypso, a CardDAV/CalDAV server

Using Calypso

Initial setup

Running calypso

Creating new calendars

Importing files

ToDo list for calypso

More Android info

Jonathan Oxer: Walktime Blog #14: Maker Faire Melbourne, Arduino Miniconf

Sridhar Dhanapalan: OLPC Australia talk at OSDC 2011

Tim Ansell: FastComplete, making bash completion fast on remote file systems

leigh morresi: Some good simple Drupal SEO

Jonathan Oxer: Walktime Blog #13: Cool tools - SSH

Mary Gardiner: An appeal for the Ada Initiative

LCA2009 News: Humanitarian FOSS Miniconf cancelled

Jonathan Oxer: Walktime Blog #12: Cool tools - Git, Dropbox, Google Docs

Mary Gardiner: Interested in women in open tech and culture? AdaCamp Melbourne wants you!

Tim Ansell: A typewriter that types colors!

Tim Ansell: Pretty incredible short film