Just A Summary

Reading 'The Traits Paper'

There appear to be two camps around the way Moose::Roles busily arguing about whether the following code should emit a warning:

package Provider {
    use Moose::Role;

    sub foo { 'foo' };
    1;
}

package Comsumer {
    package Moose;

    with 'Provider';

    sub foo { 'no, bar' }
    1;
}

One camp holds that the code should at least emit a warning and ideally blow up at compile time. The other camp (Moose as implemented), holds that it shouldn’t. The debate gets somewhat heated, people end up appealing to the Traits paper as if it were some kind of holy writ. What’s annoying is that the folk who appeal to that paper appear to have read a different paper from the one I remember reading. So I went and read it again, and here’s what it has to say about overriding methods got from traits:

Trait composition enjoys the flattening property. This property says that the semantics of a class defined using traits is exactly the same as that of a class constructed directly from all of the non-overridden methods of the traits. So, if class A is defined using trait T, and T defines methods a and b, then the semantics of A is the same as it would be if a and b were defined directly in the class A. Naturally, if the glue code of A defines a method b directly, then this b would override the method b obtained from T. Specifically, the flattening property implies that the keyword super has no special semantics for traits; it simply causes the method lookup to be started in the superclass of the class that uses the trait.

Another property of trait composition is that the composition order is irrelevant, and hence conflicting trait methods must be explicitly disambiguated (cf. section 3.5). Conflicts between methods defined in classes and methods defined by incorporated traits are resolved using the following two precedence rules.

• Class methods take precedence over trait methods.
• Trait methods take precedence over superclass methods. This follows from the flattening property, which states that trait methods behave as if they were defined in the class itself.

Which is pretty much as I remember, and strongly implies that Moose is right not to issue a warning.

The paper has more to say on overriding trait implementations in its section on ‘Evaluation against the identified problems’:

Method conflicts may be resolved within traits by explicitly selecting one of the conflicting methods, but more commonly conflicts are resolved in classes by overriding conflicts.

And (relevant to another argument around role composition that’s more or less current):

… sometimes a trait needs to access a conflicting feature, e.g., in order to resolve the conflict. These features are accessed by aliases, rather than by explicitly naming the trait that provides the desired feature. This leads to more robust trait hierarchies, since aliases remain outside the implementations of methods. Contrast this approach with multiple inheritance languages in which one must explicitly name the class that provides a method in order to resolve an ambiguity. The aliasing approach both avoids tangled class references in the source code, and eliminates code that is hard to understand and fragile with respect to change.

There are folk arguing for removing the aliasing support from Moose role composition, but I have to say that I find this argument compelling.

Who’s right

When it comes down to it, referencing the traits paper is just argument from authority, which is one of the classic logical fallacies. However, if you are going to appeal to an authority, try to make sure that you’re not misrepresenting what that authority says. The original traits paper does not suggest that overriding a method got from a trait should come with a warning. On the contrary, it recommends overriding as the right way to resolve conflicts between multiple composed roles.

You may well think that this is problematic. You may be able to show examples where silent overriding has bit you on the arse. You may even have a good argument for introducing warnings. But “Because that’s how the traits paper says you should do it!” is a lousy argument, made doubly lousy by the fact that is precisely not what the traits paper says you should do.

Reading Turing

Some things never disappoint. And reading Alan Turing is one of those things. In an earlier post I told an incorrect anecdote about Turing, and Russ Cox pointed me at proof, in Turing’s own words, that I was wrong. I don’t know why it’s taken me so long, but I finally got around to reading his Lecture to the London Mathematical Society on 20 February 1947.

Wow.

Seriously. Wow. He’s talking about programming the ACE, the ‘pilot’ version of which didn’t run its first program until 1950. And the Manchester ‘Baby’, the first stored program electronic computer, was more than a year away from running its first program. It sounds like it might be dreadfully speclative and either handwavy or as out there and daft as the usual crop of ‘futurologist’ type predictions.

As you can probably guess from the fact that I’m bothering to write this up, it was nothing of the sort. I suggest you nip off and read it for yourself. It won’t take you long and it’s well worth your time. Then come back here and find out if the same things struck you that struck me.

Back in the day

Here’s the sentence that brought me up short like a slap:

Computers always spend just as long writing numbers down and deciding what to do next as they do in actual multiplications, and it is just the same with the ACE.

I got to the end of the sentence before it clicked that back then a computer was a human being performing a computation. What we think of today as ‘a computer’ was what Turing called ‘the ACE’ and back then it certainly deserved that definite article.

Then I read it again and recognised the deep truth of it. Back in Turing’s day, the ACE was planned to have a memory store made up of 5 foot tubes full of mercury acting as an acoustic delay line. Each tube could hold 1K bits and an acoustic pulse took 1 millisecond to get from one end of a tube to the other, so the average access time for a single bit of memory was around 500 microseconds. When it was finally built, it was the fastest computer in the world, running at the mighty speed of 1MHz. Nowadays we think that a cache miss that costs 200 processor cycles is bad news and our compilers and processors are designed to do everything in their power to avoid such disasters. In Turing’s day there were no caches, every time something was fetched from memory it cost 500 cycles. (Well, in 1947 that would be 500 cycles + a year and a half before there was a computer to fetch the memory from in the first place).

Curiously, the gold standard of high performance memory in Turing’s day was the same circuit as you’ll find in high speed SRAM today - the bistable flip flop - but done with valves and hope rather than by etching an arcane pattern on a bit of silicon.

Subroutines and code reuse

Turing seems to have invented the idea of the subroutine. Admittedly it’s implicit in his implementation of a Universal Turing machine in On Computable Numbers…, but it’s explicitly described here. And, rather wonderfully, the pipedream of extensive code reuse is there in the computer science literature right from the start:

The instructions for the job would therefore consist of a considerable number taken off the shelf together with a few made up specially for the job in question.

There are several moments when reading the paper where I found myself thinking “Hang on, he means that literally rather than figuratively doesn’t he?” and this is one of them. When your code is embodied in punched Hollerith cards, a library is just that. Row upon row of shelves carefully indexed with reusable code stacked on them like so many books.

Elsewhere he says:

It will be seen that the possibilities as to what one may do are immense. One of our difficulties will be the maintainence of an appropriate discipline, so that we do not lose track of what we are doing. We shall need a number of efficient librarian types to keep us in order.

That’s my emphasis, and ain’t that the truth? I’m not sure that Turing would have foreseen that the nearest thing we have to ‘a number of efficient librarian types’ would turn out to be Google’s computers though. One wonders whether he’d be horrified or delighted.

Descrimination

Here he is, having painstakingly explained how the use of loops can reduce the size of a program:

It looks however as if we were in danger of getting stuck in this cycle and unable to get out. The solution of this difficulty involves another tactical idea, that of ‘descrimination’. ie. of deciding what to do next partly according to the results of the machine itself instead of according to data available to the programmer.

And there we have the nub of what makes computing so powerful and unpredictable. The behaviour of any program worth writing isn’t necessarily what you expect because it’s making decisions based on things you didn’t already know (if you already knew them, you wouldn’t have to compute them in the first place). This is why I’m optimistic about AI in the long run. I think that given that the behaviour of a single neuron is understandable and simulatable then, eventually we’ll manage to connect up enough virtual neurons and sensors that the emergent behaviour of those simulated neurons is as near to a ‘real’ consciousness as makes no odds. I’m far less convinced that we’re ever going to be able to upload our brains to silicon (or whatever the preferred computing substrate is by then). Whether we’ll able to communicate with such a consciousness is another question entirely, mind.

Job Security Code

The masters are liable to get replaced because as soon as any technique becomes at all stereotyped it become possible to devise a ssystem of instruction tables which will enable the electronic computer to do it for itself. It may happen however that the masters will refuse to do this. They may be unwilling ot let their jobs be stolen from them in this way. In that case they would surround the whole of their work with mystery and make excuses, couched in well chosen gibberish, whenever any dangerous suggestions were made

Oh, did Turing nail it here. 1947 and he’s already foreseen ‘job security’ code. I’ve seen this kind of behaviour all the time and it drives me up the wall. What the peddlars of well chosen gibberish always fail to see that, if you get it right, the computer ends up doing the boring parts of your work for you. And your time is then free to be spent on more interesting areas of the problem domain. Software is never finished, it’s always in a process of becoming. There’s a never ending supply of new problems and a small talent pool of people able to solve them; if you’re worth what you’re paid today then you’ll be worth it again tomorrow, no matter how much you’ve delegated today’s work to the computer. And tomorrow’s work will be more interesting too.

Automating the shitwork is what computers are for. It’s why I hate the thought of being stuck writing code with an editor that I can’t program. Why I love Perl projects like Moose and Moo. Why I’ll spend half a day trawling metacpan.org looking to see if the work has already been done (or mostly done - an 80/20 solution gets me to ‘interesting’ so much quicker).

Job security code makes me so bloody angry. There are precious few of us developers and so much work to be done. And we piss our time away on drudgery when we simply don’t have to. We have at our fingertips the most powerful and flexible tool that humanity has ever built, and we use it like a slide rule. Programming is hard. It demands creativity and discipline. It demands the ability to dig down until we really understand the problem domain and what our users and customers are trying to do and to communicate the tradeoffs that are there to be made - users don’t necessarily understand what’s hard, but they’re even less likely to understand what’s easy. But its very difficulty is what makes it so rewarding. It’s hard to beat the satisfaction of seeing a way to simplify a pile of repetitive code, or a neat way to carve a clean bit of testable behaviour off a ball of mud. Sure, the insight might entail a bunch of niggly code clean up to get things working the new way, but that’s the kind of drudgery I can live with. What I can’t stand is the equivalent of washing the bloody floor. Again. And again. And again. I’d rather be arguing with misogynists - at least there I might have a chance of changing something.

I’m not scared that I’m going to program myself out of a job. I’m more worried that I’m never going to be able to retire because as a society and a profession we’re doing such a monumentally piss poor job of educating the next generation of programmers and some of us seem to be doing a bang up job of being unthinkingly hostile to the 50% of the talent pool who are blessed with two X chromosomes. But that’s probably a rant for another day.

Big Data and Singing Crowds

I watched the rugby yesterday. England vs Wales at Cardiff Arms Pack. It was a great game of rugby - England were comprehensively outthought by a Welsh side with more experience where it counts, but by gum, they went down fighting to the very end. It’s going to be an interesting few years in the run up to the next World Cup.

While the game was going on, I found myself wondering why the crowd’s singing sounded so very good. It’s not a particularly Welsh thing (though Cwm Rhonda, Bread of Heaven and the whole Welsh crowd’s repertoire are have fabulous tunes). The Twickenham crowd getting behind Swing Low, Sweet Chariot sound pretty special too, even if I wish they still sang Jerusalem occasionally. How come a crowd of thousands, singing entirely ad lib with no carefully learned arrangements or conductor can sound so tight?

After all, if you took, say, 30 people and asked ‘em to sing a song they all know, it would sound ropey as hell (unless they were a choir in disguise and had already practiced). Three or four together might sound good because, with that few of you, it’s much easier to listen to your fellow singers and adapt, but 30’s too many for that and without some kind of conductor or leader, things aren’t likely to sound all that great.

I think it’s a statistical thing. Once you get above a certain number of singers, the fact that everyone’s going to sing a bum note now and again, or indeed be completely out of tune and time with everyone else, the song is going to start to make itself heard. Because, though everyone is wrong in a different way, everyone is right the same way. So the wrongs will start to cancel themselves out and be drowned by the ‘organised’ signal that is the song. And all those voices, reinforcing each other make a mighty noise.

That’s how big data works too. Once you have sufficient data (and for some signals sufficient is going to be massive) then the still small voices of whichever fraction of that data is saying the same thing will start to be amplified by where the noise is dissipated.

Just ask an astrophotographer. I have a colleague who takes rather fine photographs of deep space objects that are, to the naked eye nothing more than slightly fuzzy patches of space, only visible on the darkest of nights but which, through the magic of stacked imaging can produce images of stunning depth and clarity.

If you’ve ever taken photographs with a digital camera at the kind of high ISO settings that Mike used to take this, you’ll be used to seeing horrible noisy images. But it turns out that, by leveraging the nature of the noise involved and the wonder of statistics, great photographs like this can be pulled out of noisy data. It works like this:

Any given pixel in a digital photograph is made up of three different componants:

• Real light from the scene in front of the camera
• Systematic error which is the same in every image
• Thermal (usually) noise

The job of an astrophotographer is to work out some way of extracting the signal at the expense of the noise. And to do that, they have one massive advantage compared to the landscape or portrait photographer. The stars and nebulae may be a very very long way away. They may be very dim. But they don’t move. Once you’ve corrected for the motion of the earth, if you point your scope at the horsehead nebula today it’s going to look the same as it did yesterday and the day before that. Obviously, things do change, but, from the distance we’re looking, the change only happens on multi-hundred year timescales. This constancy makes the astrophotgraphers task, if not easy, at least possible.

So… the stars (like the tune of Cwm Rhonda) are unchanging, but the noise is different with every exposure (that’s why it’s called noise after all). Even if, on any given exposure the noise is as strong as the signal, by taking lots and lots of exposures and then averaging them, the noise will get smeared away to black (or very dark grey) and the stars will emerge from the gloom. Sorry. The stars and the systematic error will emerge from the gloom. So, all that remains to do is to take a photograph of the systematic error and take that away from the image.

Huh? How does one take a photograph of systematic error? You do it by photographing a grey sheet. Or, because it’s probably easier, by throwing your telescope completely out of focus so what you see is to all intents and purposes a grey sheet and taking a photograph (or lots of photographs - you’ve still got noise to contend with…) and subtracting the resulting error map from your stack of photographs and bingo, you’re left with an image that’s mostly signal. All that remains is to mess with the levels and curves and possibly to stack in a few false colour images grabbed from the infra red or the hydrogen alpha line where there’s lots of detail and you’re on your way to a cracking photograph.

Obviously, it’s not as easy as that - telescope mounts aren’t perfect, they drift, camera error changes over time. It’s bloody cold outside on a clear night. Sodium street lights play merry hell with the sky. And so on. But if you persevere, you end up with final images like the one above. That sort of thing’s not for me, but I’m very glad there are folk like Mike taking advantage of every clear night to help illuminate the awesome weirdness of our universe.

Noisy data is a pain, but, we’re starting to realise that, if you have enough data and computing power, you can pull some amazing signals out of it. Whether that’s the sound of thousands of Welsh rugby fans combining to sound like the voice of God; an improbably clear photograph of something that happened thousands of years ago a very long way away; your email client getting the spam/ham classification guess right 99 times out of 100; or Google tracking flu epidemics by analysing searches, if you have enough data and the smarts to use it, you can do some amazing things.

Some of them are even worth doing.

Getting Softer

Welcome back. I realise that I left off without telling you how I’d chosen to wire the matrix up. I’m basing my layout on the Jesse’s “Blue Shift” layout:

However, the Maltron has a slightly different layout and I’m less gung ho about getting rid of the extra little finger keys, especially the left hand control and the shifts. The layout I’m starting from looks a little like this:

If you count that up, it’s 60 keys. There are 112 keys in the original Maltron layout arranged in 8 rows of 16 columns (which means that the total number of keys that could be accommodated in the matrix is 16 * 8 or 128. Because I was using only 60 keys, I could fit everything in an 8x8 matrix, which I wired up like this:

Once all the keys were wired up, I tacked ribbon cable in place to pick up signals, crimped terminations on the other end, plugged in the Teensy++ and went searching for firmware software.

Jesse had settled on the Humble Hacker Keyboard Firmware, but I found I couldn’t get on with it, and I ended up with the tmk firmware if only because it’s the first one I managed to get working and I found the documentation a wee bit more comprehensible. However, it was driving me up the wall for while because I simply couldn’t get it to recognise key presses as single keypresses. Keys would bounce, or wouldn’t register and I couldn’t work out what was going on until I read this tip on the Teensy website. It turns out that electronics is more subtle than I realised.

Pull up resistors (at last)

I’m a software guy. So I thought that the effective way of detecting a signal was to look for a positive voltage on your controller input pin. So zero volts implies that the input bit is false (zero in boolean logic). It’s a little bit more complicated than that though. It turns out that you get a clearer signal if you treat a pin being pulled to ground as true. To do this, we need some way of arranging for our input pin to be at 5V when the switch is open and, which (if you don’t know the trick) is more tricky, at 0V when the switch is closed and current is not flowing. Enter the pullup resistor. Consider the following schematic:

All we need to know to understand what’s going on now is Ohm’s Law. Ohm’s Law is almost laughably simple but once you’ve grasped it, understanding electronics gets much easier. The law states that the voltage (V) dropped across a load is equal to the product of the current flowing (I) in Amps and the resistance in Ohms (R).

So, when the switch is open (as in the diagram), we can see that the voltage between P0 and ground is equal to 5V - IR, but no current is flowing which makes IR equal to zero and so P0 is at 5V. So… what happens when the switch is closed?

We know that the voltage between the power rail and ground is 5V and we choose R so that the resistance of the switch might as well be zero. Which means that the voltage at P0 is 0V, or as near as makes no odds, so we have our two logic levels. When the switch is open, the input pin is at 5V, which we call false, and when it’s closed the pin is pulled down to ground (0V), which we call true.

So, if we recast our matrix driver so that, rather than applying a voltage to each column in turn and check the row pins to see if they’re high, we set up pull up resistors on the column pins and, set all our rows to 5V. Then, to scan the matrix, we set a row to ground and check which columns go to ground too and on to the next. The beauty of the Teensy is that we can do that without any extra hardware, we just set a couple of registers to appropriate values and we’re golden. Once I’d done this and rebuilt my debugging firmware suddenly the debugging output was making more sense. No missed keys. No strange repeats. No keys I hadn’t touched suddenly deciding they’d been pressed. Lovely.

There’s another possible problem with keyswitches called ‘bouncing’ that the firmware takes care of for me out of the box. In theory keyswitches are dead simple. You press the button and circuit goes from not conducting to conducting with no shilly shallying around. In practice… watching the voltage across even the best switch with a suitable oscilloscope is a lesson in the damnable imperfection of mechanical bits and pieces. The voltage is high. Then low. Then high. Then low. Then low. Then high. Then low and staying there. If you don’t take this into account in your driver you’re going to be registering far too many keypresses. Which is why any firmware worthy of the name has software debouncing (there are hardware debouncing solutions, but it’s much, much cheaper and more convenient to do the compensation) and the tmk firmware is no different. I’m sufficiently lazy that I’ve not really looked at how it works in any detail. Basically, if it detects a switch change it reads the same pin multiple times and, assuming the switch state is still changed at the end of that process, then it’s a real keyup or keydown event.

Faking it ‘til you make it

The tmk firmware is substantially more competent than I’ve explored in any depth yet. I’m experimenting with what I want to do with the blue shift layer and distinguishing between taps, chording and other possibilities by setting up my ‘blue shift’ keys to send the ‘F12’ and F13’ keycodes and I’m using KeyRemap4Macbook to do most of my messing with stuff, but once I’ve worked out what I want, I expect to push as much as possible into the firmware so I don’t have to duplicate a bunch of work (and indeed find appropriate driver software) when I want to use the keyboard on a Linux or, in extremis, Windows box.

Traditions are there to be overthrown too

The keyboard on your computer is (unless you’re a weirdo like me and you’ve got a kinesis, maltron or some other alternative input device) is a living fossil. It takes the form it does because back when typewriters were invented, the mechanical constraints of needing to have typebars strike paper forced the designers to stagger the rows of keys. The keyboard layout was (allegedly) designed not to slow typists down, but to try and avoid getting keys tangled up with each other during typing by keeping common key combinations apart (I’m not entirely convinced that his is true, given that ‘e’ is next to ‘r’ and ‘t’ and ‘h’ are such near neighbours, but it’s pretty obvious that the qwerty layout isn’t really designed to minimise finger travel while touch typing (one wonders if they’d even thought of touch typing when they designed the thing). There’s no real reason to remain tied to this design. The Maltron case is designed so that there’s not much lateral movement of your fingers or wrist flexion while typing. Once you’ve learned the layout, it’s a delight to type with. But even with the radical case design and rejigged layout the Maltron is a surprisingly conservative design. The microcontroller I’m using to drive the keyboard is a pretty capable 8bit computer running at 16MHz, 8K RAM, 4K of EEPROM and 128K of flash memory to hold the program. Scanning an 8x8 matrix doesn’t come close to pushing it.

So, if we’re not tied to ‘one key one action’, what can we do?

Here’s what I’ve been experimenting with so far:

Distinguishing between tapping and press and then release. And between typing a key by itself and using it as a modifier. So at the moment I have:

If I tap (press and release quickly without pressing another key) the left blue shift, then pretend I actually tapped the tab key. If I press the key and, while holding it down, hit another key, send the ‘blue shift’ symbol associated with that key or just send L_ALT + the original keycode if there’s no blue shift symbol. The right blue shift works similarly but instead of sending tab on tap, we send RET. If I press either key hold it for a while and then release it, we don’t send anything.

The two keys on the far left (shift and control) send ESC when tapped.

I’ve also arranged things so that both ALT keys send RALT. I realise that might seem weird, but I’ve also configured my Emacs to treat RALT as a SUPER key which lets me bind actions to blue shifted keys. So when I’m in Emacs, all those keys without a blue symbol on them have more or less complicated actions associated with them. Others have used teensy based firmwares to have certain key combinations move the mouse pointer or trigger complex sequences of actions.

I’ve also got enough pins spare on the teensy that (and enough holes in the case) that I’m seriously considering using hot glue to mount a few RGB LEDs behind some of the holes in the middle of the case so that, If I end up cooking up more keyboard layers, I can indicate the keyboard (and Emacs perhaps?) state with blinkenlights. Because how can a project be complete if there aren’t blinkenlights?

Next steps

Where next? I’m not sure. I’m still experimenting with the possibilities that open up once you realise that just because we’ve always simulated a mechanical typewriter there’s no reason to keep doing it. Hardware doesn’t have to be dumb.

And then there’s the fact that a sixty key layout in a case designed to hold over a hundred keys looks scruffy. Until I started hacking my keyboard I’d tended to think that a desktop 3d printer was, for me at least, a solution looking for a problem. But now I’m trying to work out how to build a better keyboard case… Well, I think I’ve found my problem.

Fun with solder

Where were we? Ah yes, I’d just unwired my Maltron, pulled out all the switches, ordered some Cherry MX brown stem keyswitches from a Deskthority Group buy and a Teensy++ from Pieter Floris. Now all I had to do was work out how I was going to wire the thing up. Jesse’s article had some great pointers, but as I disassembled the Maltron wiring loom, I gained a great deal of respect for their decision to use fine enamelled wire (which a bit of googling revealed to be solderable copper magnet winding wire - I bought some 30SWG stuff from wires.co.uk) which, because it’s thin and solid core is easy to bend into shape and, because the enamel coating melts into solder flux, is easy to solder without worrying about stripping insulation.

One thing that worried me about both Jesse and Maltron’s wiring was the fiddly nature of the way the wired the diodes in. The Maltron wiring only had diodes on a few keys, but I was looking to experiment with some serious remapping and possibly chording layouts - hardwiring a limited set of modifier keys wasn’t in my plan.

Diodes? Why diodes?

Before I talk about how I solved that problem, I’d best explain what the problem is. Consider the average computer keyboard with 105 or so keys. How do you work out which keys have been pressed without needing 105 I/O pins on your microcontroller? You arrange things in a matrix. We’ll worry about the physical layout of the board later, but here’s a schematic of a 5x5 key matrix which, with a bit of cunning, allows us to read which one of 25 keys is pressed with only 10 microcontroller pins

Suppose we apply a signal to the first column then, by looking at the pins attached to the rows, we can tell which switches in the selected column have been pressed by checking for the signal.

By cycling the symbol from column to column rapidly, we can scan the whole matrix.

When two keys are pressed at the same time, we can spot them with this arrangement, but what happens when, say, three keys are pressed? Let’s press switches S1, S11 and S13 and find out. When we scan column one, all is well, we get the signal out on rows 1 and 3 as we expected, but when we scan column 3, which only has one key held down, we get a signal out on rows 1 and 3 as well. What’s going on?

Let’s trace the signal and see if we can work out how it gets to row one. The signal comes in on column 3, through S13 and onto row 3. But S11 is also on row 3, which means that the signal can flow through S11 onto column 1 and once it’s on column 1, it flows through S1, onto row 1 and confusion reigns.

Welcome to the wonderful world of ghosting.

If you’re happy to live with only detecting when two keys are pressed simultaneously, you can correct for this in software by ignoring ambigous symbols (or by ignoring all signals which show more than two keys if you’re feeling lazy) and you can correct things for WASD gamers with some careful matrix layout to make sure that most of the common 3+ key combos aren’t ambiguous. Or you can spend a quid on a hundred 1N4148 diodes and invest some extra soldering time to wire things up like so:

<< Signal: C3. Closed: C1R1, C1R3, C3R1. All Diodes >>

With the diodes in place, the signal can’t go back through S1 and confuse things and your controller software can be much simpler.

The original Maltron wiring doesn’t put diodes on every key, just on all the modifier keys. Diodes may be cheap, but getting them wired into the matrix in the way Maltron did is a complete pain, doing it for over 100 keys was never going to be fun. But dammit, diodes on every key is just the Right Thing. There had to be a better way.

Diodes without (much) pain

If you’ve ever spent time soldering stuff, you’ll be aware that humans have a hard time soldering due to an acute shortage of hands. Generally you need to hold one component in one hand, another component in the other, the soldering iron in your other other hand and the solder wire in the… hmm… we appear to have run out of hands. Which is where things like vises come in handy. That way you can hold one component in the vise, lock the wire or component you’re trying to attach to it by twisting, wrapping or beinding things, leaving you with hands free for the iron and solder. Wiring diodes up to the keyswitches, which have to be installed in the keyboard case while you’re doing it, is the very definition of fiddly.

However, if you go and read things like the MX keyswitch’s datasheet, it makes reference to switches with diodes fitted and when you look at the bottom of a switch you’ll see a diode symbol and four small holes.

Time to crack a switch open

There’s quite a bit going on in there, but right where the four holes are is the interesting bit. We can bend the legs of a 1N4148 diode and feed ‘em through the holes, pop the lid back on and it fits, clean as a whistle. We’re onto something here.

I’d decided to go with a reduced layout based on the ‘blue shift’ layout that Jesse cooked up so, although it was fiddly it didn’t take that long to pop another 59 keys and put diodes in there. After that, it was easy enough to wrap the anode lead around one of the switch’s pins, solder it in place and clip off the excess wire:

Now I had 60 keyswitches with diodes installed which I could pop into the keyboard case and wire up with magnet wire. I love solderable magnet wire. It’s magic stuff. Just wrap it tight round a pin, or the cathode lead of the diode and it stays in place while you solder it in place. I’m not going to pretend it was the work of moments, but it was pretty straightforward and I haven’t needed to resolder a single joint. There’s something very satisfying about watching capillary action suck molten solder into the joint. Physics is awesome.

Tune in next time to really learn about the importance of the pull-up resistor, how to roll your own (or someone else’s) keyboard firmware and some thoughts on where next.

In which Piers prepares to void the warranty...

Some years ago (I have the awful feeling it was 1999) I was stricken with a bout of tingly numbness in my right hand. When you’re a computer programmer, the thought of being unable to type, and thus unable to program isn’t something you ever want to deal with. Terry Pratchett’s words about gnawing the arse out of a dead badger if it would make it better spring to mind. So, I replaced my mouse with a trackball, got a better chair and invested three hundred and some pounds of my own money in a Maltron keyboard.

The Maltron design makes the Microsoft Natural split keyboard look like an achingly conservative piece of peripheral design (though the original MS natural keyboard had a very nice feature that the Maltron lacks in its reverse tilt). It looks like something from a science fiction film (and maltron keyboards have been used as props in sf movies) and once I’d got used to it, it was marvellous. Comfortable. Fast to type on. Weirdly laid out.

But… slightly flimsy in feel. It’s understandable really, the keyboards sell in such low volumes that each one has the feel of a ‘production prototype’. Which is great when you’re a programmer like me and want the layout tweaked slightly from the basic spec - Maltron were more than happy to wire things up so that, where their ‘standard’ model had a caps lock key, there was a control key, or to make a normally ‘dead’ key into an extra quote/double quote. After all, when you’re forming a sheet of plastic over a plaster of paris mold, cutting holes in it with a punch and then wiring every keyswitch into the matrix by hand, there’s not much more work in rerouting a couple of wires here and there, so at the cost of a slightly longer wait I got a customized keyboard.

Time went by, the tingles went away, I started coding on laptops, and Macbooks at that. Laptops that had neither AT or PS/2 keyboard interfaces, only USB. And all those keyboards that let you plug your PS/2 plug into a simple USB adaptor do it by having USB hardware onboard, they detect when they’ve been plugged into a USB socket and switch protocols. Not so much with keyboards made in the 20th century, so the Maltron got put away in its box.

When I was working for the BBC, the tingles came back, so I pulled the Maltron out of storage, hooked it up and discovered that the time in storage hadn’t been kind to it. But the BBC is the kind of employer that really doesn’t like the thought of an expensive programmer not being able to do his job and arranged for me to have a Kinesis Advantage.

The Kinesis is a problematic keyboard for us Maltron users. It’s what the Maltron could have been, given the kind of investment and volume sales that small UK companies don’t get. You only have to look at one to realise that its heavily inspired by the Maltron. Putting partisan issues to one side, the Kinesis is a more solid keyboard than the Maltron. When you open one up it’s obvious why the Kinesis manages to feel solid and cost less. There’s a flexible PCB in there, fewer ‘real’ keys and a heavier, more rigid, injection molded, case. The initial setup and tooling costs must’ve been phenomenal by comparison to the Maltron way of building, but once those are accounted for, the cost per unit is pulled way down. And Kinesis had a second mover advantage, they’re not building a keyboard from scratch, they’re developing an already existing design. And developing it very well. If you look at the ‘bowl’ of a Maltron keyboard, each column has its own radius, the angle of each key is tuned in the case, so the case shape is complex and presumably hard to make with injection molding. The Kinesis case shape is rather simpler; the ‘middle’ finger column has greater radius than the others, but apart from that the bowl is basically cylindrical. However, when you use the advantage each key feels as right as it does on a Maltron. Kinesis do this by altering the profile of the keycaps. If you pull the all the keycaps in a bowl off their stems and put ‘em on a flat surface, you’ll see that no two have the same shape. Developing that profile can’t have been a simple process, but again, it’s much easier to cast one hundred or so keycaps in different shapes than a whole case in a more complex shape. After all, if a keycap goes wrong when its cast, you can just dump that keycap and grab another one off the pile. If a case goes wrong, you’ve got rather more plastic to melt down.

So… once I’d remapped the Kinesis to use the Maltron layout (It may have been years since I’d used the Maltron, but fingers don’t forget - when I’m typing on a flat keyboard, my fingers sit over ASDFJKL; but whey I’m typing on a contoured keyboard, they’re mapped to ANISTHOR and I find typing in QWERTY mode nearly impossible; I’m reduced to hunt and peck.) I was very happy with it, and with the help it gave to my posture, I was soon typing comfortably again.

And then I left the Beeb and came to Cornwall. And, bless them, the Beeb held on to ‘my’ keyboard (I hope whoever it ended up with worked out how to remap it back to QWERTY). But, I wasn’t hurting, so a flat keyboard was fine.

Until it wasn’t and the tingles. And good as Headforwards are as employers, they don’t really have the kind of budget that the BBC has, so I’ve ended up buying my own Kinesis for work - if nothing else, I’m not going to have the problem of leaving my keyboard behind if I have to change jobs.

But I’m a geek. I don’t just code at work, I code and write at home. It’s all very well having a lovely ergonomic keyboard at work and a painful flat one at home. So I dusted off the Maltron again, picked up an active USB protocol converter and discovered that time had definitely not been kind. Keys were stuck. Or not registering. The Shift Lock key wasn’t like every other key, when you pressed it, the controller held odwn a virtual shift key until you pressed it again. And, because this was hard wired in a control software blown into an EPROM there was no way I could remap it.

I was about to give up on getting a contoured keyboard up and working at home and resign myself to carrying the Kinesis back and forth from the office, but then my friend Nat Torkington’s Four Short Links post pointed me at my friend Jesse’s fantastic pair of blog posts about building a flat, Maltron/Kinesis inspired keyboard from scratch and I was off to the on-line shops in search of Teensy++ microcontrollers, diodes, soldering irons and new Cherry MX brown keyswitches; reading obssessive webfora like GeekHack and Deskthority; and unscrewing the baseplate of my painfully expensive hand built custom keyboard and enthusiastically voiding the warranty with the aid of a pair of diagonal cutters.

Tune in next time to learn how a keyboard works, why magnet wire is lovely and the importance of the pull-up resistor.

Belated OSCON writeup

I had such fun. Though I’m never, ever, livecoding half an unwritten talk in an Emacs window again.

You want proof?

Also. I’m not dead, I’m just writing a book on Higher Order Coffeescript for O’Reilly and I alternate between bouts of horrid mental block and massive splurges of disorganized content where everything seems to be more important than everything else.

Turing

Today is Alan Turing’s 100th birthday. I’ve been thinking about him lately, in particular about a story that demonstrates the perils of working with genius.

The story goes that, when Turing was working with the Manchester Baby (the first stored program computer ever built. Just) a colleague wrote the first ever assembler which would turn (relatively) human readable assembly language and turn it into the ones and zeroes of machine code that the machine could actually execute. He showed it to Turing, who blew up at him, ranting that using the computer to do ‘trivial’ jobs like that were a massive waste of expensive computer time.

The problem with working with a genius, from the point of view of more ordinary mortals, is that the genius has only a very rough idea of what is actually easy, and what is only easy to them. From today’s vantage point, when computers are as freely available as they are now, the idea of not letting the computer do the shitwork for you seems utterly ludicrous – programmer time is more valuable than computer time.

What’s less obvious is that the same was true in Turing’s time (when there was precisely one computer) too. It only takes one programmer to make a mistake in translating from assembler to machine code and run a job that, for instance, gets stuck in an infinite loop and you’ve probably wasted more computer time (and programmer time) than if you’d just run it through the assembler in the first place. Turing didn’t see that, because the process of translating from symbols to binary wasn’t something that he found particularly complicated. To him, what was needed wasn’t more and better tools, it was more and better Turings.

I’m not entirely sure that I believe the story (and I can’t remember where I heard it, so it may be a phantom of my memory). It certainly doesn’t chime with the Turing who was instrumental in mechanising the shitwork of finding the days rotor settings so that Enigma traffic could be cracked. The history of Bletchley Park is a story of building better and better machines to do the dully repetitive jobs that humans find so hard to stick to and which machines excel at.

The “Just write code without bugs in the first place” school of programming is alive and well today. It’s not my school though. I’m very much an exploratory coder. I like to have tests to show me the way and to keep me honest so that I don’t go breaking things when I change this bit here. I’ve customised my editing environment to help me as much as possible. I have a nasty habit of writing bare words that I should have quoted, so I have a keystroke that shoves the last word into quotes for me. Another keystroke will align all the broad arrows at the current level of nesting. I’ve got snippets set up that fill in standard boilerplate when I start a new class, a huge list of common spelling mistakes that get autocorrected while I’m not looking and an expanding list of other little helpers that I need to write when I get around to it. Automation lets me go faster, make more and better mistakes and recover from them faster. I am a of little brain and I want all the help I can give myself.

I’m not sure that Turing would be in the same camp as me.

Here’s the thing though – Turing was, definitely, a genius. But his paper “On Computable Numbers, with an Application to the Entscheidungsproblem” (the one that gave the world the Turing Machine) has a bug in it. In fact, it has two. The first is obvious enough that I spotted it when i read the paper for the first time. The second bug is rather more subtle (but still fixable. It’s okay, the field of computing is not build on sand).

I love that The Paper – The one that’s the theoretical basis for the modern world (first publishd 75 years ago, round(ish) number fans) – has a bug. It gives me a strange kind of hope. Fallibility gets us all – even people who have saved the world. I think we should celebrate the humanity as well as the genius. Turing was, by all accounts, a very odd fish, but the world is undeniably richer for his contributions.

So, let’s raise a glass to the memory of Alan Turing tonight, marathon runner, gay icon, saviour of the world and the pleasantly fallible inventor of the modern world. Not a bad CV, when you think about it.

Updated

Apparently I’m wrong about it being Turing who didn’t like the assembler but Von Neumann (another genius). See below for details. And phew! How nice to find that a hero doesn’t have feet of clay.

An instructive joke for all occasions

Two bulls were grazing at the bottom of the big pasture, when the farmer let a load of heifers in at the top gate.

“Hey,” said the young bull to the old, “What do you say we run up there and fuck us a couple of heifers?”

“Well,” said the old one, “You’re welcome to do that if you want to, but I plan on walking up there and fucking all of them”.

I’ll leave any interpretation up to you.

Pretty Saro

Pretty Saro by pdcawley

I’m a wee bit late writing this because it’s mostly about my summer trip to the Swannanoa Gathering at Warren Wilson College near Asheville, North Carolina. It just so happened that YAPC (Yet Another Perl Conference) was held in Asheville this year, the week before the gathering’s Traditional Song Week. Well, I’m a perl hacker. I sing traditional songs. My employer was paying for me to attend YAPC and were willing for me to extend my stay in America by a week. It was a no brainer really.

YAPC was bloody good this year. Perl 5 development is moving forward and the community is buzzing because of it. Lots of “… and we’re hiring” slides. And that’s before we get to the pleasure of catching up with friends that I only see online most of the time. If you’re working in any field that has grassroots conferences associated with it, I can’t recommend attending them highly enough.

On to Swannanoa

At Swannanoa, I met Sheila Kay Adams and immediately switched my schedule to spend as much time singing with her as possible. Sheila’s a seventh generation ballad singer from the Sodom Laurel community in Madison Count. Her “grannie Berzil” Wallin remembered Cecil Sharp coming to Madison County and collecting songs from the family.

Pedigree in singing shouldn’t matter, but it turns out it does. Sheila grew up in a community which was changing, but which still had old ‘love songs’ as an important part of how it understood the world. Today, not so much. People still relate to the world and understand it through songs, but the songs are more likely to be contemporary. Lyrically, many of the love songs that Sheila and her family sing could have been written yesterday, but their performance is radically different from contemporary style. One voice, unaccompanied, a style that requires the listener to concentrate on the song rather than any aspects of production. Not something you’re going to dance to at your wedding, say.

Sheila’s classes, on Meeting House Songs (more later) and her Ballads class with Bobbie McMillon were just wonderful; I won’t forget in a hurry the sound Bobbie singing “A conversation with Death” as a thunderstorm grumbled across the campus in the background. Spine tingling stuff. Sheila’s description of how she learned songs “knee to knee” has been helpful too. The way it would work was that the teacher and student would be sat out on the porch, often doing some chore or another, and the teacher would sing the first verse of a song. The student would sing it back and the teacher would sing the second verse. The student would then sing the first two verses then the teacher would sing the third verse and the student would sing the first three verses and so on, until the student was singing the whole song back to the teacher. A time consuming process to be sure, but it works.

I know this because I learned Pretty Saro from a recording of Sheila’s late husband Jim Taylor, using a variant of the method, “knee to CD” if you like. I’d play the first verse, hit pause and sing it back, play the second verse, pause, repeat the first two… and so on. And in very short order I had the words and tune (up to a point; I listened back to that recording again recently and I’m singing a different tune now) and could start the process of actually learning the song, which involved singing it it lots, listening to other recordings, singing it some more and then taking the song out and trying it out in front of audiences and listening to what works and what doesn’t. I’ve never really finished learning a song; this recording is a snapshot. I hope you enjoy it.