Successlessness

Saucony Hattori review

2011-12-22T20:57:00.005+11:00

I've just bought a pair of Saucony Hattoris and they really good!

They're minimal, but have a lot more cushioning than Vibrams or the Evos since the EVA+ sole is quite soft and about 10mm thick. They're zero-drop and ultra light weight (250g/pair), about half the weight of the Evos and lighter than Vibram Sprints.

They're a really comfy fit, with no laces they are held on by a tight, thin "lycra" upper. Mine are maybe a little on the small side, so the velcro straps don't do anything useful.

The tight fitting upper has another advantage that no other shoes I've got come close too: It's virtually impossible to get sand in them, which is great as that's extra annoying when running sock-less.

Time will tell how the sole wears, but the hard reinforcements are in all the places I usually wear through, so I can't see why they won't last a fair while.

Please unsubscribe me from the World Vision newsletter

2011-08-12T12:09:00.002+10:00

God knows I've tried enough times, maybe posting it to a blog is how you get off the list?

Terra Plana Evo review

2011-08-07T17:18:00.006+10:00

My Vibrams are worn out (or through) so after having a good experience with the Merrell Trail Gloves I decided to get a pair of Terra Plana Evos.

It's a pain to get Vibram FiveFingers in Australia at the moment. They're hard to find in shops, can't be shipped here from the USA and they're about twice the price even though our dollar is currently trading higher than the USD.

The Evos are more expensive than the Vibrams but a little less "freaky" and provide a similar minimal experience to the Merrells. That is, your toes are free, there's no support or cushioning and you get a fair degree of "ground feel".

I considered a pair of Merrell True Gloves (the "road" equivalent of the Trail Gloves) but they had too much cushioning for my liking and unlike the Evos, they didn't have removable inner soles.

I bought them in the black/red style, which was probably not that smart. They contrast too much against my skin and tend to look a bit odd, they're a very small shoe and black emphasises that.

One thing to note, is that the women's model is identical to the men's, but quite a bit cheaper! Unfortunately they only go up to size 41, but it's something to take advantage of if you have small feet. Speaking of which, the Evos are sized pretty small, the 42 is plenty big enough for me, which is a size or two smaller than my regular shoes.

The lacing is the same as for the Merrell Trail Gloves, holding snuggly to the middle of the foot and leaving your toes free to splay. They don't lock onto my heel as nicely however and they tend to feel a little loose after running a few kms.

They are a minimal shoe and still a long way from barefoot, with less ground feel than Vibrams, even wearing them without socks and taking out the inner soles. Still, you feel pebbles and the contrast between concrete, asphalt, gravel, dirt, grass surfaces well enough and it doesn't seem to interfere with my stride. They don't have a lot of grip however and slip pretty easily if it gets muddy. While they'd probably do fine on dry trails, I don't think they feel very suitable for general trail running.

Not wearing socks did cause some significant rubbing issues however. That was partly due to me jumping straight into 12km runs with them, and I have to resort to prepping with vaseline. It's definitely something to be aware of as they press on different parts of your foot than the Five Fingers do.

I've done about 85km in them now and they feel good. They don't feel like they'll wear out too quickly, aren't too cold (though our winter has been fairly mild so far), and don't soak up water between the toes like the Five Fingers. They feel like they'll be fine in warmer weather too, the upper is a very thin and breathable mesh.

Overall, other than the price, they're great and I'd recommend them to any experienced minimalist shoe runners.

No, wait, that's not right, here's how I really solved it.

2011-06-22T20:14:00.004+10:00

Yesterday I posted the solution to a puzzle in response to one created by Reilly. The solution is fine, but I explained it badly. The explanation just describes the finished code, which is all well and good, but it doesn't elucidate the steps I took to get there and isn't likely to help anyone as a result.

The following steps are much closer to what I actually did:

1. While reading Reilly's Prolog version I realized that SQL could probably solve the puzzle by generating a list of all the possible combinations of pieces (either by a join or in a temporary table) and then filtering those in some way to remove invalid solutions.

2. I left it at that for a while, but a few thoughts popped into my head during the day on the structure of a "pieces" table and an INSERT statement that could generate the 4 possible rotations of each piece. I also ran the numbers in my head and realized there were 4^9 = 2^18 = 262,144 combinations which is not many at all really.

3. I initially ran into a bit of a dead end imagining how a SELECT statement would generate all the combinations of pieces, but decided to just start writing it out and see where that led.

This was the crucial step: Starting with the desired result and working backwards.

So, I started with what I wanted: A list of piece numbers and rotations in each position of the puzzle:

SELECT A.piece_id AS A, A.rotation AS Ar, B.piece_id AS B, B.rotation AS Br, C.piece_id AS C, C.rotation AS Cr,
       D.piece_id AS D, D.rotation AS Dr, E.piece_id AS E, E.rotation AS Er, F.piece_id AS F, F.rotation AS Fr,
       G.piece_id AS G, G.rotation AS Gr, H.piece_id AS H, H.rotation AS Hr, I.piece_id AS I, I.rotation AS Ir

Then I started joining tables together to match the constraints of the puzzle (i.e. the animals on the adjacent edges match head-to-tail.) At this point I didn't know what fields were going to be in the table, or what it would be called, so I just made them up so the SELECT statement read nicely. First, I'll show the overall structure and come back to the "matches" bits:

FROM rotated_pieces A 
INNER JOIN rotated_pieces B ON -- A.right matches B.left --
INNER JOIN rotated_pieces C ON -- B.right matches C.left --
INNER JOIN rotated_pieces D ON -- A.bottom matches D.top --
INNER JOIN rotated_pieces E ON -- D.right matches E.left AND B.bottom matches E.top --
INNER JOIN rotated_pieces F ON -- E.right matches F.left AND C.bottom matches F.top --
INNER JOIN rotated_pieces G ON -- D.bottom matches G.top --
INNER JOIN rotated_pieces H ON -- G.right matches H.left AND E.bottom matches H.top --
INNER JOIN rotated_pieces I ON -- H.right matches I.left AND F.bottom matches I.top --

My "matches" criteria was initially something like:

(   (A.right = B.left AND A.right_head = 'head' AND B.left_head = 'tail')
 OR (A.right = B.left AND A.right_head = 'tail' AND B.left_head = 'head'))

This makes sure the animals are the same and it's either head-tail or tail-head.

That seemed like it might work which was a pretty good sign I was on the right track. I wasn't sure how many combinations this would match, so decided to leave a WHERE clause for later. I was hoping this would be a unique solution, or there may be just a couple and I could pick the proper solution out by hand. Either way, that was a problem for later.

4. From this sketch of a SELECT statement I was able to come up with an appropriate table structure for rotated_pieces:

CREATE TABLE rotated_pieces (
  piece_id     INTEGER, # unique piece id
  rotation     INTEGER, # 0 - top, 1 - left, 2 - bottom, 3 - right
  top         CHAR(10), # animal at top edge
  top_head    CHAR(10), # head or tail at top edge
  `right`     CHAR(10),
  right_head  CHAR(10),
  bottom      CHAR(10),
  bottom_head CHAR(10),
  `left`      CHAR(10),
  left_head   CHAR(10));

And then I started on an INSERT statement to populate the table with the piece layouts:

INSERT INTO rotated_pieces 
       (piece_id, rotation, 
              top,       top_head, `right`,   right_head, bottom,    bottom_head, `left`,    left_head) 
VALUES (1, 0, 'cheetah', 'tail',   'tiger',   'tail',     'lion',    'head',      'tiger',   'head'),
       (2, 0, 'lion',    'tail',   'lion',    'head',     'tiger',   'tail',      'cheetah', 'head'),
       (3, 0, 'tiger',   'tail',   'lion',    'head',     'panther', 'head',      'tiger',   'tail'),
       (4, 0, 'panther', 'tail',   'cheetah', 'head',     'panther', 'tail',      'lion',    'tail'),
       (5, 0, 'tiger',   'head',   'tiger',   'head',     'cheetah', 'head',      'lion',    'tail'),
       (6, 0, 'panther', 'tail',   'panther', 'head',     'cheetah', 'tail',      'tiger',   'head'),
       (7, 0, 'panther', 'head',   'cheetah', 'tail',     'cheetah', 'head',      'lion',    'tail'),
       (8, 0, 'panther', 'tail',   'lion',    'head',     'panther', 'head',      'cheetah', 'tail'),
       (9, 0, 'cheetah', 'head',   'tiger',   'head',     'panther', 'head',      'lion',    'tail');

While I was doing this I copy/pasted Reilly's equivalent bit of Prolog code and it was a simple text transformation between the two languages, another very good sign:

Prolog:
tile(1,     cheetah_tail,     tiger_tail,     lion_head,     tiger_head).
SQL:
    (1, 0, 'cheetah','tail', 'tiger','tail', 'lion','head', 'tiger','head'),

5. At this point, I realized that the "matches" criteria could be simplified a lot:

A.right = B.left AND A.right_head != B.left_head

6. Now I was able to write out the "rotation" function I'd envisioned earlier in the day and realized I wasn't going to need a "pieces" table at all, this program was only going to need the 36 rotated_pieces records, another good sign that I was on the right track.

# rotate (top --> left)
INSERT INTO rotated_pieces (piece_id, rotation, top, top_head, `right`, right_head, bottom, bottom_head, `left`, left_head) 
  SELECT piece_id, rotation+1, 
         `right` AS top, right_head AS top_head, 
         bottom AS `right`, bottom_head AS right_head, 
         `left` AS bottom, left_head AS bottom_head, 
         top AS `left`, top_head AS left_head 
    FROM rotated_pieces;

# rotate (top --> bottom, left --> right) 
INSERT INTO rotated_pieces (piece_id, rotation, top, top_head, `right`, right_head, bottom, bottom_head, `left`, left_head) 
  SELECT piece_id, rotation+2, 
         bottom AS top, bottom_head AS top_head, 
         `left` AS `right`, left_head AS right_head, 
         top AS bottom, top_head AS bottom_head, 
         `right` AS `left`, right_head AS left_head 
    FROM rotated_pieces;

7. Finally, I had all the code sketched out. I checked the syntax for a few things since I'd just been doing it off the top of my head and then executed it. After escaping `right` and `left` as they're keywords (not necessarily the best choice of field names...) the CREATE TABLE and INSERT statements ran. I did a sanity check of the rotated_pieces table and it looked good.

The result of my SELECT statement was disappointing though. As I sort of suspected, there are lots of solutions if you allow a piece to be used more than once and the SELECT returned 1272 rows.

8. I couldn't think of a neat way to check for duplicates so ended up just writing a WHERE clause to do it by brute force:

WHERE A.piece_id != B.piece_id AND A.piece_id != C.piece_id AND A.piece_id != D.piece_id
  AND A.piece_id != E.piece_id AND A.piece_id != F.piece_id AND A.piece_id != G.piece_id
  AND A.piece_id != H.piece_id AND A.piece_id != I.piece_id
  AND B.piece_id != C.piece_id AND B.piece_id != D.piece_id AND B.piece_id != E.piece_id
  AND B.piece_id != F.piece_id AND B.piece_id != G.piece_id AND B.piece_id != H.piece_id
  AND B.piece_id != I.piece_id AND C.piece_id != D.piece_id AND C.piece_id != E.piece_id
  AND C.piece_id != F.piece_id AND C.piece_id != G.piece_id AND C.piece_id != H.piece_id
  AND C.piece_id != I.piece_id
  AND D.piece_id != E.piece_id AND D.piece_id != F.piece_id AND D.piece_id != G.piece_id
  AND D.piece_id != H.piece_id AND D.piece_id != I.piece_id
  AND E.piece_id != F.piece_id AND E.piece_id != G.piece_id AND E.piece_id != H.piece_id
  AND E.piece_id != I.piece_id
  AND F.piece_id != G.piece_id AND F.piece_id != H.piece_id AND F.piece_id != I.piece_id
  AND G.piece_id != H.piece_id AND G.piece_id != I.piece_id
  AND H.piece_id != I.piece_id;

9. This had the desired result and I got back 4 rows.

'A', 'Ar', 'B', 'Br', 'C', 'Cr', 'D', 'Dr', 'E', 'Er', 'F', 'Fr', 'G', 'Gr', 'H', 'Hr', 'I', 'Ir'
2, 1, 1, 0, 6, 0, 8, 3, 9, 3, 7, 2, 5, 3, 3, 0, 4, 0
6, 1, 7, 3, 4, 1, 1, 1, 9, 0, 3, 1, 2, 2, 8, 0, 5, 0
4, 2, 3, 2, 5, 1, 7, 0, 9, 1, 8, 1, 6, 2, 1, 2, 2, 3
5, 2, 8, 2, 2, 0, 3, 3, 9, 2, 1, 3, 4, 3, 7, 1, 6, 3

The first row clearly matched Reilly's solution, but it wasn't immediately obvious what the other 3 rows were. After reformatting them into a 3x3 grid and staring for a little while I picked out the pattern of pieces 2-1-6 around the edges and realized it was the whole board being rotated. So there was a single solution after all.

2, 1, 6, 
8, 9, 7, 
5, 3, 4

6, 7, 4, 
1, 9, 3, 
2, 8, 5

4, 3, 5, 
7, 9, 8, 
6, 1, 2

5, 8, 2, 
3, 9, 1, 
4, 7, 6

10. The next step was to pretend I knew all that from the beginning and write it up in the previous blog post. Later that evening I thought again and realized that wasn't so truthful and that I should have another shot at writing it up.

Postscript: The ugly WHERE clause was still bugging me and I realized that it could be improved by making some kind of function involving all of the piece ids and checking the answer e.g. if they all add to 45 then that's a solution. Unfortunately, addition isn't a uniquely identifying function, and neither is multiplication or addition of 2^piece_id. What will work though is changing the piece ids to prime numbers and multiplying them all together. Keeping the first digit the same the ids are now:

11, 23, 31, 41, 53, 61, 71, 83, 97

The product of these primes has no other factors and we don't care about the order since multiplication is cummutative, so to ensure each piece appears only once the WHERE can now simply be:

WHERE A.piece_id * B.piece_id * C.piece_id 
    * D.piece_id * E.piece_id * F.piece_id 
    * G.piece_id * H.piece_id * I.piece_id
    = 11*23*31*41*53*61*71*83*97;

SQL: 3x3 Square Picture-Puzzle Solution

2011-06-21T18:53:00.007+10:00

Update: A second attempt at a write up.

After Reilly solved a 3x3 square picture puzzle using Prolog I made a flippant remark that it would be interesting to solve using SQL and well, here goes...

First, create a table and populate it with the pieces:

# table of all pieces in all possible rotations
CREATE TABLE rotated_pieces (
  piece_id     INTEGER, # unique piece id
  rotation     INTEGER, # 0 - top, 1 - left, 2 - bottom, 3 - right
  top         CHAR(10), # animal at top edge
  top_head    CHAR(10), # head or tail at top edge
  `right`     CHAR(10),
  right_head  CHAR(10),
  bottom      CHAR(10),
  bottom_head CHAR(10),
  `left`      CHAR(10),
  left_head   CHAR(10));

# unrotated pieces (top)
INSERT INTO rotated_pieces 
       (piece_id, rotation, 
              top,       top_head, `right`,   right_head, bottom,    bottom_head, `left`,    left_head) 
VALUES (1, 0, 'cheetah', 'tail',   'tiger',   'tail',     'lion',    'head',      'tiger',   'head'),
       (2, 0, 'lion',    'tail',   'lion',    'head',     'tiger',   'tail',      'cheetah', 'head'),
       (3, 0, 'tiger',   'tail',   'lion',    'head',     'panther', 'head',      'tiger',   'tail'),
       (4, 0, 'panther', 'tail',   'cheetah', 'head',     'panther', 'tail',      'lion',    'tail'),
       (5, 0, 'tiger',   'head',   'tiger',   'head',     'cheetah', 'head',      'lion',    'tail'),
       (6, 0, 'panther', 'tail',   'panther', 'head',     'cheetah', 'tail',      'tiger',   'head'),
       (7, 0, 'panther', 'head',   'cheetah', 'tail',     'cheetah', 'head',      'lion',    'tail'),
       (8, 0, 'panther', 'tail',   'lion',    'head',     'panther', 'head',      'cheetah', 'tail'),
       (9, 0, 'cheetah', 'head',   'tiger',   'head',     'panther', 'head',      'lion',    'tail');

Then, rotate each of those pieces so we have all 36 variations:

# rotate (top --> left)
INSERT INTO rotated_pieces (piece_id, rotation, top, top_head, `right`, right_head, bottom, bottom_head, `left`, left_head) 
  SELECT piece_id, rotation+1, 
         `right` AS top, right_head AS top_head, 
         bottom AS `right`, bottom_head AS right_head, 
         `left` AS bottom, left_head AS bottom_head, 
         top AS `left`, top_head AS left_head 
    FROM rotated_pieces;

# rotate (top --> bottom, left --> right) 
INSERT INTO rotated_pieces (piece_id, rotation, top, top_head, `right`, right_head, bottom, bottom_head, `left`, left_head) 
  SELECT piece_id, rotation+2, 
         bottom AS top, bottom_head AS top_head, 
         `left` AS `right`, left_head AS right_head, 
         top AS bottom, top_head AS bottom_head, 
         `right` AS `left`, right_head AS left_head 
    FROM rotated_pieces;

Then, it's a pretty simple query to return the result. The code is messed up by the WHERE clause which ensures that each piece is used only once in the solution, if there's a neater way to do this I'd like to know.

# find solution
SELECT A.piece_id AS A, A.rotation AS Ar, B.piece_id AS B, B.rotation AS Br, C.piece_id AS C, C.rotation AS Cr,
       D.piece_id AS D, D.rotation AS Dr, E.piece_id AS E, E.rotation AS Er, F.piece_id AS F, F.rotation AS Fr,
       G.piece_id AS G, G.rotation AS Gr, H.piece_id AS H, H.rotation AS Hr, I.piece_id AS I, I.rotation AS Ir
  FROM rotated_pieces A 
INNER JOIN rotated_pieces B ON (A.`right` = B.`left` AND A.right_head != B.left_head)
INNER JOIN rotated_pieces C ON (B.`right` = C.`left` AND B.right_head != C.left_head)
INNER JOIN rotated_pieces D ON (A.bottom = D.top     AND A.bottom_head != D.top_head)
INNER JOIN rotated_pieces E ON (D.`right` = E.`left` AND D.right_head != E.left_head
                                AND B.bottom = E.top AND B.bottom_head != E.top_head)
INNER JOIN rotated_pieces F ON (E.`right` = F.`left` AND E.right_head != F.left_head
                                AND C.bottom = F.top AND C.bottom_head != F.top_head)
INNER JOIN rotated_pieces G ON (D.bottom = G.top     AND D.bottom_head != G.top_head)
INNER JOIN rotated_pieces H ON (G.`right` = H.`left` AND G.right_head != H.left_head
                                AND E.bottom = H.top AND E.bottom_head != H.top_head)
INNER JOIN rotated_pieces I ON (H.`right` = I.`left` AND H.right_head != I.left_head
                                AND F.bottom = I.top AND F.bottom_head != I.top_head)
WHERE A.piece_id != B.piece_id AND A.piece_id != C.piece_id AND A.piece_id != D.piece_id
  AND A.piece_id != E.piece_id AND A.piece_id != F.piece_id AND A.piece_id != G.piece_id
  AND A.piece_id != H.piece_id AND A.piece_id != I.piece_id
  AND B.piece_id != C.piece_id AND B.piece_id != D.piece_id AND B.piece_id != E.piece_id
  AND B.piece_id != F.piece_id AND B.piece_id != G.piece_id AND B.piece_id != H.piece_id
  AND B.piece_id != I.piece_id AND C.piece_id != D.piece_id AND C.piece_id != E.piece_id
  AND C.piece_id != F.piece_id AND C.piece_id != G.piece_id AND C.piece_id != H.piece_id
  AND C.piece_id != I.piece_id
  AND D.piece_id != E.piece_id AND D.piece_id != F.piece_id AND D.piece_id != G.piece_id
  AND D.piece_id != H.piece_id AND D.piece_id != I.piece_id
  AND E.piece_id != F.piece_id AND E.piece_id != G.piece_id AND E.piece_id != H.piece_id
  AND E.piece_id != I.piece_id
  AND F.piece_id != G.piece_id AND F.piece_id != H.piece_id AND F.piece_id != I.piece_id
  AND G.piece_id != H.piece_id AND G.piece_id != I.piece_id
  AND H.piece_id != I.piece_id;

This returns 4 rows, the first is our solution:

2,1, 1,0, 6,0, 
8,3, 9,3, 7,2, 
5,3, 3,0, 4,0

The remaining 3 are the same solution with the entire board rotated:

6,1, 7,3, 4,1, 
1,1, 9,0, 3,1, 
2,2, 8,0, 5,0

4,2, 3,2, 5,1, 
7,0, 9,1, 8,1, 
6,2, 1,2, 2,3

5,2, 8,2, 2,0, 
3,3, 9,2, 1,3, 
4,3, 7,1, 6,3

I would argue the SQL version is clearer, but the Prolog version is slightly terser and should be easier to extend to a 4x4 puzzle.

Merrell Trail Gloves review

2011-02-21T13:31:00.004+11:00

A quick review of the Merrell Trail Glove minimalist trail running shoes.

After reading a few reviews of these shoes I was curious enough to fork out the money for them (Barefoot Running University and I run far.)

On your foot

The fit is snug in the heel and the laces really grip around the mid-foot and arch. I only have the Vibram FiveFingers and walking shoes to compare to, maybe regular running shoes are more like this too, but they are different to anything I have run in before. The toes have tonnes of room and are completely free to move, but don't slide around at all because the midfoot is held solidly.

They breath pretty well, a lot of the upper is mesh/fabric, the tongue's a bit thick so they're pretty warm but regular shoes+socks would be worse.

They let a bit of dirt in around the ankle and I think through the mesh/fabric (I wear them without socks), but because it can move around inside the shoe easily I don't think it's as annoying as when wearing Vibrams.

Most of my shoes are 43 or 44 (Vibrams are 42), and I got the Merrells in size 44 to make sure my toes would have room to splay and wouldn't hit the end heading downhill. This was a bit of a mistake, the way they fit means you won't slide inside them and the bit of extra length means I catch my toe on steps a bit more than I'd like. Not a show stopper, I just have to lift my feet higher.

On the ground

The sole has a bit of cushioning and feels a little soft on the footpath, but you don't notice it as soon as you get off road. It's much thicker and stiffer than the Vibrams, with less ground feel, but still very flexible and nothing like a normal running/trail shoe.

The tread has maybe 2-3mm deep chunks, bigger on the toe and heel. It's very curved and foot shaped, no attempts to control side ways stability or pronation, which is great and they are a tiny bit lighter than Vibram KSOs.

They feel bombproof, no fear treading on rocks/stones/sticks and I'm definitely not going to catch my little toe again like I did wearing Vibrams.

Conclusion

So, these are pretty much exactly what I've been waiting for. I'll only wear them on rough/rocky trails, and they're not "barefoot" by any stretch of the imagination, but they are great minimal shoes and the biggest problem I can see is that I'm going to be exhausted, the Vibrams give you an excuse to slow down occasionally "to negotiate tricky terrain" :)

Dandenong Ranges

LED Bike Light update

2011-02-04T12:45:00.004+11:00

My home built LED bike light performed well for many years, but when I got my new bike, the battery holder and handlebar mounting solutions no longer functioned adequately so it was time for a redesign.

I had been planning on a new simplified version, eliminating the micro-controller and putting in a simple dimmer potentiometer. It turned out I never really used the battery monitoring features and simply used a multimeter to decide when to recharge, and using a dimmer would allow setting the brightness easily and quickly and also allow the "high beam" mode to remain on through hazardous sections when necessary.

I was still pretty keen on building this myself as all it really required was cannibalising my old light for parts and making a new case. Commercial solutions had dropped a lot in price but are still many times more expensive than DIY (a few friends have these AYUP lights for example and they seem great).

Then I discovered semi-cheap LED torches as an option and ended up buying a Fenix LD20 LED torch and a small clamp from Ebay (that I can't find again, glad I bought 2...)

This puts out almost exactly the same amount of light as my old version but is in a small waterproof and easy to use package. As a bonus, it's a torch I can take camping etc. too.

I'm simply rotating through the NiMH AA batteries from the old light and am a bit disappointed with the battery life, seeming to have to change them more often than I'd like, but that's pretty quick and an extra set of batteries isn't too much of a struggle to carry around.

All in all, for $80 you can't go too wrong.

Preparing an audiobook for listening on an mp3 player

2010-12-16T15:08:00.003+11:00

I recently ripped a 7-CD audiobook to listen to on my Sansa Clip+ MP3 player. After a bit of fiddling around with bash I was able to end up with a collection of 183 chapters named 1_01.mp3 through to 7_26.mp3 (the first bit was the disc number, then the track number.)

Simply copying these onto my player didn't work as the tracks were all out of sequence. In the past, with smaller books, I've simply skipped around between the sections to listen in order, but that wasn't going to be an option here.

After some experimenting I discovered the Sansa orders the tracks by the Track Number in the ID3 tag, so I fired off the following bash command to set up the ID3 details appropriately:

n=0 ;
for f in *.mp3 ;
do n=$((n+1)) ;
id3tag --artist="author name" --album="book name" --song="${f/.mp3/}" --track=$n $f ;
done

This numbers all the tracks from 1 to 183 in order and also sets the artist and album so I can find it easily in the Audiobook list. The .mp3 is stripped from the displayed filename too.

The id3tag program came from the libid3-3.8.3-dev package, not sure why/when I installed that one...

Python+Emacs made easy with emacs-for-python

2010-12-04T11:08:00.003+11:00

I don't know where this has been all my life but Gabriele Lanaro has put together a really easy to use package for adding lots of Python goodies to Emacs.

Just unzip the archive into ~/.emacs.d/ and add one line to your .emacs and your done (well, also remove all the now unnecessary random cargo-cult prior additions to your .emacs):

(load-file "~/.emacs.d/emacs-for-python/epy-init.el")

I had also previously installed a handful of Python/Emacs packages via Synaptic that may or may not be required to make it all work: python-rope, python-ropemacs, pyflakes.

Increasing podcast tempo (playback speed) with Mplayer and Lame

2010-07-16T19:02:00.006+10:00

The following commands will re-encode an mp3 file at a faster speed without increasing the pitch and making it sound like a chipmunk. How this actually works is pretty neat.

mplayer -vo null -vc null \
        -speed 1.33 \
        -af scaletempo,volume=0,resample=44100:0:1 \
        -ao pcm:fast:waveheader:file=temp.wav \
        source.mp3
lame -b 64 --resample 22.05 temp.wav faster.mp3

It will also set the bitrate to 64kbit which was for compatibility with my old player and to reduce the filesize for some podcasts which are unnecessarily big.

There should be a way to use mkfifo instead of temp.wav, and run the mplayer process and lame at the same time, I'll update this when I figure it out.

Update: The above method broke after upgrading to Ubuntu 11.10 (one or both of mplayer or lame changed something I suspect...), so I swapped it out for the simpler and better SoX:

sox --show-progress --norm source.mp3 dest.mp3 \
    tempo -s 1.33 channels 1 rate 22050

This version is easier to understand, faster, doesn't leave need a temp file and also, as a bonus, normalizes the audio to prevent clipping and not hurt my ears.

Using Sansa Clip+ MP3 Player for Podcasts

2010-07-16T18:53:00.007+10:00

A mini-review after buying the 2GB model to replace my old mp3 player. I'll only consider it for the purpose of listening to podcasts, other music related features can be found on the SanDisk site or other reviews easily enough.

3 key features for listening to podcasts:

No software required, just plug in as a USB drive and copy files across (using a script or manually.)
Ability to easily delete files from the player after listening to them.
Increase the tempo/speed of playback.

The Sansa Clip+ does all of these, however the fast playback speed is pretty useless as the pitch is increased giving "chipmunk" effects. See the next post for a work-around.

The player remembers the position in each track, which is very nice.

A couple of other issues I've noticed immediately are that it is very small, making one-handed operation difficult, and it has no "hold" button to prevent inadvertent bumping of the buttons. Hopefully being able to clip it outside clothing/bags will alleviate this, time will tell.

It also has a built in battery so you can't carry a spare and must plug it in to charge.

Update: So far, everything is working well. I'm not used to the built in battery, and waiting for it to recharge if I let it go flat is a major pain, but I'll just have to keep it topped up and an hour of charging goes a long way.

One other minor annoyance is that you can't quickly see the duration of a podcast when skipping through them.

Update: One issue that I initially didn't notice, as the ID3 tags were stripped during my reencode process, is that the podcasts are organised by show and there is no "play all" option. This is a pain, I'm used to just listening to them in a more or less random order and not having to stop and select a new show when one finishes.

To quickly fix this, use the id3v2 utility to delete all ID3 tags from the podcast before copying them to the player. This way they are all categorised under a single "Unknown" show menu and the filename is usually sufficient to identify the program.

Making use of all the space on a hard drive

2010-05-12T19:10:00.003+10:00

If you have a big data drive, there's a good chance that Linux is reserving 5% of it's available space for "emergencies".

This is necessary for your root filesystem as if the disk fills up there may be no way for root to log in and clean things up without this buffer of space for log files etc. to be written into.

However, I have a couple of large drives that store only data and 5% is 100GB I'd rather be able to use.

Fortunately, it's easy to reclaim the space, the following will remove all "reserved blocks":

sudo tune2fs -m 0 /dev/sdXX

This can be reset back to the default 5% (or any percentage) if you need to:

sudo tune2fs -m 5 /dev/sdXX

iPhone woes

2010-01-22T20:05:00.004+11:00

My iPhone 3GS died 2 days into our holiday. Something like an IMEI/ICCID not found error, and the only thing it would display was a screen asking to be connected to iTunes to be restored.

Connecting to iTunes wouldn't let me restore as it said my phone was locked with a passcode. Using the forced restore mode, iTunes wouldn't restore without an Internet connection. Once I got a connection, the restore failed with an "Error 23" and the log file contained "radio" errors. Didn't sound good. Tried a DFU mode restore, same "Error 23".

So, it was bricked. My last backup was in October. My bad I guess, but I can only use iTunes via my partner's laptop and usually I don't need to. Thankfully it was still covered by the 1 year warranty.

Calling Apple went smoothly. They acknowledged that I'd done everything I could and booked me an appointment to take the phone into the Apple store.

The Apple store is crazy, there were more staff than customers and the Genius bar was booked out for 2 days. That's where your extra $ go when you buy a Mac, buying blue t-shirts...

The Genius replaced my handset with no troubles, which was great.

Restoring from my backup didn't work very well.

My Apps weren't restored at all, only the links to Web Apps I'd saved to the home screen were restored. This is possibly because I hadn't "authorised" iTunes with my account. Hard to recall if I was warned about that, I didn't know it was necessary in any case.

Restoring Apps manually is a pain, trawling through iTunes receipt emails, figuring out which I hadn't uninstalled and which I wanted to keep using.

Additionally, you have to buy the App again, only once that's done does it confirm that you're not going to have to pay for it again (and gives you a cancel button, God knows why.) If you choose the wrong App then bad luck, the one-click purchase will install it with no cancel button.

One tiny bit of good news: Many App's settings were backed up, and the settings were restored after re-purchasing the App. This wasn't clear initially, but obviously it backs up and restores all the files in the "home" folder, whether the App is present or not.

The music wasn't synced at all as I had iTunes set to manage it manually. Fair enough in that case, but still tedious to get it back.

The lesson to be learnt is that an iPhone backup isn't really a backup at all. It apparently backs up your "home" folder and relies on the Sync to do the rest. This is sort of obvious in hindsight, as it's not like there's going to be room for a 32gb backup, and theoretically that's mostly copies of files (music) that's on the computer anyway.

Just because I don't want everything to be automatically Synced doesn't mean I don't want it backed up.

So, I'll be changing my habits from now on. I won't be switching to using iTunes, that's not an option on Linux, but I'll definitely treat the iPhone as a terminal only and try to keep as much as possible "in the cloud". Using Google Sync for my contacts in addition to my mail and calendars from now on for example.

Canvas Game on iPhone

2009-09-27T18:37:00.003+10:00

I just ported my little canvas game to the iPhone:

Go play!

GPSLog Labs

2009-08-13T15:30:00.002+10:00

GPSLog Labs is a site I've been working on where you can upload logs from your GPS tracking device and map, graph and analyse them.

It will let you track your exercise and training whether your a cyclist or runner, and can also be used to track your mileage and keep a diary of your activity.

It builds on a few of the things I've posted about here before, and there's some more information on the GPSLog Labs blog.

Signing up is trivial as it supports OpenID, I'd appreciate any of your comments and suggestions and hope the site interests some of you.

Hopeless Exetel customer service

2009-07-14T16:34:00.002+10:00

I got the following message regarding a line fault issue we have with our telephone service:

Please note that the supplier technician who has attended to your service issue has confirmed to us that there is no issue with in the infrastructure/network boundary point and or main distribution frame (MDF). Please re-check your equipment.

That was a surprise to me, as on Saturday the technician had found a fault in the line but had been unable to pull the cable and had said that Optus would have to dig it up and we'd find out more later.

Calling Exetel help was a struggle, after waiting on hold and explaining my issue, the call dropped out when they seemed to put me back on hold. Twice.

Thoroughly pissed off I spent another 20 minutes on hold and listening to complete silence while the help desk person was "just one second" (turns out he was talking to Optus, would have been nice to be told why I was waiting).

So, at this point, 50 wasted minutes later, I find out that Optus is going to fix the line after all, and that the message I received was due to Exetel "closing" the initial fault ticket and was completely wrong.

Idiots.

Merging GPS logs and mapping them all

2008-10-11T12:04:00.000+11:00

Inspired by cabspotting and Open Street Map, I wanted to merge all my GPS logs and create a map showing all the routes I've logged lately.

This is pretty easy using gpsbabel, but I needed to use a little Python to get the list of input log files. (I'm sure there's a way to do it in bash but that's beyond me for now.) My GPS stores files in nmea format, and the directory structure/purpose of my Python script should hopefully be apparent.

>>> import os
>>> from path import path
>>> logs = " ".join([" ".join(["-i nmea -f %s"%log
                               for log in sorted((raw/"raw").files("GPS_*.log"))]) 
                     for raw in path("/home/tom/docs/gpslogs").dirs() 
                     if raw.namebase.isdigit()])
>>> logs
'-i nmea -f /home/tom/docs/gpslogs/200810/raw/GPS_20080930_221152.log -i nmea -f /home/tom/docs/gpslogs/200810/raw/GPS_20081001_071234.log ...'
>>> os.system("gpsbabel %s -o kml,points=0,labels=0,trackdata=0 -F /home/tom/docs/gpslogs/all200810.kml" % logs)

The result of that is a 36.5 MB kml file I could load into Google Earth:

There was one spurious point somewhere in the log file at 0° E, 0° N, and the log has a lot of jitter when I'm walking near home.

A simple game using JavaScript and the CANVAS element

2008-09-18T16:01:00.008+10:00

This should work in Opera and Firefox (though really slowly, why I don't know) and probably Safari. It's not going to work in IE because I can't be bothered getting the iecanvas thingi into the blog code.

Update: There is now an iPhone version.

The only thing to note is less clicks == better score.

Clicks

0
0	0	0	0

Analysing GPS Logs with Awk

2008-08-25T18:56:00.003+10:00

This post describes the first two "chop" functions that fit into the partitioning framework outlined last post.

def chopToSpeedHistogram(dest, p):
    # create histogram of speeds from nmea written to stdout
    os.system("cat "+sh_escape(dest)+".log"
              + " | awk -F , '{if($1==\"$GPVTG\" && int($8)!=0){count[int($8+0.5)]++}}"
              + " END {for(w in count) printf(\"[%d,%d],\\n\", w, count[w]);}'"
              # sort it
              + " | sort -g -k 1.2"
              # output json of histogram
              + " > "+sh_escape(dest)+".hist")

def chopToHeadingHistogram(dest, p):
    # create histogram of headings from nmea written to stdout (ignore heading when stopped)
    os.system("cat "+sh_escape(dest)+".log"
              + " | awk -F , '{if($1==\"$GPVTG\" && int($8)!=0){count[5.0*int($2/5.0+0.5)]++;}}"
              + " END {for(w in count) printf(\"[%d,%d],\\n\", w, count[w]);}'"
              # sort it
              + " | sort -g -k 1.2"
              # output json of histogram
              + " > "+sh_escape(dest)+".head")

Both functions use awk to create a histogram from the speed (in km/h) and heading (or bearing, in degrees) from the NMEA VTG sentences. The speed is rounded to an integer, and the bearing to the nearest 5 degrees. The data logger records on reading per second, so this gives a measure of how much time was spent at each speed/bearing.

The histogram is output in a "json" array format that can be inserted straight into a webpage where the flot library is used to generate some graphs.

Speed Histogram

The average and standard deviation (shaded at ±0.5σ) are indicated on the graph for two bike rides along the same route, and match pretty closely with that recorded by my bike computer:

	GPS log			Bike computer
Ride 1 (brown)	2hrs 59 min, minus 41 min stopped	63.4km	27.7km/h	2 hrs 16 min	64.01km	28.00km/h
Ride 2 (dark green)	2 hrs 25 min, minus 13 min stopped	63.4km	29.0km/h	2 hrs 10 min	63.85km	29.30km/h

The two rides went in different directions, the first in the "uphill" direction and the second with a bit of a tail wind. I got a flat tire on the first ride too, hence the extra time spent stopped.

Heading Histogram

Up is north and the radius represents the time spent heading in that direction (normalized during the plotting process and "expanded" by taking the square root to show a little more detail.)

Automatically Partitioning GPS Logs with gpsbabel

2008-08-21T14:47:00.002+10:00

My GPS logger is capturing lots of useful information but it's difficult to efficiently capture data for regular activities. Geotagging photos is easy, and manually working with the logs for a special event is possible, but it's not feasible to put in that much work to analyze commutes for example.

The logger creates a separate log file each time it's switched on and off, and while these logs could be sorted into categories for analysis, it's easy to forget to turn it on and off at the start and end of a section of interest and activities are then merged in the logs. In addition, there is often "junk" data at start and end of logs while leaving or arriving at a destination.

I wanted to be able to automatically capture the information about my daily activities by simply switching on the logger and carrying it around with me. I then simply want to plug the logger into the computer and have the logs automatically chopped into segments of interest that can be compared to each other over time.

The rest of this post roughly outlines the Python script I created to perform this task, minus some of the hopefully irrelevant details.

Firstly, I collect the lat/long coordinates of places that I am interested in collecting data while I'm there and traveling between them. These include my home, work, the climbing gym and so on. Each point has a radius within which any readings will be considered to be in that place.

#         id:  name lat         long        radius
places = { 1: ("A", -37.123456, 145.123456, 0.050),
           2: ("B", -37.234567, 145.234567, 0.050),
           3: ("C", -37.345678, 145.345678, 0.050) }
otherid = 4

For each of these places of interest, I then use gpsbabel's radius filter to find all the times where I was within that zone:

# create a list of all raw log files to be processed
from path import path
month = path("/gpslogs/200808")
logs = " ".join(["-i nmea -f %s"%log 
                 for log in sorted((month/"raw").files("GPS_*.log"))])

for (id,(place,lat,lon,radius)) in places.items():
   os.system("gpsbabel "
             # input files
             + logs
             # convert to waypoints
             + " -x transform,wpt=trk,del"
             # remove anything outside place of interest
             + (" -x radius,distance=%.3fK,lat=%.6f,lon=%.6f,nosort"%(radius,lat,lon))
             # convert back to tracks
             + " -x transform,trk=wpt,del"
             # output nmea to stdout
             + " -o nmea -F -"
             # filter to just GPRMC sentences
             + " | grep GPRMC"
             # output to log file
             + (" > %s/processed/place%d.log"%(month,id)))

And all points outside any of the specific places of interest are sent into an "other" file:

os.system("gpsbabel "
          # input files
          + logs
          # convert to waypoints
          + " -x transform,wpt=trk,del"
          # remove anything in a place of interest
          + "".join([" -x radius,distance=%.3fK,lat=%.6f,lon=%.6f,nosort,exclude"%(radius,lat,lon)
                     for (id,(place,lat,lon,radius)) in places.items()])
          # convert back to tracks
          + " -x transform,trk=wpt,del"
          # output nmea to stdout
          + " -o nmea -F -"
          # filter to just GPRMC sentences
          + " | grep GPRMC"
          # output to log file
          + (" > %s/processed/place%d.log" % (month, otherid)))

These files are filtered with grep to contain only minimal data as we only require the timestamps for this part of the process. Specifically only the NMEA GPRMC sentences are kept.

To provide a brief illustration, the following picture shows two log files of data, a blue and a green, between three points of interest:

The above process would create four files, one for each point A, B and C and one for "Other" points that would contain something like the following information, where the horizontal axis represents time:

I then read all those log files back in to create a "time line" that for each timestamp stores my "location" in the sense that it knows whether I was "home", at "work" or somewhere between the two.

# dict of timestamp (seconds since epoch, UTC) to placeid
where = {}
for placeid in places.keys()+[otherid,]:
   for line in (month/"processed"/("place%d.log"%placeid)).lines():
      fields = line.split(",")
      # convert date/time to seconds since epoch (UTC)
      t, d = fields[1], fields[-3]
      ts = calendar.timegm( (2000+int(d[4:6]), int(d[2:4]), int(d[0:2]),
                             int(t[0:2]), int(t[2:4]), int(t[4:6])) )
      where[ts] = placeid

This is then summarised from one value per second to a list of "segments" with a start and end time and a location. Unlogged time segments are also inserted at this point whenever there are no logged readings for 5 minutes or more.

# array of tuples (placeid, start, end, logged)
# placeid = 0 indicates "unknown location", i.e. unlogged
summary = []
current, start, stop, last_ts = 0, 0, 0, None
for ts in sorted(where.keys()):
   # detect and insert "gaps" if space between logged timestamps is greater than 5 minutes
   if last_ts and ts-last_ts > 5*60:
      if current:
         summary.append( [current, start, stop, True] )
      current, start, stop = where[ts], ts, ts
      summary.append( [0, last_ts, ts, False] )
 
   last_ts = ts

   if where[ts] != current:
      if current:
         summary.append( [current, start, stop, True] )
      current, start, stop = where[ts], ts, ts
   else:
      stop = ts
summary.append( [current, start, stop, True] )

(If there's a more "Pythonic" way of writing that kind of code, I'd be interested in knowing it.)

"Spurious" segments are then removed. These show up because when the logger is inside buildings the location jumps around and often out of the 50m radius meaning that, for example, there will be a sequence of Home-Other-Home-Other-Home logs. The "Other" segments that are between two known points of interest and less than 5 minutes long are deleted, as are "Other" segments that sit between a known place of interest and an unlogged segment.

Based on the above graphic, the summary might look something like the following:

start	end	location
10.00am	10.05am	A
10.05am	10.30am	Other
10.30am	10.35am	B
10.35am	11.00am	Other
...

The "Other" segments are then labelled if possible to indicate they were "commutes" between known locations:

start	end	location
10.00am	10.05am	A
10.05am	10.30am	A-B
10.30am	10.35am	B
10.35am	11.00am	B-C
...

Some segments cannot be labeled automatically and are left as "Other". This may be a trip out to a "one-off" location and back again, which can be left as "Other". However, sometimes it is because the logger didn't lock onto the satellites within the 50m radius on the way out of a place of interest and these can be manually fixed up later.

Once a list of "activities" has been obtained, with start and end times, it is easy to use gpsbabel again to split logs based on start and end of time segments:

for (place, start, stop, place_from, place_to, logged) in summary:
    dest = month / "processed" / ("%s-%s"%(time.strftime("%Y%m%d%H%M%S", time.localtime(start)),
                                           time.strftime("%Y%m%d%H%M%S", time.localtime(stop))))

   for (ext, chopFn) in [(".log", chopToLog),
                         (".kml", chopToKml), 
                         (".speed", chopToSpeedVsDistance), 
                         (".alt", chopToAltitudeVsDistance), 
                         (".hist", chopToSpeedHistogram),
                         (".head", chopToHeadingHistogram),
                         (".stops", chopToStopsVsDistance)]:
      if not (dest+ext).exists():
         chopFn(dest, locals())
         # make the file in case it was empty and not created
         (dest+ext).touch()

This generates a bunch of files for each segment, named with the start and end timestamps of the segment and an extension depending on the content. The first "chop" function generates an NMEA format log file that is then processed further by the remaining "chop" functions. The other chop functions will probably be explained in a later post, the first two are:

def chopToLog(dest, p):
    # filter input file entries within times of interest to temp file
    os.system("gpsbabel " + p["logs"]
              + (" -x track,merge,start=%s,stop=%s"
                 % (time.strftime("%Y%m%d%H%M%S", time.gmtime(p["start"])),
                    time.strftime("%Y%m%d%H%M%S", time.gmtime(p["stop"]))))
              + " -o nmea -F "+sh_escape(dest)+".log")

def chopToKml(dest, p):
    # create kml file with reduced resolution
    os.system("gpsbabel -i nmea -f "+sh_escape(dest)+".log"
              + " -x simplify,error=0.01k"
              + " -o kml -F "+sh_escape(dest)+".kml")

def sh_escape(p):
    return p.replace("(","\\(").replace(")","\\)").replace(" ","\\ ")

(Again, if there's a better way to handle escaping special characters in shell commands, I would like to know it.)

Using this, I can simply plug in the logger, which launches an autorun script, and the end result are nicely segmented log files that I can map and graph. More about that in another post.

GMail tip: "To be filtered" label

2008-08-18T22:28:00.000+10:00

I use GMail's filters a lot, and in particular have a "Newsletters" label that keeps "bacn" from getting into my Inbox (and instead just lights up my ambient email notifier green.)

I've added a "to be filtered" label that I can apply to anything that slips into the Inbox and then later on, when I have time, I can look at all these messages, select them in groups and use the "Filter messages like these" command to make sure they don't bother me again.

This solves two problems: being taken out of the "flow" to create a filter, and perpetually handling the items manually and never getting around to creating a filter.

AMOD AGL3080 GPS data logger

2008-08-18T12:53:00.003+10:00

I bought the AMOD AGL3080 GPS data logger about a month ago now and am very happy with it. Thanks Richard for a very helpful review.

I've left it on the default settings, logging everything in NMEA format at 1 second intervals. It seems quite easy to change the logging interval and detail but at this stage, I'm hard pressed to imagine a situation where I'll need to as I've logged about 62 hours so far and only used about 50% of the 128mb capacity.

It's a little larger than I'd like because it takes three AAA batteries. I'm using eneloops and getting between 10 and 12 hours per charge (changing them as soon as I notice the battery warning light flashing, I don't know how long it would continue running if left to run flat completely.)

The logging seems quite accurate while driving and cycling, but has a lot of "jitter" while walking for some reason. It generally locks onto the signal quickly enough once I get outside, but again while walking it seems to take an unusually long time to lock on which is a little frustrating.

It's best feature is that it doesn't require any special software to access the logs. It simply mounts as a USB drive allowing the NMEA format log files to be copied off. This is great and makes it particularly attractive for Linux and Mac users.

Using the Gosget GPS Data Logger in Linux via Wine (Ubuntu 8.04)

2008-06-19T19:19:00.000+10:00

I managed to get the Gosget GPS Data Logger S1 to work under wine on Ubuntu 8.04 using (roughly) the following steps:

Plug it into a usb port, Ubuntu will hot-plug it as a serial port automatically (probably /dev/ttyUSB0).
Set the baud rate on the serial port:
stty 115200 < /dev/ttyUSB0
(I'm not sure if this is persistent after a reboot or not...)
Create a "com" port in wine:
ln -s /dev/ttyUSB0 ~/.wine/dosdevices/com1
Put in the CD and run the installer in Wine:
wine d:\PC\DataLogUtility\DataLogUtility.exe
Run the Data Log Data Downloader via wine, put in the com port, connect, set a download folder and you'll be able to get .nmea files from the data logger.
To convert the .nmea file to a .kml file I used gpsbabel -i nmea -o kml source.nmea dest.kml.

Figuring that out gave me a headache, hopefully this post will save someone else one. Please note however, that I'm writing the commands from memory so there may be something (hopefully harmless) wrong.

Also, if you switch the data logger into "G_Mouse" mode, it's possible to use gpsd to get GPS data in real-time. I switched the data-logger into "G_Mouse" mode via a Windows machine, but I presume the Data Log Data Downloader can do that via wine too.

And, gpsd and gpsbabel are both in the Ubuntu 8.04 repositories.

Ambient Email Notifier in Linux

2008-05-18T17:31:00.000+10:00

I finally got around to hooking up my Ambient Email Notifier under Ubuntu. No idea why I waited so long, given it pretty much worked as soon as I plugged it in!

The Linux kernel has drivers for the CP2102 USB-to-RS232 chip built in, so as soon as it was plugged in it showed up as /dev/ttyUSB0.

The Python-Serial module is in the Ubuntu repositories, so after installing that, I just changed my checkinbox.py script to use /dev/ttyUSB0 instead of COM3 and added a cron job to check every 10 minutes.

Ubuntu 8.04 + Logitech iTouch Keyboard = Dead Battery?

2008-05-18T13:50:00.001+10:00

How is this possible? I've got a Logitech iTouch wireless keyboard that's been working fine for years now and with Ubuntu 7.10 for 5 months, but after upgrading to Ubuntu 8.04, my keyboard batteries have gone flat 4 or 5 times now. One or more of the batteries seems to "short out" to 0.1v or even go -ve. I've tried 2 sets of NiMH batteries, one brand new and one that had been working fine for years. Could it be anything other than a coincidence?

Update: After about 2 weeks of the batteries dying every day or so, I've now been on the same charge for about 2 weeks now. So, I've no idea what update fixed it, but it's better now!

Ubuntu 8.04 and NVidia TVOut

2008-05-06T21:22:00.000+10:00

Upgrading to Ubuntu 8.04 has been fairly successful, but it took me a while to find out what had stopped my TV from working. Nothing seemed to be different, same xorg.conf file, no errors in /var/log/Xorg.0.log, "space" in the desktop for the TV, but the TV was black.

I finally found this nvnews forum post that gave the clue, and adding the following to the "Device" section in my xorg.conf file did the trick:

Option "TVOutFormat" "SVIDEO"
Option "TVStandard" "PAL-G"

IT Conversations Premium Editions and wget

2008-03-13T14:23:00.000+11:00

The new premium editions of the IT Conversations shows are great, but I couldn't download them using wget, e.g.:

$ wget http://www.conversationsnetwork.org/audio/premium/username|email/ITC.Rails-TimBray-2007.05.19.mp3

--14:27:31--  http://www.conversationsnetwork.org/audio/premium/username
           => `username'
Resolving www.conversationsnetwork.org... 74.53.160.146
Connecting to www.conversationsnetwork.org|74.53.160.146|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
14:27:32 ERROR 404: Not Found.

The problem was caused by the | character separating the username and email address in the personalised URL. "URL encoding" this to "%7C" fixed the problem.

Ambient Email Notifier (some code)

2008-02-29T22:09:00.002+11:00

Will asked about the code I was using in my ambient email notifier. The full code is a bit difficult to figure out because I've got it tied to a system tray icon thingi, which can go in another post another day, but here are some relevant bits.

First, you can get the number of emails with a particular label in gmail with the following Python code:

import feedparser
def msgCount(uid, pwd, filter):
    inbox = feedparser.parse("https://%s:%s@gmail.google.com/gmail/feed/atom%s" % (uid, pwd, filter))
    return len(inbox["entries"])

uid is your gmail address without the @gmail.com bit, filter is "" for the inbox, and "/label/" to get messages tagged with a particular label.

So, after calling msgCount a few times, for different labels, I compute the colour of the RGB LED:

colour = (inbox > 0 and 1 or 0) + (news > 0 and 2 or 0) + (work > 0 and 4 or 0)

This is sent over the serial port to the picaxe which decodes bit 0 for blue, 1 for green and 2 for red.

def triggerAmbient(colour):
    com = serial.Serial("COM3", 2400, timeout=0.25)
    for attempt in range(0,10):
        com.write("%c" % (colour) )
    com.close()

It tries to send a few times in case the picaxe doesn't get it the first time. The code on the picaxe just listens for a byte on the serial input and outputs the lowest 4 bits to the output pins:

main:
 serout 0, n2400, ("Ok")
 serin 3, n2400, b0
 gosub nibble3
 b0 = b0 & 7
 serout 0, n2400, (#b0)
 goto main
 
nibble3:
 if bit2 = 1 then
  high 1
 else
  low 1
 endif
 if bit1 = 1 then
  high 2
 else
  low 2
 endif
 if bit0 = 1 then
  high 4
 else
  low 4
 endif
 return

This means the python script has control over the colour and you can test that it's working by simply opening up a terminal on COM3 and typing away (A is 01000001 in ASCII, meaning pin 1 is switched on, B is 01000010 so pin 2 is on, etc.)

Update: I've detailed the changes I made to get this working under Linux.

abcde — A Better CD Encoder

2008-02-29T20:51:00.002+11:00

ABCDE is a nice CD ripper/encoder utility for Linux.

It's all nice and automatic, and I managed to configure it to suit my mp3 management style pretty quickly. My .abcde.conf file:

OUTPUTDIR=/home/tom/Desktop
OUTPUTTYPE=mp3
LAMEOPTS='-h -b 224'
ACTIONS=default,playlist
OUTPUTFORMAT='${ARTISTFILE}/${ARTISTFILE} - ${TRACKFILE}'
VAOUTPUTFORMAT='Various/${ARTISTFILE} - ${TRACKFILE}'
PLAYLISTFORMAT='${ARTISTFILE}/${ARTISTFILE} - ${ALBUMFILE}.m3u'
VAPLAYLISTFORMAT='Various/Various - ${ALBUMFILE}.m3u'

That sets it up to rip to mp3s at 224kbs and write out a playlist file. The files are saved onto the desktop to test before I move them to my mp3 folder in the "Artist/Artist - Track.mp3" name format that I prefer.

mungefilename () {
 echo "$@" | sed s,:,\ --\ ,g | sed s,/,\ --\ ,g | tr \* + | tr -d \"\?\[:cntrl:\]
}

I also changed the filename processing so that colons and forward slashes are converted to double dashes (my scripts use single dashes to split the artist from the track title.) And I don't convert spaces to underscores and I leave single quotes alone.

MAXPROCS=2

Runs an encoder process on each core so it goes super fast.

I run it from an icon on the desktop with the following command line:

gnome-terminal -x abcde

Update: I just discovered the "gnome-terminal -x" is unnecessary, there's an option to launch an application in a terminal window automatically...

Backup Log Visualisation

2008-02-23T09:42:00.001+11:00

I'm having a bit of trouble figuring out how to describe this. It's a page that shows the current status and history of my backup strategy, and hopefully it's fairly obvious what's going on.

Each time the rsync process runs, the results are logged to a text file. A python routine parses the log file to determine the time the rsync was run, the number of files copied etc.

A little calendar indicates if a copy was made in each time slot of the day, this gives an interesting indication of when my computer is switched on. A histogram at the top shows the all time counts, the calendar only shows the last 30 days.

I'm using the Google Chart API to graph the number of files, total size, number of files copied and size of files copied.

The results of sbackup are determined by looking at the dated backup files and their sizes and graphed on a calendar. The size of the incremental and full backups are plotted on separate scales.

I'm also running a dump from MySQL once a day.

Podcast and Audiobook Queue Visualisation

2008-02-17T14:34:00.003+11:00

Some screenshots of a little web app I created to visualise my podcast and audio book listening queue:

This shows files currently on my mp3 player. The width of each file represents it's size, the "window width" being 128mb.

Files that will be copied during the next "sync" are indicated with a red dot. Files are copied according to priority, to fill the player and to have only 3 audio book chapters at any time on the player unless there is nothing else to listen to.

Audio book chapters that have been copied/listened to already are highlighted, as are those that have been downloaded but not yet queued.

The displayed title is taken from the id3 tags if possible, otherwise the filename is used. Dates and redundant "book titles" are removed from the title if possible to maximise the amount displayed, and there is a "hover panel" to display other details.

There's also a pretty calendar to provide a gauge of the age of podcasts in different categories.

Photos by Day of Year

2008-01-22T23:54:00.000+11:00

One last "heat-map", by day of the year:

	Jan	Feb	Mar	Apr	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec	Total
1	174	390	1	21	3	9	13	51	13	10	16	72	773
2	7	332	20	3	13	19	8	19	49	83	20	15	588
3	37	271	13	9	35	7	93	43	115	39	48	40	750
4	6	86	12	19	4	24	30	186	133	38	144	34	716
5	70	99	2	12	4	5	45	49	70	12	74	52	494
6	73	88		5	57	55	29	93	84	166	120	17	787
7	169	102	2	55	11	83	30	23	123	39	34	49	720
8	203	206	3	70		20	58	22	88	39	45	20	774
9	130	88	7	21	4	89	47	15	51	106	12	78	648
10	65	154	9	17	5	51	51	25	12	146	12	27	574
11	30	381	6	46	2	100	58	42	42	134	48	1	890
12	23	199	20	38	13	35	90	38	29	102	48	3	638
13	36	190	19	23	3	21	115	59	15	49	2	19	551
14	50	263	23	10	39	7	104	159	70	17	53	23	818
15	57	197	3	9	14		32	263	21	92	24		712
16	37	199	16	39	10	57	10	188	101	60	45	42	804
17	66	238	23	20	12	22	80	63	69	16	15	2	626
18	109	223	15	43	12	70	96	54	53	12	5	9	701
19	163	239	87	117	17	20	20	110	101	14	108	13	1009
20	164	31	52	97	68	31	64	18	105	5	18	37	690
21	200		32	73	19	21	28	72	37	22	5	10	519
22	194	16	1	43	1	63	24	21	112	36	7	20	538
23	287	7	33	80	9	27	80	18	60	5	32	104	742
24	176	108	3	4	8	16	47	106	49	17	8	187	729
25	308	31	15	95	28	34	94	159	56	5	46	333	1204
26	239	4	15	4	23	50	46	70	88	41	3	90	673
27	307	24	3	43	15	32	7	67	65	51	41	71	726
28	239	11	4	43	10	6	26	14	49	41	10	13	466
29	166		20	7	11	11	32	3	6	24	32	11	323
30	278		4	36	22	10	27		54	77	44	104	656
31	336		7		13		17	83		32		192	680
Total	4399	4177	470	1102	485	995	1501	2133	1920	1530	1119	1688	21519

Interesting that there are still some "holes" in there after 8 or so years.

Photos by Time of Day

2008-01-22T23:13:00.000+11:00

Another "heat-map", photos by time of day for each day of the week:

	12am	1	2	3	4	5	6	7	8	9	10	11	12pm	1	2	3	4	5	6	7	8	9	10	11	Total
Mon																									2449
Tue																									2828
Wed																									2774
Thu																									2658
Fri																									2545
Sat																									4395
Sun																									3870
Total	222	138	34	12	8	18	26	328	360	617	1223	1912	1938	1826	2049	2094	1984	1978	1248	962	1376	550	389	227	21519

(Hover for a tooltip indicating the actual number of photos in each 15 minute segment.)

Photo Calendar Heatmap

2008-01-22T13:58:00.000+11:00

A "heat map" of the number of photos I've taken over the years:

Year	Jan	Feb	Mar	Apr	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec	Total
2000										274	337	62	673
2001	237	193	242	100	103	323	972	1335	1214	1053	475	347	6594
2002	270	310	202	125	70	113	19	191	120	18	84	102	1624
2003	1427	810	26	377	45	90	45	45	106	175	39	97	3282
2004	80	38	21	94	43	71	46	138	110	38	211	257	1147
2005	190	80	24	53	58	80	95	55	81	52	43	392	1203
2006	1975	2697	141	330	178	171	48	384	98	15	102	372	6511
2007	298	248	62	129	23	154	246	466	178	179	172	127	2282
2008	165												165
Total	4642	4376	718	1208	520	1002	1471	2614	1907	1804	1463	1777	23502

Green represents the average number of photos, yellow the maximum.

Different events can clearly be seen, some of note:

June—October 2001, Travelling in Europe
January—February 2003, Travelling in the USA
January—February 2006, Roadtrip around Victoria

I'm going to have to find something to do during May...

Cover-art Wallpaper under Ubuntu

2008-01-18T13:09:00.000+11:00

This is an update to the previous post to create a cover art "stack" for Windows XP wallpaper to generate and update the wallpaper in Ubuntu.

Not much had to change, except the setWallpaper function:

def setWallpaper( bmp ):
    os.system( "gconftool-2 -t string -s /desktop/gnome/background/picture_filename %s" % bmp.replace(" ","\\ ") )

Ubuntu File System Setup

2008-01-18T12:22:00.000+11:00

The following is a rough summary of how I've setup my Ubuntu file system, including some links to resources I found useful.

I have 3 hard-drives, a 500gb drive for the system and "non-critical" files, and two 300gb drives to hold "valuable" data. "Valuable" data includes documents, photos, source-code, music and some videos, less valuable "non-critical" data includes recorded TV programs, movies and so on.

On my Windows XP system I was using a RAID-1 setup to automatically mirror my data drive to protect against hard-drive failure. Using a RAID-1 array keeps everything nice and simple, but under Ubuntu I decided that the tools were more powerful and I would be able to "manually" mirror the data to gain a little more control. In particular, while I want a copy of the data in case of hard-drive failure, a slightly delayed backup in case of catastrophic user error would also be very valuable.

So, each drive is mounted separately, and the 500gb drive is partitioned to give some space for the system and the majority for "stuff" (videos, backups etc.)

I highly recommend partitioning the drives before/during installation, as while it was possible to move everything around, it was time consuming and fiddly. I used a Gparted LiveCD to split my main drive to get a separate /stuff partition, but this took forever (about 15 hours, and my system isn't slow...). I used the GParted LiveCD because this tutorial on installing WinXP on a Linux system suggested that the version of GParted on the Ubuntu LiveCD wouldn't work properly.

This guide on moving /home to a separate partition was helpful, though I think I just used cp -a to copy the files, rather than the complicated find | cpio command..

Once I had it all up and mounted, I created a script to run rsync to mirror the drives:

#!/bin/bash
date >> /home/tom/backup_mirror_home.log
rsync --archive --delete --stats /home/ /homeMIRROR/ 2>&1 >> /home/tom/backup_mirror_home.log

This was added in as a cron job to run every 3 hours, which should be often enough (sudo crontab -e):

# m h  dom mon dow   command
0 */3 * * * /home/tom/backup_mirror_home.sh

The rsync process took a while to copy the 200gb or so across the first time, but is very fast (probably less than a minute) when just synchronizing the existing files as most data on the drive is photos and music that don't change very often.

A useful trick to keep an eye on the progress of the copy operations is watch df. df shows the current usage on the various disks, and watch will run it every 2 seconds so you can watch the percentages change slowly and be convinced something is actually happening.

Once all that was done, I setup sbackup to backup to the /stuff drive and an external USB drive.

My actual process was a little more complicated than described above, as the 300gb drives already contained NTFS partitions with the critical data on them and I wanted to move it over to ext3 while at no point having less than 2 copies of the data. The steps were roughly as follows:

Mount ntfs drive (/dev/sdb), copy data to 500gb drive (/dev/sda)
Format /dev/sdb as ext3, copy data from /dev/sda to /dev/sdb.
Move /home to /dev/sdb, and reorganise it a bit (rename/move folders/files about.)
Format /dev/sdc as ext3, mirror from /dev/sdb

Installing Ubuntu was easy!

2007-12-04T12:06:00.000+11:00

I just got myself a new computer and started running Ubuntu on it, I'm astonished at how easy the whole process of installation was. Over the years I've installed Windows countless times, always a painful process, though I haven't installed Vista so maybe they finally figured it out.

So, I built the computer, burnt an Ubuntu live CD, put it in and the first suprise was that I hadn't stuffed anything up putting the machine together. The live CD is really functional, and I was able to start up Pidgin, browse the web and most usefully, remote desktop into my old PC.

I only have one keyboard/mouse and monitor and two machines, both of which I want to continue to use for a while as I'm expecting that transitioning from my old system to the new one will take some time as the machine is used for development, as a full media center as well as web browsing etc. and I don't know how long it will take to learn how to do all that under Linux. Being able to remote desktop back to the old machine solves the problem nicely.

After playing with Ubuntu a bit and chatting with friends for a while I clicked "Install". A bit of deliberation was required to determine the partition layout I want, as in the future I expect I will install XP and dual boot. Finding an article on installing XP after Linux set my mind at ease so I figured I'd be able to repartition as necessary later. So I chose all the default install options and that plugged away in the background for a little while.

The truly impressive bit was that while this was happening, I was able to continue chatting with friends to get advice on installation options, search the web for answers and so on. It only required a quick restart and I was all done, and I didn't really stop "working" the whole time.

I did hit two small snags:

During the restart I realized I'd "boxed" in the CD drive so had to hastily hit the power switch, pull of the front bezel and open a slot so it could eject... my bad :)
Installing Opera didn't go very smoothly, I tried using Add/Remove programs but it failed mysteriously with the following error:
Downloading the .deb file from the Opera website failed the first time too, but when I ran it again it worked, so there may be something broken with the way the dependencies went on and once they'd be put on by previous attempts it succeeded.

I guess I probably won't have to reinstall Ubuntu as often as I did Windows, but at least I won't dread the process.

Now Playing Album Cover Art Desktop Wallpaper

2007-11-19T15:58:00.000+11:00

A little Python program I wrote to update my Windows desktop wallpaper with the album cover art of each album I play. The idea is to represent a stack of CDs with the latest played one on top. This concept could easily be extended to other images, a random photo each day springs to mind.

The details of how this script is run when an album is played and where the album cover art comes from aren't really relevant, and aren't going to work outside my system anyway, but the concept can be used easily enough.

Windows requires a little messing about to change the wallpaper dynamically, and this will only work with BMP files:

def setWallpaper( bmp ):
   import win32api, win32con, win32gui
   k = win32api.RegOpenKeyEx(win32con.HKEY_CURRENT_USER,"Control
Panel\\Desktop",0,win32con.KEY_SET_VALUE)
   win32api.RegSetValueEx(k, "WallpaperStyle", 0, win32con.REG_SZ, "0")
   win32api.RegSetValueEx(k, "TileWallpaper", 0, win32con.REG_SZ, "0")
   win32gui.SystemParametersInfo(win32con.SPI_SETDESKWALLPAPER, bmp, 1+2)

The following functions generate the wallpaper. The main function takes the filename of the wallpaper file, and the name of the new cover image, which will be pasted over the top of the existing wallpaper.

# read bmp in old_filename, "paste" current_filename over the top 
# and write out to new_filename (usually the same as old_filename)
import sys, os, urllib, math, Image, random
def makeBackgroundCollage(old_filename, current_filename, new_filename):
   if os.path.exists( old_filename ):
      new = Image.open( old_filename )
   else:
      new = Image.new( "RGB", (1280, 1024), (58, 110, 165) )
   if os.path.exists( current_filename ):
      current = Image.open( current_filename)
      if max(current.size) < 300:
         current = current.resize( (current.size[0]*2, current.size[1]*2) )
      if max(current.size) > 600:
         current = current.resize( (600, 600 * current.size[1]/current.size[0]) )
      border = Image.new("RGB", (current.size[0]+10, current.size[1]+10), 0xffffff)
      shadow = Image.new("RGB", border.size, 0x000000)
      angle = int(random.betavariate(2,2) * 50 - 25)
      (shadow, shadow_mask) = rotate2(shadow, border.size, angle, 128)
      (current, current_mask) = rotate2(current, border.size, angle)
      (border, border_mask) = rotate2(border, border.size, angle)
      pos = (int(random.betavariate(2,2) * (new.size[0] - border.size[0])),
             int(random.betavariate(2,2) * (new.size[1] - border.size[1])))
      new.paste(shadow, (pos[0]+5, pos[1]+5), shadow_mask)
      new.paste(border, pos, border_mask)
      new.paste(current, pos, current_mask)
   new.save( new_filename )

def rotate2(img, box, angle, alpha=255):
   img2 = Image.new(img.mode, boundingBox(box, angle * math.pi/180.0), 0)
   img2.paste(img, ( img2.size[0]/2 - img.size[0]/2,
                     img2.size[1]/2 - img.size[1]/2 ) )
   mask2 = Image.new("L", img2.size, 0)
   mask2.paste(Image.new("L", img.size, alpha), ( img2.size[0]/2 - img.size[0]/2,
                                                  img2.size[1]/2 - img.size[1]/2 ) )
   return (img2.rotate( angle, Image.BICUBIC ),
           mask2.rotate( angle, Image.BICUBIC ))

def boundingBox(box, angle):
   (x,y) = (box[0]/2.0, box[1]/2.0)
   (r, a) = (math.sqrt(x*x+y*y), math.atan2(y, x))
   (x1, y1) = (r * math.cos( a+angle ), r * math.sin( a+angle ) )
   (x2, y2) = (r * math.cos( -a+angle ), r * math.sin( -a+angle ) )
   return (int(math.ceil(max(abs(x1),abs(x2))*2)),
           int(math.ceil(max(abs(y1),abs(y2))*2)))

The position and rotation of the album is chosen using a beta distribution. This gives a nice distribution weighted towards the center of the screen, borders and a "drop shadow" are added to enhance the "stack" effect.

Picaxe breadboard adapter

2007-07-31T18:46:00.000+10:00

A quick project to simplify Picaxe prototyping on my breadboard. The idea is to house the Picaxe chip (either an 08M or 14M) and "standard" serial interfacing circuit on a "breakout" board that can be easily plugged into the breadboard while providing easy access to the chip's pins.

power input, about 5V, will be regulated by LM7805
3.5mm stereo serial socket
serial mode selector, switches serial in between "pin 5" (program mode) and "pin 3" (control mode)
power switch
jumpers to allow isolating standard serial circuit to "override" via breadboard
14 pin socket
header pins to provide 5V and 0V to "power rails" of breadboard
header pins to connect Picaxe to breadboard

serial schematic

power schematic

For some unknown reason, I had to add an extra 10k resistor between leg 2 and ground (soldered on the underside of the pcb, not shown on schematic.)

The board works pretty much as I'd hoped. Unfortunately it is quite difficult to insert/remove from the breadboard so a future version would only have a single row of header pins. Another addition would be a DC socket to allow powering from a "wall wart" and I'd improve the layout a little to allow easier access to the switches.

Handlebar "Remote" for LED Bike Light

2007-07-24T15:40:00.000+10:00

Changing modes and using the high-beam was awkward using the switches mounted on the original LED bikelight box, so I added a "remote" within easy reach of my thumb. This will make the high-beam more useful when approaching treacherous areas of paths that require more light but are exactly the places where you don't want to take your hands off the handlebars.

This uses a "rocker" navigation switch to change modes, pushing it switches on the high beam. The mode-up and down buttons are simply wired in in parallel with the existing mode rocker switch, however the high-beam was a bit trickier and uses a BD681 darlington pair transitor to "pull" the control signal to the BuckPuck to ground. This has the same effect as the existing NC push-button switch.

This was necessary because all three switches on the navigation switch share a common wire, the Picaxe software and circuit is expecting the switches to provide +5v, so it was easier to make the high-beam use a transistor to cut-out the control signal than rework the remainder of the circuit to pull the Picaxe pins to ground when the switches are pushed. If I was building this again, using ground as the common wire would be much simpler.

There was enough room on the existing pcb to solder in the transistor, resistor and wires from the navigation switch. That's one advantage of using proto-board over a printed pcb.

One further mod would be to have a capacitor attached to the BD681 some how so that a single push holds the high-beam on for say 5 seconds, but I couldn't figure out how to get that to work. Alternatively, putting in a flip-flop to toggle the high-beam on/off would work too, but there was definitely not enough space on the pcb for that option.

Ambient Email Notifier

2007-07-06T22:41:00.002+10:00

Small project to provide email notification via an RGB LED hooked up to the USB port.

Uses a 4d-micro-USB module to provide power and RS232 to a Picaxe 08M controlling an RGB LED. Can control colour (seven colours: work email in red, newsletters in green, other in blue) and brightness by sending characters to the COM port, this is done by a Python script run every 10 minutes by Task Scheduler with an accompanying system tray icon to allow LED to be turned off again (see python gmail check in four lines of code, pySerial and pySystray).

The schematic is as simple as possible:

The relevant code (python and picaxe basic) is pretty self-evident, I will provide it if anyone is interested. Update: Another post has some code.

CCTV Filmmaking

2007-05-30T17:16:00.000+10:00

BBC article about a film, Faceless, made entirely using CCTV footage. Pretty neat, the manifesto states you can't take your own camera or lighting and the protaganist must appear in every shot to allow them to obtain the footage via freedom of information/privacy legislation.

Wow, fixed already...

2007-05-27T19:52:00.000+10:00

Interesting article about the reconstruction of the freeway in San Francisco that collapsed after a truck fire. No, really.

LED bike light project

2007-05-19T19:15:00.000+10:00

Design

The goal was to create a high power LED bike light (some similar projects: a 3 LED mountain bike light, a whole series of evolving light designs or this simple and neat 3 LED light).

After discovering the BuckPuck LED driver has a 5V output and a dimmer control circuit, I decided to control it with a Picaxe microcontroller. The primary purpose of the microcontroller was to provide a battery level indicator of some sort. Secondary features would be to allow multiple brightness modes etc.

I couldn't find a simple battery level circuit or simple way to calculate the remaining capacity of a battery so decided instead to simply use the Picaxe to time the amount of time the light has been switched on and use this to determine when a recharge is required.

The aim is to discharge the batteries about 30% between each recharge. The circuit should draw a maximum of about 1A for the LED, and say 0.2A for the Picaxe and bargraph LEDs. Using 2400mAH batteries, and 100% efficiency, would give 2 hrs at 100% discharge. However the light will rarely be on at full power; assuming it's on at 30% power, a 30% discharge will therefore take about 2 hours... or so I'm guessing, it's much more complicated than that, but that'll do for now.

The controller will require two buttons to change up and down through the brightness modes, and one to test the battery level (i.e. display the timer value.) Two buttons are required to avoid having to cycle through all the modes to get back to the start, like all the stupid flashing LED tail lights require...

The pins for the buttons plus one output for the LED controller and one to drive the LED bargraph uses all 5 input/output pins on the Picaxe 08M.

One further feature was added, which is a "high beam" button. This will override the current mode from the Picaxe and put the LED on full power. This will be used to briefly illuminate dark parts of paths and to flash at cars that might not have seen me or have cut in front of me or something like that. Fortunately, this button does not need to be connected to the Picaxe, as there are no free pins.

Schematic

The Picaxe usese 3 standard input circuits. For the outputs, Pin2 is used for PWM output in the background to drive the LED controller. This will allow different brightnesses to be set by varying the duty cycle.

To drive the LED bargraph, the basic reference circuit from the LM3914 datasheet is used to measure the voltage over a resistor/capacitor circuit. The capacitor is charged via a PWM output from the picaxe. Varying the duty cycle allows the capacitor to charge to different levels which is reflected in the bargraph.

I added a fuse between the battery and the BuckPuck too, just in case. No idea if it's really needed...

Parts list

Components
1	Cree XR-E 3W LED + 8° Lens + BuckPuck 3023-D-E-1000 Driver (datasheet)	(Cutter Electronics)
1	Picaxe 08M microcontroller	(Microzed)
1	LM3914 LED bargraph driver (datasheet)	(DSE)
1	10 seg LED bargraph	(DSE)
3	1k resistors
4	10k resistors
1	1k2 resistors
1	3k6 resistors
1	0k2 resistors
1	10uF 25V electrolitic capacitor
Hardware
1	Plastic "Jiffy" box (30x54x83mm)	(DSE)
1	Veroboard (24 tracks x 18 holes)	(DSE)
1	Heatsink	(Jaycar)
1	3.5mm mono plug + socket	(Jaycar)
1	SPDT rocker switch (on-off-on)	(Jaycar)
1	Push button (N/O)	(DSE)
1	Push button (N/C)	(DSE)
1	Fuse + Holder (M205 2A)	(Radioparts)
1	8 Pin DIL socket	(DSE)
1	18 Pin DIL socket	(DSE)
1	20 Pin DIL socket (machine pin strip)	(DSE)
2	4 AA battery holder	(Jaycar)
2	9v Battery snaps	(DSE)
8	AA NiMH batteries (2400mAH)	(Jaycar)
2	velcro straps 12.5cm of hook + 12.5cm fuzz	(Spotlight)
1	inner tube rubber
Materials
1	Arctic Silver Thermal Adhesive	(Mittoni)

Note: The above suppliers are not necessarily where I got the part, just the first link I could find...

Software

The above circuit was first layed out on a breadboard with a regular LED in place of the 3W Cree LED and using 4 AA batteries directly rather than via the BuckPuck.

Note that when I got the BuckPuck and LED, the 4 AA batteries were not sufficient to provide enough voltage from the Ref pin to switch the LED off. This was fixed by using all 8 AA batteries.

LED bargraph output calibration

After realising I had no real idea how to get the LM3914 input range to match the output from the Picaxe, or how to choose the right resistor/capacitor combination, it occurred to me that it didn't matter and the picaxe could be calibrated to output voltages that lit up the bargraph LEDs the right amount. Hooray for software!

The values output by the following program when each of the 10 LEDs lit up were written down and used in the final program.

symbol x = b0
symbol y = b1
symbol dot = b2

main: 
 for x = 0 to 10
  dot = x
  gosub display_dot
 next x
 for x = 10 to 0 step -1
  dot = x
  gosub display_dot
 next x
 goto main


display_dot:
 lookup dot, (0, 89, 134, 156, 169, 177, 184, 190, 198, 207, 214), y
 
 pwm 1, y, 100
 
' pause 500
 
 return

Controller program

' 252 bytes program + 2 bytes data

' pins
symbol led_pin = 2
symbol dot_pin = 0
symbol test_btn = pin3
symbol modeUp_btn = pin4
symbol modeDn_btn = pin1

' variables
symbol mode = b0
symbol time = w4
symbol time_counter = w6

symbol dot = b1
symbol dot_duty = b2
symbol dot_cycles = b3

symbol led_period = b4
symbol led_duty = w5


' constants
symbol minMode = 0
symbol maxMode = 10
symbol flashMode = 9
symbol maxTime = 1200 ' minutes * 10
symbol dotTime = maxTime / 10
symbol time_interval = 1122 ' roughly 11215 ticks per minute

' data
eeprom 0, (0, 0) ' time as a word

' initialize
high led_pin

mode = 0
read 0, word time

' wakeup
gosub strobe

main:
 dot_cycles = 50

 time_counter = time_counter + 1
 if time_counter > time_interval then
  ' clock tick (4 bytes)
  high dot_pin
  high dot_pin
  low dot_pin

  time = time + 1
  goto write_time ' return to main
 endif

 if modeUp_btn = 1 and mode < maxMode then
  mode = mode + 1
  goto show_mode ' return to main
 endif
 if modeDn_btn = 1 and mode > minMode then
  mode = mode - 1
  goto show_mode ' return to main
 endif
 
 if test_btn = 1 then
  dot = time / dotTime max 10
  dot_cycles = 100
  gosub do_dot
  ' hold to reset (8 bytes per delay loop)
  if test_btn = 1 then
   gosub do_dot
   if test_btn = 1 then
    gosub strobe

    time = 0
    goto write_time ' return to main
   endif
  endif
 endif

 if mode >= flashMode then
  gosub do_mode
 endif

 goto main

write_time:
 write 0, word time
 time_counter = 0
 goto main

show_mode:
 gosub do_mode
 dot = mode
 gosub do_dot
 goto main

do_mode: ' high - led off, low - led on
 led_period = 255
 'led_duty = 4 - mode * 250
 lookup mode, (0, 1010, 980, 950, 900, 800, 600), led_duty
 select mode
  case 0 ' off
   led_period = 0
   high led_pin
'  case 1 ' dim 1
'   led_duty = 750
'  case 2 ' dim 2
'   led_duty = 500
'  case 3 ' dim 3
'   led_duty = 250
  case 8 ' full
   led_period = 0
   low led_pin
  case 9 ' dim short flash
   led_period = 0
   'low led_pin
   'pause 50
   pwm led_pin, 150, 10
   high led_pin
   pause 350
  case 10 ' long flash (17 bytes)
   led_period = 0
   low led_pin
   pause 100
   high led_pin
   pause 300
 endselect

 pwmout led_pin, led_period, led_duty
 
 return

do_dot:
 lookup dot, (0, 89, 134, 156, 169, 177, 184, 190, 198, 207, 214), dot_duty
 
 pwm dot_pin, dot_duty, dot_cycles
 low dot_pin
 
 return

strobe:
 dot_cycles = 10
 for dot = 1 to 10
  gosub do_dot
 next dot
 return

Some points to note:

The time the light has been running in "minutes" is stored in the EEPROM memory of the Picaxe.
Pressing the "test" button will display the current time on the bargraph using 1 LED for each 12 minutes (120 minutes total).
Holding the "test" button for a few seconds will reset the timer. This will be used when freshly charged batteries are inserted.
There are 8 brightness modes (plus "off") and two flash modes.

Timer calibration

The above controller flashes the bargraph briefly each 6 seconds or so. A few minutes worth of these flashes were counted and timed and the program adjusted so that the bargraph will reach 10 after approximately 120 minutes.

Construction

First, all the required holes were drilled in the box, and the aluminium lid was bent into shape to hold the LED and heatsink. The box lid was also shaped to fit the aluminium lid sticking out the side.

The spacers inside the box were cut down to lower the circuit board so there was enough room for the LED bargraph to stick out the top and the switches and BuckPuck to fit underneath.

The wires from the BuckPuck were soldered to the LED (with an extra wire joined to LED -ve output to become the ground for the circuit board.)

The LED was glued to the heatsink using the Arctic Silver epoxy, and when it was dry the heatsink and LED were glued to the aluminium lid.

The first version of the battery pack was put together at this stage too. This failed as the battery holders were made of Delrin which is ungluable... So, I drilled tiny holes in the two cases and "sewed" them together with fine jewellery or tie wire. An on/off switch was soldered to the battery wires and the socket assembled.

The circuit board layout was sketched out, the board cut to size and the tracks between the IC sockets cut. The sockets were then soldered in and the the resistors and capacitor soldered in too. The chopped off legs from the resistors were used as bridges between the +ve LED bargraph pins and the ground 10k resistor line. Some legs were also used to form some jumpers, with soldered in for others.

Next, I measured out the wire and soldered to the switches etc. and soldered in the BuckPuck and circuit board.

At this point, the circuit was tested and found to work (incredibly!) so the bottom of the circuit board was insulated with some electrical tape and the whole lot squeezed into the box.

A thin strip of inner tube rubber was added as a gasket on top of the aluminium lid to seal it in case of rain (which wasn't really considered in the design...)

For the first test ride, the whole lot was attached to my bike with rubber bands and the batteries were lashed to the frame with an inner tube! I was very happy with the results so started working out a more permanent mounting mechanism.

I removed the socket from my old Cateye light and bolted/Araldyted it to the box, which took care of mounting the light itself. It is adjustable/tiltable too which is a nice bonus.

Initially the batteries were mounted using an elastic strap, but this proved to be a little difficult to attach/remove with less than 3 hands and I dropped it breaking the damn thing. Because the battery holder can't be glued I had to start again with a different construction method.

This time, the battery holders were "sewn" together using jewellery wire again and the whole lot held in a cradle made from inner tube rubber (helpfully sewn by Alison) that is velcroed to the bike.

This looks quite professional and is a lot easier to take on and off!

Results

I'm extremly happy with the results, the LED is way too bright to look at and lights the road extremely well when on "full". Under normal conditions "mode 3" was quite sufficient to see the road.

The following image was taken by shining the light onto my door from about 3m away and photographing at 1/15 sec, f2.8, ISO100. The door has a slightly gloss paint, so the middle of the beam is being reflected back.

Some minor changes I would make:

The "high beam" button really neads to be accessible without moving your hand to the light. I'm not sure how best to achieve this.
A switch on the battery box would be useful (this was initially planned but died with the first battery box.)
A "console light" to light up my speedo.
Waterproof design!
A future version may well use a proper battery capacity meter (such as this one for lead acid batteries or this one for R/C cars.)

What if It's All Been a Big Fat Lie?

2007-04-30T22:56:00.000+10:00

New York Times piece from 2002 questioning the received "fat is bad, mkay?" wisdom. The first 8 pages are good, then it starts promoting the Atkins diet a little too specifically, which given that the main message I take out of the whole article is "there's no silver bullet" seems a little short sighted...

You Are What You Grow

2007-04-25T13:48:00.000+10:00

Michael Pollan in the New York Times with a glimmer of hope that the US farm bill could get a run for it's money this time around...

Humans hot, sweaty, natural-born runners

2007-04-25T13:44:00.000+10:00

Interesting article on the evolution of long distance running as a hunting strategy for early hominids.

Recovering Accidentally Deleted Files with Undelete Plus

2007-02-13T15:01:00.000+11:00

After accidentally deleting about 100gb of recorded TV programs,
I spent a bit of time looking around for a utility that would
let me recover them easily (God knows why this isn't built into Windows...)

After trying a lot of utilities, I eventually found Undelete PLUS. This scanned and found all the deleted files very quickly and I was able to copy them to another drive.

The program's UI is not particularly intuitive, and a general flaw with all these programs is that you must copy the recovered files to a separate disk to avoid destroying other yet-to-be recovered files (which is a pain with 100gb of files...)

However, it's free, and it works! Download or bookmark it in case you need it one day.

(I'm using Windows XP SP2 with NTFS formatted drives.)

Restaurant reviewer with an ethical stance

2007-01-15T22:03:00.000+11:00

From now on, if a restaurant does not offer me tap water, politely, unsarcastically, and before they offer mineral water, then they will be penalised.

Scaring children to death

2006-10-31T13:41:00.000+11:00

Robyn Williams, In Conversation with Bill Durodie, 28th September:

"Going back to what you're saying about risk, I was fascinated the other day to hear something on the BBC concerning the fear of paedophiles; occasionally of course, it's a real, real thing that there is a nasty man in a coat and there is a poor vulnerable child. However for every one of those – apparently now that everyone drives their kids to school, there are 300 fatal accidents – so for each one child that you're saving from the paedophile, 300 are dying in car crashes."

I'd love to find a reference for that...

Things will get better!

2006-10-16T21:28:00.000+10:00

Well, maybe that's not all that positive... treehugger post.

Update: on World Changing.

The Onion: Retired S1Ws Recalled To Active Duty

2006-10-07T10:43:00.000+10:00

Too good.

Well put...

2006-10-07T10:12:00.000+10:00

An article by Emma Tom on the inconsistent imposition of "Australian values" on different religions.

Nice quote from a post 9-11 article in The Onion:

"Look, I don't know, maybe I haven't made myself completely clear, so for the record, here it is again," God was quoted as saying. "Somehow, people keep coming up with the idea that I want them to kill their neighbour. Well, I don't. And to be honest, I'm really getting sick and tired of it. Get it straight. Not only do I not want anybody to kill anyone, but I specifically commanded you not to, in really simple terms that anybody ought to be able to understand."

24hrs of searching

2006-09-01T23:25:00.000+10:00

roentgenizdat

2006-08-31T16:58:00.000+10:00

Ruby and Ocaml

2006-08-18T23:21:00.000+10:00

Not sure whether I'll learn either of these properly, but so far Ruby is winning simply because of this book:

Why's (Poignant) Guide to Ruby

Fox #1: Have you noticed this book is basically written by a lunatic?
Fox #2:Yup.

The licensing for the book also permits it's use for cigarette advertising, which may come in handy...

The other (slightly more distant) contender is Objective CAML.

Dabblers and Blowhards

2006-08-18T14:11:00.000+10:00

An interesting counterpoint to Hackers and Painters. I'm not entirely convinced, but it does make quite a few fair points.

Death to Caps Lock

2006-08-17T19:36:00.000+10:00

Whoa... JavaScript just got scary!

2006-08-14T22:54:00.000+10:00

A very clever technique to scan any network using JavaScript (as designed, no bugs required...) And for the pdf challenged.

President George W. Bush vs Governor George W. Bush on The Daily Show

2006-08-12T22:20:00.000+10:00

"Strong words from two very different men"!

Smart Pig!

2006-08-10T13:04:00.000+10:00

No more eating pork... An interview with Sy Montgomery about her pig and book The Good Good Pig.

The Meatrix

2006-08-03T10:10:00.000+10:00

Also reading The Omnivores Dilemma, which is very good.

Travel-time Maps of the UK

2006-05-26T21:30:00.000+10:00

Cool maps showing the relative travel times for getting from one part of the UK to another. Just need one for Melbourne now...

THEY'RE MADE OUT OF MEAT by Terry Bisson

2006-05-15T22:04:00.000+10:00

Mmmm, nutella!

2006-05-01T21:01:00.000+10:00

rrrrrrrrrrrrrrRRRRRRR!!!

2006-04-28T18:28:00.000+10:00

An awesome voice-controlled blender!

Computing power!

2006-04-27T21:43:00.000+10:00

It takes a pound of coal to create, package, store, and move every 10-20 megabytes of data...

Where's that taxi?

2006-04-14T23:09:00.000+10:00

A cool project for San Francisco's Exploratorium graphically mapping the movements of taxis.

Wind Turbines and Birds

2006-04-06T22:32:00.000+10:00

A comparison of bird deaths by collision with wind turbines and other man made structures. Summary: We already kill so many birds that any additional deaths as a result of wind turbines are pretty much irrelevant.

NEJM: Should doctors assist with executions?

2006-03-30T22:50:00.000+11:00

Another argument, as if one was needed, as to why the death penalty can't be made to work. Chilling stuff.

Idlewords.com: The Collapse of the Perito Moreno

2006-03-26T20:40:00.000+11:00

"A glacier forms when more snow falls on it than can melt in a given year. This implies the existence of a point (let us call it point A) where so much snow is falling that it can make up for the spectacular daily losses at the glacier's face (point B). There is also a third point, point C, where it is warm and dry and you can sit at an outdoor table drinking beer in the sun. However, the physics of glacier formation ensure that points A and B are very close together, and point C, if it exists at all, is very far away, most likely in El Calafate, eighty kilometers to the east."

Probabilities in the Game of Monopoly®

2006-02-27T21:57:00.000+11:00

Get out of (voice)jail, free

2006-02-27T21:51:00.000+11:00

gethuman.com has a database of codes to transfer directly to an operator! They also have (globally) useful tips. (This card may be kept until needed or sold.)

Playing Flickr?

2006-01-10T21:40:00.000+11:00

Johann Hari - Friends, Romans, Countrymen. Lend me your ears about ancient Rome...

2005-12-24T16:32:00.000+11:00

Johann Hari - 'The Power and the Idealists' by Paul Berman a review

2005-12-24T16:27:00.000+11:00

PARK(ing)

2005-12-17T21:42:00.000+11:00

FlickrFS

2005-11-08T19:33:00.000+11:00

This is brilliant.

Republic Dogs

2005-10-30T21:55:00.000+11:00

Paul Graham at OSCON

2005-10-30T21:02:00.000+11:00

An excellent talk (or essay.)

I/O Brush

2005-10-29T23:04:00.000+10:00

This is awesome, taking the clone tool to a whole other level!

The Clock of the Long Now

2005-10-20T22:28:00.000+10:00

Oak Island Mystery

2005-10-20T22:22:00.000+10:00

Speech Bubble Stickers

2005-09-20T19:22:00.000+10:00

worldometers.info

2005-09-01T21:53:00.000+10:00

A Collection of Word Oddities and Trivia

2005-08-23T21:43:00.000+10:00

WorldChanging: Another World Is Here: Quake III, Now With GPL Goodness

2005-08-23T20:42:00.000+10:00

WorldChanging: Another World Is Here: Alonovo: Shopping for a Better World

2005-08-23T20:40:00.000+10:00

www.zoomquilt.org

2005-07-31T14:42:00.000+10:00

Wired News: A Hacker Games the Hotel

2005-07-31T14:06:00.000+10:00

WORLDPROCESSOR Catalog

2005-07-30T21:26:00.000+10:00

I want one...

Boing Boing: Microsoft "Genuine Advantage" cracked in 24h: window.g_sDisableWGACheck='all'

2005-07-30T20:57:00.000+10:00

Brilliant. Why validation of data must happen at the server and the browser...

Introducing Slate Audio Tours - The commentary museums don't want you to hear. By Andy Bowers

2005-07-30T12:16:00.000+10:00

Places & Spaces > You are not here

2005-07-28T21:46:00.000+10:00

Just in case you thought the world was a rich place, point to where you live...

Wired News: Blind Teen Gamer Obliterates Foes

2005-07-28T21:40:00.000+10:00

"'I freak people out by playing facing backwards.'"

Something Awful: The Mars Volta

2005-07-18T22:35:00.000+10:00

"Some people think it’s dull, overwrought, pretentious crap, but some people think it’s a work of prog-rock genius. I would contend that there’s no difference between the two."

And about covers:

"In the hierarchy of artistic credibility, the Ironic Punk Cover falls somewhere just below erotic Harry Potter fan-fiction and just above Anne Geddes photography."

Wired News: Google Growth Yields Privacy Fear

2005-07-18T21:20:00.000+10:00

Wired News: ITunes Mints Podcasting Stars

2005-07-18T20:58:00.000+10:00

Sprol.com » Gunns, Ltd Woodchipping Old Growth Forests

2005-07-13T21:35:00.000+10:00

Images from Google Earth show very clearly what clear-felling looks like.

MICHAEL LIGHT | 100 SUNS

2005-07-12T22:53:00.000+10:00

Photos from the USA's atomic tests.