Saturday, May 31, 2008

Book reviews

For the past few months, every time I read a book, I put a review up on the goodreads website. I now have dozens of reviews. I'd especially encourage you to read what I wrote about Erewhon.

Thursday, May 29, 2008

Sparse codes, part two

The explanation in the previous post was maybe a little confusing. I'll try again with a different analogy.
We're trying to understand how neurons in the brain store memories. In this analogy, each neuron corresponds to one key on a piano. One possibility is that each neuron stores one memory. So playing one note would bring to mind one memory. Obviously, I could only store 88 total memories this way.
Another possibility is that every possible combination of notes brings to mind one memory. This could allow for a huge number of memories-- 2 to the 88th power -- but it would be very tricky to bring to mind any particular memory because I'd have to know whether each of the 88 keys should be up or down.
The third possibility is that each memory is associated with a chord. I have only 10 fingers, so there are about 88 to the 10th power different chords I can play (assuming my fingers can stretch that far!) It's still plenty of possible memories I can play, but I only have to keep track of the position of 10 fingers, rather than all 88 keys. This is called a sparse coding (because the chords form a kind of secret code for the memories) and it's probably the system the brain actually uses.

Wednesday, May 28, 2008

Sparse codes, fractional bases, neural representations


Recently my brothers and I were discussing this comic. Is it really possible to represent any value with a single 1 and a lot of zeros? Of course it is; this is just a variation on base 1 notation. We don't usually use base 1 for anything except counting on our fingers because the number of places required is so large.
We usually use base 2 in computers, because it is the densest base you can use that only uses the symbols 1 and 0. But this denseness comes with a price: in order to use all the possible states, the mapping between each individual symbol and it's meaning is lost. With base 1, each symbol stands for one specific thing. With base 2, in order to pick out one specific thing, you need to specify every single symbol.
What would be nice is some kind of code that is somewhere in between base 1 and base 2. (Perhaps a fractional base like 1.3 or base Phi.) For example, you could have one symbol mean "red" when activated, one symbol mean "round," and one symbol mean "tasty." Then 111 would represent "apple" while 110 would represent "tomato." Of course, in a real system there would be a lot of other meanings ("smart," "fast," "salty," etc...) that wouldn't be used in this example, corresponding to extra zeros in the code. It would be a sparse code, with a lot of zeros and very few ones. In fact, if it took no energy to represent a zero, but some energy to represent a one, this would be the sort of code you would want to use.

That's why it's the system the brain appears to use to store information, and is becoming widely used for machine learning tasks. In fact, the brain's representation may be very sparse. Some neurons seem to fire only when shown a certain person's face. (And another paper on the subject.)
The image shown at the top is a sparse code for representing a small patch of an image for image compression. The patch would be the sum of the activated elements from the code.

Wednesday, May 21, 2008

Monte Roraima


At the point where Brazil, Venezuela, and Guyana meet, stands a vast plateau. The Pemon Indians of the Gran Sabana say that it is the stump of a world-size tree that held all the fruits of the world. It is surrounded on all sides by steep rock cliffs, except for a single narrow approach. Water pours over these cliffs, forming some of the highest waterfalls in the world.
On the top of this plateau are some of the largest sinkholes ever discovered. No one knows how exactly they were formed. The holes are so deep that it is impossible to climb into them except by lowering a rope more than a quarter mile directly down into them. These sinkholes are isolated from the surrounding animal and plant life, and have unique forms of life found no where else on earth.
The first explorers to climb to the plateau did so in 1884 with the help of the Royal Geographical Society of London. Their story was the inspiration for Arthur Conan Doyle's The Lost World.










Monday, May 19, 2008

Sunday, May 18, 2008

Shrimp and Seahorse




I was feeling inspired by Karl Sim's creatures and thinking that it would be fun to design a game around evolving underwater robots. The game designer would create robotic looking primitives (instead of cuboids and cylinders) and then just turn them loose for you to compete against for survival.
I downloaded some L-Systems code and some OpenGL code but got tired of it before I got anything decent programmed. What I really felt like doing was making concept sketches. And then I realized how hard it was to get perspective right, and decided to make them out of Legos instead.
So here are two sea creatures, a shrimp-bot and a seahorse-bot. Some other designs would include octopus, coelocanth, starfish, eels, creatures from the Devonian shale, and deep sea fish with lots of teeth. Of course the evolution algorithms would mix them all up.

Sunday, May 11, 2008