Data is a gas...

As a chemist and a computer geek, it was inevitable that I'm going to write that down at some point. My data expands to fill the space available, regardless of how much there is.

I discovered that one of the software packages I'm using to process data can generate anywhere from 200k to 17Gb of data per run. That's a frightening amount of data. I guess it has to be taken into account that it's processing a couple Megs to a couple Gigs of data to start with. Still, I remember a time when my father upgraded our 10 Megabyte hard drive to a 40Megabyte hard drive, and proclaimed we'd never be able to fill it up.

Still, this whole process is interesting. The Solexa machines are generating so much data that we have to seriously question how we're using space, and how we work. I had a great discussion this afternoon with a co-worker about how infrastructure is becoming more and more important, in a way that just wasn't there before this technology came on line.

And that ties in with the one key piece about a paradigm change: you have to re-evaluate all of your basic assumptions and "rules of thumb", because they may no longer be valid once your paradigm changing technology starts ramping up.