IBM Global Technology Services forecasts explosive growth in worldwide storage requirements in a July 2006 white paper called “The Toxic Terabyte: How Data-Dumping Threatens Business Efficiency”. The document is an interesting read and has some statistics, some of which verge on being amazing.
For example, the paper starts with the observation that in 2006, 26,000 million million transitors will be pumped out by electronics manufacturers worldwide, more they claim, than all the grains of rice grown on earth in 2006.
The report attributes the explosive growth of storage to the proliferation of computer transitors and chips that operates on Moore’s Law. Basically there are more chips running, more data being processed, and more data being stored.
The report goes on to report incredible numbers for how fast data will grow, although it doesn’t try to assign any weight to the importance to the kinds of data that’s being amassed. For example, billions of processors logging transaction statistics every millisecond might be creating massive amounts of uncompressed historical information, but that data may serve no real purpose. From the report:
It is projected that just four years from now, the world’s information base will be doubling in size every 11 hours. So rapid is the growth in the global stock of digital data that the very vocabulary used to indicate quantities has had to expand to keep pace. A decade or two ago, professional computer users and managers worked in kilobytes and megabytes. Now schoolchildren have access to laptops with tens of gigabytes of storage, and network managers have to think in terms of the terabyte (1,000 gigabytes) and the petabyte (1,000 terabytes). Beyond those lie the exabyte, zettabyte and yottabyte, each a thousand times bigger than the last.
The toxic side of having all of this data is that searching through all of it becomes ever more difficult. Even with today’s plummeting storage costs and shrinking physical size of storage units, the report suggests that some analysts are worried that ultimately we will have so much data that we won’t be able to afford or create enough storage devices to hold all of it.
[...] IBM’s? ‘Toxic Terrabyte’ predictions may be a bit over the top in estimating how quickly today’s storage demands will scale to out-of-control proportions.? A more conservative estimate, but one that still indicates robust growth in the industry, was released from The Advisory Council (TAC).? The TAC report estimates that Storage Demands will grow consistently 30-50% annually through 2010. [...]