Too Much Information & How To Process?

Sadagopan's weblog on Emerging Technologies,Thoughts, Ideas,Trends and Cyberworld points to aq great article about corporations facing the drinking-out-of-the-firehose problem, driven by cheap storage and too much data to process.

"A decade ago the biggest data centers in the U.S. had 10 terabytes of storage, and there were only five or ten of them, today there are enterprises with 2 or 3 petabytes," says Gil Press, senior director , EMC. Visa, the credit card company, manages more than a petabyte, or 1,000 terabytes. EMC says one of its biggest customers, a global retailer, expects to buy 3 petabytes of capacity this year (not all from EMC). Two years ago the same company bought 300 terabytes. It's the curse of cheap storage. All that customer data is out there, and it seems a shame to throw it away. But doing something with it is almost beyond the reach of the available microprocessors and database software. How do you scroll through a spreadsheet 1 billion rows deep?

Another excerpt...

If Harrah's planned $9 billion acquisition of Caesars Entertainment goes through, Stanley says he might need to double his Teradata system to handle Caesar's volume. He may double his Teradata system next year anyway, deal or no deal. "You just have this endless desire to put more information into the warehouse and to do more with what's already in there."

Does your company feel that way about their data warehouse?