IBM Set New Big Data Record

IBM Break Storage Performance Record

Jessica Thornsby

IBM researchers scan 10 billion data files in 43 minutes.

IBM researchers have successfully scanned 10 billion data files on a single system, in 43 minutes, according to a press release. This beats the previous record of one billion files in three hours by a factor of 37, which was also set by IBM researchers at the Supercomputing 2007 conference in Reno. This was achieved using General Parallel File System (GPFS) running on a cluster of 10 eight core systems and solid state storage.

“Today’s demonstration of GPFS scalability will pave the way for new products that address the challenges of a rapidly growing, multi-zettabyte world,” said Doug Balog, vice president of storage platforms at IBM. “This has the potential to enable much larger data environments to be unified on a single platform and dramatically reduce and simplify data management tasks such as data placement, aging, backup and migration of individual files.”

According to the press release, the size of digital data increased by 47% over the past year, putting businesses under pressure on how to manage and store the additional data.

Author
Comments
comments powered by Disqus