IBM Taps Hadoop for Social Media Analytics, Funds ‘Big Data’ Research
Updated · May 23, 2011
IBM (NYSE: IBM) has unveiled news software and services to address “Big Data” challenges like mining massive amounts of social media data for business intelligence.
The new software and services are part of Big Blue’s Big Data Initiative that will enable organizations to integrate and analyze tens of petabytes of data in its native format and gain critical intelligence in sub-second response times.
The new analytics capabilities pioneered by IBM Research will enable CIOs to construct specific, fact-based financial and business models for their IT operations, according to Big Blue.
“The volume and velocity of information is generated at a record pace,” said Steve Mills, senior vice president and group executive for IBM Software and Systems. “This is magnified by new forms of data coming from social networking and the explosion of mobile devices. Through our extensive capabilities in business and technology expertise, IBM is best positioned to help clients not only extract meaningful insight, but enable them respond at the same rate at which the data arrives.”
IBM Big Data Software Taps into Hadoop
In its announcement, IBM said it is making available new InfoSphere BigInsights and Streams software that allows clients to gain fast insight into information flowing in and around their businesses.
BigInsights software is the result of a four-year effort of more than 200 IBM Research scientists and is powered by the open source technology Apache Hadoop. The software provides a framework for large-scale parallel processing and scalable storage for terabyte to petabyte-level data. It incorporates Watson-like technologies, including unstructured text analytics and indexing that allows users to analyze rapidly changing data formats and types on the fly.
IBM InfoSphere Streams software analyzes data coming into an organization and monitors it for any changes that may signify a new pattern or trend in real-time. This capability helps organizations to capture insights and make decisions with more precision, providing an opportunity to respond to events as they happen.
New advancements to Streams software make it possible to analyze Big Data such as Tweets, blog posts, video frames, EKGs, GPS, and sensor and stock market data up to 350 percent faster than before, IBM says. BigInsights complements Streams by applying analytics to an organization’s historical data as well as data flowing through Streams.
According to IBM, the analytics cycle becomes more powerful as more data and real-time analytic results are available to be modeled for improvement.
IBM’s portfolio of Hadoop-based offerings also include IBM Cognos Consumer Insight, which integrates social media content with traditional business analytics, and IBM Coremetrics Explore, which segments consumer buying patterns and drills down into mobile data. Additionally, Hadoop is the software framework the IBM Watson computing system uses for distributing the workload for processing information, which supports the systems breakthrough ability to understand natural language and provide specific answers to questions at rapid speeds.
IBM to Invest $100 Million in Massive Scale Analytics Research
IBM also announced a $100 million investment for continued research on technologies and services that will enable clients to manage and exploit data as it continues to grow in diversity, speed and volume. The initiative will focus on research to drive the future of massive scale analytics, through advancing software, systems and services capabilities.
Over the past five years, IBM has invested more than $14 billion in 24 analytics acquisitions. Today, more than 8,000 IBM business consultants are dedicated to analytics and over 200 mathematicians are developing breakthrough algorithms inside IBM Research.
For more on social media analytics, see Social Business Intelligence – What Is It, and Do You Need It?