Big Data

Big Data:

“Small data is gone. Data is just going to get bigger and bigger and bigger, and people just have to think differently about how they manage it.” –Scott Zucker

Every year, companies are seeing dramatic increases in data coming from more and more disparate sources. According to IDC, we create 2.5 quintillion bytes of data. It is not believable but it is a fact that 90% of the data in the world today has been created in the last two years alone. As per International Data Corporation (IDC), it is imperative that organizations and IT leaders focus on the ever-increasing volume, variety and velocity of information that forms big data.

Data is generated from everywhere: social media, mobile, educational information, individual information etc. This data is big data.

Big data spans multiple dimensions:

· Volume: Many factors contribute to the increase in data volume – transaction-based data stored through the years, text data constantly streaming in from social media, increasing amounts of sensor data being collected, etc. In the past, excessive data volume created a storage issue. But with today's decreasing storage costs, other issues emerge, including how to determine relevance amidst the large volumes of data and how to create value from data that is relevant.

Enterprises are awash with ever-growing data of all types, easily amassing terabytes—even petabytes of information.

Ø Turn 12 terabytes of Tweets created each day into improved product sentiment analysis

Ø Convert 350 billion annual meter readings to better predict power consumption

· Variety: Data today comes in all types of formats– from traditional databases to hierarchical data stores created by end users and OLAP systems, to text documents, email, meter-collected data, video, audio, stock ticker data and financial transactions. By some estimates, 80 percent of an organization's data is not numeric! But it still must be included in analyses and decision making.

Sometimes 2 minutes is too late. For time-sensitive processes such as catching fraud, big data must be used as it streams into your enterprise in order to maximize its value.

Ø Scrutinize 5 million trade events created each day to identify potential fraud

Ø Analyze 500 million daily call detail records in real-time to predict customer churn faster

· Velocity: According to Gartner, velocity "means both how fast data is being produced and how fast the data must be processed to meet demand." RFID tags and smart metering are driving an increasing need to deal with torrents of data in near-real time. Reacting quickly enough to deal with velocity is a challenge to most organizations.

Big data is any type of data - structured and unstructured data such as text, sensor data, audio, video, click streams, log files and more. New insights are found when analyzing these data types together.

Ø Monitor 100’s of live video feeds from surveillance cameras to target points of interest

Ø Exploit the 80% data growth in images, video and documents to improve customer satisfaction

· Veracity: 1 in 3 business leaders don’t trust the information they use to make decisions. How can you act upon information if you don’t trust it? Establishing trust in big data presents a huge challenge as the variety and number of sources grows.

· Variability: In addition to the increasing velocities and varieties of data, data flows can be highly inconsistent with periodic peaks. Is something big trending in the social media? Perhaps there is a high-profile IPO looming. Maybe swimming with pigs in the Bahamas is suddenly the must-do vacation activity. Daily, seasonal and event-triggered peak data loads can be challenging to manage – especially with social media involved.

· Complexity: When you deal with huge volumes of data, it comes from multiple sources. It is quite an undertaking to link, match, cleanse and transform data across systems. However, it is necessary to connect and correlate relationships, hierarchies and multiple data linkages or your data can quickly spiral out of control. Data governance can help you determine how disparate data relates to common definitions and how to systematically integrate structured and unstructured data assets to produce high-quality information that is useful, appropriate and up-to-date.

Big data is more than simply a matter of size; it is an opportunity to find insights in new and emerging types of data and content, to make your business more agile, and to answer questions that were previously considered beyond your reach. Until now, there was no practical way to harvest this opportunity. Today, IBM’s platform for big data uses state of the art technologies including patented advanced analytics to open the door to a world of possibilities.

There’s no doubt that big data is playing an increasingly big role in business, politics, healthcare, education, retail and numerous other industries. With the right tools and expertise, organizations can slice and dice data to reveal trends and other information that will inform decisions about future strategy and direction.

v Big data into politics: During the 2012 election, big data was used to great advantage—namely, by Barack Obama, in his re-election campaign. A November 2012 article in Time magazine, “Inside the Secret World of the Data Crunchers Who Helped Obama Win, reported, “Data-driven decision making played a huge role in creating a second term for the 44th President and will be one of the more closely studied elements of the 2012 cycle. … In politics, the era of big data has arrived.”

The role of big data will only increase in the 2016 presidential election, with practitioners using technology to hone in on data much more granularly than ever before. In addition, say experts, the use of big data will overlap with social media and other platforms.

v Big data - generated by tracking personal data: Two devices that are [slowly] gaining ground and adoption (mainly by early-adopters who are already listening to their bodies) are Jawbone’s UP bracelet and the Nike+ Fuel band. Both devices are meant to track personal activities and help individuals achieve personal goals and understand their daily habits.

v According to Gartner, 42 percent of IT leaders have invested in Big Data or plan to do so in a year, a Silicone ANGLE headline read that Big Data was finally “going mainstream.” According to a recent article in The New York Times, as more Major League Baseball teams use data to make team-building decisions, they are leaning on sportscasters to help audiences better understand what the data means. So, Big Data analytics is going mainstream.

There are multiple uses for big data in every industry – from analyzing larger volumes of data than was previously possible to drive more precise answers, to analyzing data in motion to capture opportunities that were previously lost. A big data platform will enable your organization to tackle complex problems that previously could not be solved.

Big data = Big Return on Investment (ROI)

While there is a lot of buzz about big data in the market, it isn’t hype. Plenty of customers are seeing tangible ROI using big data solutions to address their big data challenges:

Ø Healthcare: 20% decrease in patient mortality by analyzing streaming patient data

Ø Telco: 92% decrease in processing time by analyzing networking and call data

Ø Utilities: 99% improved accuracy in placing power generation resources by analyzing 2.8 petabytes of untapped data