Big Data:
“Small data is gone. Data is
just going to get bigger and bigger and bigger, and people just have to think
differently about how they manage it.” –Scott Zucker
Every year, companies are seeing dramatic
increases in data coming from more and more disparate sources.
According to IDC, we
create 2.5 quintillion bytes of data. It is not believable but it is a fact
that 90% of the data in the world today has been created in the last two years
alone. As per International Data Corporation (IDC), it is imperative that
organizations and IT leaders focus on the ever-increasing volume, variety and
velocity of information that forms big data.
Data
is generated from everywhere: social media, mobile, educational information,
individual information etc. This data is big
data.
Big data spans multiple dimensions:
· Volume: Many factors
contribute to the increase in data volume – transaction-based data stored
through the years, text data constantly streaming in from social media,
increasing amounts of sensor data being collected, etc. In the past, excessive
data volume created a storage issue. But with today's decreasing storage costs,
other issues emerge, including how to determine relevance amidst the large
volumes of data and how to create value from data that is relevant.
Enterprises are awash with
ever-growing data of all types, easily amassing terabytes—even petabytes of
information.
Ø Turn 12 terabytes of Tweets created each day into improved
product sentiment analysis
Ø Convert 350 billion annual meter readings to better predict
power consumption
· Variety: Data today comes
in all types of formats– from traditional databases to hierarchical data stores
created by end users and OLAP systems, to text documents, email,
meter-collected data, video, audio, stock ticker data and financial
transactions. By some estimates, 80 percent of an organization's data is not
numeric! But it still must be included in analyses and decision making.
Sometimes 2 minutes is too
late. For time-sensitive processes such as catching fraud, big data must be
used as it streams into your enterprise in order to maximize its value.
Ø Scrutinize 5 million trade events created each day to
identify potential fraud
Ø Analyze 500 million daily call detail records in real-time to
predict customer churn faster
· Velocity: According to Gartner, velocity "means both
how fast data is being produced and how fast the data must be processed to meet
demand." RFID tags and smart metering are driving an increasing need to
deal with torrents of data in near-real time. Reacting quickly enough to deal
with velocity is a challenge to most organizations.
Big data is any type of data
- structured and unstructured data such as text, sensor data, audio, video,
click streams, log files and more. New insights are found when analyzing these
data types together.
Ø Monitor 100’s of live video feeds from surveillance cameras
to target points of interest
Ø Exploit the 80% data growth in images, video and documents to
improve customer satisfaction
· Veracity:
1 in 3 business leaders don’t
trust the information they use to make decisions. How can you act upon
information if you don’t trust it? Establishing trust in big data presents a
huge challenge as the variety and number of sources grows.
· Variability: In addition to the increasing velocities and
varieties of data, data flows can be highly inconsistent with periodic peaks.
Is something big trending in the social media? Perhaps there is a high-profile
IPO looming. Maybe swimming with pigs in the Bahamas is suddenly the must-do
vacation activity. Daily, seasonal and event-triggered peak data loads can be
challenging to manage – especially with social media involved.
· Complexity: When you deal with huge
volumes of data, it comes from multiple sources. It is quite an undertaking to
link, match, cleanse and transform data across systems. However, it is
necessary to connect and correlate relationships, hierarchies and multiple data
linkages or your data can quickly spiral out of control. Data governance can
help you determine how disparate data relates to common definitions and how to
systematically integrate structured and unstructured data assets to produce
high-quality information that is useful, appropriate and up-to-date.
Big data is more than simply a matter of size;
it is an opportunity to find insights in new and emerging types of data and
content, to make your business more agile, and to answer questions that were
previously considered beyond your reach. Until now, there was no practical way
to harvest this opportunity. Today, IBM’s platform for big data uses state of
the art technologies including patented advanced analytics to open the door to
a world of possibilities.
There’s no doubt that big data is playing an
increasingly big role in business, politics, healthcare, education, retail and
numerous other industries. With the right tools and expertise, organizations
can slice and dice data to reveal trends and other information that will inform
decisions about future strategy and direction.
v Big
data into politics: During the 2012 election, big data was used to great
advantage—namely, by Barack Obama, in his re-election campaign. A November 2012
article in Time magazine, “Inside the Secret World of the Data Crunchers Who
Helped Obama Win, reported, “Data-driven decision making played a huge role in
creating a second term for the 44th President and will be one of the more
closely studied elements of the 2012 cycle. … In politics, the era of big data
has arrived.”
The role of big data will
only increase in the 2016 presidential election, with practitioners using
technology to hone in on data much more granularly than ever before. In
addition, say experts, the use of big data will overlap with social media and
other platforms.
v Big data -
generated by tracking
personal data: Two
devices that are [slowly] gaining ground and adoption (mainly by early-adopters
who are already listening to their bodies) are Jawbone’s
UP bracelet and the Nike+ Fuel band. Both devices are meant to track personal activities and
help individuals achieve personal goals and understand their daily habits.
v According
to Gartner, 42 percent of IT leaders have invested in Big Data or plan to do so
in a year, a Silicone
ANGLE headline read that Big Data was finally “going
mainstream.” According to a recent article in The New York Times, as more Major League Baseball teams use data to make
team-building decisions, they are leaning on sportscasters to help audiences
better understand what the data means. So, Big Data analytics is going
mainstream.
There are multiple uses for
big data in every industry – from analyzing larger volumes of data than was
previously possible to drive more precise answers, to analyzing data in motion
to capture opportunities that were previously lost. A big data platform will
enable your organization to tackle complex problems that previously could not
be solved.
Big data = Big Return on Investment (ROI)
While there is a lot of buzz about big data in
the market, it isn’t hype. Plenty of customers are seeing tangible ROI using
big data solutions to address their big data challenges:
Ø Healthcare: 20% decrease in patient mortality by analyzing
streaming patient data
Ø Telco: 92% decrease in processing time by analyzing
networking and call data
Ø Utilities: 99% improved accuracy in placing power generation
resources by analyzing 2.8 petabytes of untapped data
