Big Data – a term used to describe the exponential growth and availability of structure and unstructured data, made commonplace by Internet companies like Google and Facebook who process massive amounts of data and develop innovative ways of harnessing that to gain insight into users.
The Three V’s of Big Data
- Volume: big data defines the increase in data volume. This is as a result of transaction-based data stored through the years, such as unstructured data streaming from social media and increasing amounts of sensor and machine-to-machine data being collected. IN the past, excessive data volume was a storage issue, but now that storage costs are continually decreasing with economies of scale, that is no longer an issue. However new issues are emerging, such as how to determine relevance within large data volumes and how to use analytics to create value from relevant data
- Velocity: data streams at unbelieveable speeds and must be processed in a timely manner. RFID tags, sensors and smart metering are tasked with processing literal torrents of data in near-real time. The speed required to process and react to this amount of data is a challenge for most organizations.
- Variety: this data is coming through in all sorts of formats, such as structured, numeric, or, in the case of building automatin, time-series. Other formats are unstructured text documents, email, video, audio and financial transactions. Managing, merging and governing all of these different varieties of data is also a challenge for most organizations.