Big data describes a massive volume of both structured and unstructured data that is so large it is difficult to process using standard database and software methods. It has been described in The Gartner IT glossary as “high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making”. Big data:
- uses massive, diverse, complex, longitudinal, and/or distributed datasets that are generated by, or collected from, a variety of different devices, sensors and transactions (volume);
- brings together data from different sources, both structured and unstructured (variety); and
- is processed quickly, often exceeding current processing capacity (velocity).
As big data is a burgeoning phenomenon, the legal framework is quickly developing to try to keep up with and manage compliance with data protection laws. The Information Commissioner Office (ICO) published a report on big data in the UK in 2014 which is a useful tool for understanding big data and your obligations (https://ico.org.uk/media/for-organisations/documents/1541/big-data-and-data-protection.pdf).
Although much of big data is not personal data (for instance world climate and weather data) there are examples where big data analytics include the processing of personal data (for instance data from monitoring devices on patients in clinical trials, mobile phone location data, data on purchases made with loyalty cards and biometric data from body-worn devices). As such, the authorities have decided that big data should fall within the scope of data protection laws and therefore must comply with the eight data protection principles.
In particular, businesses processing big data should:
- Abide by the rules of fairness and transparency and meet the reasonable expectations of the data subject in processing data;
- Explain the benefits of analytics to the data subject and obtain prior consent;
- Collect and use data for specified, explicit and legitimate purposes;
- Use and collection of data must be adequate, relevant, not excessive and must not be kept longer than is strictly necessary;
- Anonymise data;
- Respect the rights of data subject; and
- Consider carrying out a privacy impact assessment to assess how big data analytics is likely to affect individuals whose data is being processed and where such use is fair.
If you would like advice in relation to big data and how you can comply with your data protection obligations please contact our Corporate Commercial team.