IDC (on behalf of EMC) have stated that in 2014, the digital universe – the data we create and copy annually - will equal 1.7Mb per minute for every person on Earth. Furthermore, they estimate that the size of the digital universe will double every year, reaching 44 zettabytes by 2020.
This data is generated from a wide variety of sources including social media, transaction records, digital pictures and videos, intelligent sensors, surveys, telephone records and health records. As a result, companies, academia and governments have amassed a wealth of data. The data ranges from knowing our likes and dislikes to the mapping of the human genome and tracking climate change. This has led many to coin this as the era of 'Big Data'.
The possibilities and challenges of 'Big Data' have been widely debated in academia, industry and governments with both US President Barack Obama and UK Prime Minister David Cameron espousing the possibilities in recent speeches.
Fortunately, the past couple of decades have also seen a dramatic increase in both processing power and data storage to deal with this data explosion. However, in order to harness the power of 'Big Data' we must utilise data analytics. Analytics are transforming the way in which companies, governments and public bodies are doing business. Data analysis is bringing organisations and governments closer to their customers and citizens, allowing them to better understand their businesses and citizens' behaviours and plan for the future.
The IBM Institute for Business Value showed that data analytics are helping the smartest companies turn insights into actions. Famous examples are: LinkedIn, which analyse individual’s professional contacts to infer other potential connections; Facebook and Twitter use people's connections and preferences to deliver targeted marketing; whilst Amazon claim to know their customers so well that they are considering dispatching items before they have even been ordered!
'Big Data' is even being used in politics and for predictive policing to cut crime. Barack Obama’s 2012 re-election campaign was founded on 'Big Data' and this allowed them to better understand and target voters. The Conservative Party (UK) have since hired Obama’s campaign strategy adviser, Jim Messina, to guide them through the 2015 general election.
In 2011 McKinsey Global Institute published “Big data: The next frontier for innovation, competition and productivity”. This comprehensive report examined the global landscape for 'Big Data' and the current progress and future developments in several key sectors. They identified several areas in which 'Big Data' could create value:
- Creating Transparency
- Enabling experimentation to discover needs, expose variability and improve performance
- Segmenting populations to customise actions
- Replacing/supporting human decision making with automated algorithms
- Innovating new business models, products and services
Companies and governments alike have bought into this narrative withe Big Data related spending set to skyrocket over coming years. This is highlighted by research from Bain & Co. which examined the expected growth in Big Data spending across a range of sectors
As well as these opportunities, the 'Big Data' era also presents numerous challenges, such as how can data be used to improve businesses and society? How can we analyse all this data? Traditional analysis is often inadequate when dealing with such vast amounts of data and simplistic methods can often lead to misleading or no actionable results. This is where Spectra Analytics truly differentiates itself from other firms in this sector.
Spectra Analytics focuses on 'Big Information' as opposed to 'Big Data'. This is because applying simplistic methods can lead to little or no useful information for client businesses, regardless of data size. Whereas 'Big Information' requires utilising the most appropriate and sometimes most sophisticated data analysis techniques currently being developed in academia. This is where the interdisciplinary nature of Complexity Science (upon which Spectra Analytics is founded), with its focus on mathematical modelling and statistics, has the opportunity to revolutionise the area. To complement these skills the firm also has expertise in economics, finance, psychology, machine learning, pattern recognition, network theory, natural language processing and sentiment analysis.
The McKinsey report identified several key implications for both organisational leaders and policy makers. We present a summary of their findings here:
- Inventory data assets: proprietary, public and purchased
- Identifying potential value creation opportunities and threats
- Build up internal capabilities to create a data-driven organisation
- Develop enterprise information strategy to implement technology
- Address data policy issues
- Build human capital for 'Big Data'
- Align incentives to promote data sharing for the greater good
- Develop policies that balance the interests of companies wanting to create value from data and citizens wanting to protect their privacy and security
- Establish effective intellectual property frameworks to ensure innovation
- Address technology barriers and accelerate R&D in targeted areas
- Ensure investments in underlying information and communication technology infrastructure
In a 2010 global survey of nearly 3,000 executive managers and analysts, MIT Sloan Management Review in collaboration with IBM Institute for Business Value, found that the best performing companies used a greater degree of analytics and across a broader range of business functions. They found that top companies use analytics to prescribe actions rather than to support or validate decisions. Furthermore, top companies use analytics to guide day-to-day operations as well as future strategies.
- Innovating to achieve competitive differentiation
- Growing revenue
- Reducing costs and increasing efficiencies
Top obstacles to the adoption of analytics:
- Lack of understanding how to use analytics to improve the business
- Lack of bandwidth due to competing priorities
- Lack of skills internally in the line of business
- Financial management and budgeting
- Operations and production
- Strategy and business development
- Sales and marketing
In a 2013 survey of 700 North American enterprises, IDC (on behalf of EMC) found that less than 1% achieved the highest level of 'Big Data' usage – 'Big Data' operationalised and continuously providing process improvement realisation.
There are essentially three ways to implement data analysis:
- Conduct the analysis 'in-house'
- Buy an off-the-shelf product
- Hire a data analysis consultancy/research firm
Hiring an 'in-house' analytics team has some major benefits as it is highly flexible and there are no privacy concerns. However, it is not always the most cost-effective solution. This is because, as noted by the McKinsey Institute and Harvard Business Review, data scientists are a scarce resource and very expensive to hire. Furthermore, hiring is problematic because there are few managers qualified to judge the credentials of data scientists. Beyond the costs, there are other drawbacks to in-house analysis. This is because the in-house team will only be exposed to a narrow range of research in their day-to-day activities. This means that they will not have access to the latest research in other fields which can inhibit their future ability to produce cutting-edge solutions.
An off-the-shelf product is often cheap and can offer quick benefits for your organisation. Unfortunately, it is a one size fits all approach and is inappropriate for many organisations. It almost certainly will not meet all of your needs and could end up costing far more than originally envisaged to develop and integrate the product with existing systems. Furthermore, if other organisations within your sector are adopting the same package, the benefits will be reduced. It will not allow your organisation to differentiate itself and move ahead of your competitors.
We believe that the first two options are only really viable for small, simple businesses. For larger, complex organisations a bespoke consultancy is the only realistic approach. This is where Spectra Analytics, with our close links to academia and inter-disciplinary research, can provide real value for many clients.
Beyond selecting the method for developing a data-driven organisation, one must also consider changes and policies to be implemented within the business itself. The McKinsey report identifies several of these issues:
- Data policies
- Technology and techniques
- Organisational change and talent
- Access to data
- Industry structure
'Open Data' refers to publicly available data. This includes data produced by governments, social media and even by some corporations such as Google.
In June 2013, G8 countries adopted an "Open Data Charter" which makes it the default policy of governments to make data freely available.
This produces many potential opportunities for companies. McKinsey estimates that 'Open Data' could be worth in excess of $3trn per annum.
Companies can either analyse the 'Open Data' directly or integrate it with proprietary data to identify opportunities and reduce risks which would not have been possible with proprietary data alone.
back to top...