Big Data Solutions
Big data refers to data sets that are so large or complex that traditional data processing applications are inadequate. When manipulating and analysing data of such sizes typical challenges include:
- Data curation
- Information privacy
Analysis of data sets so large can find new correlations that would not have been identified with smaller data sets. This provides the ability to spot business trends, prevent diseases, combat crime and so on.
The Apache™ Hadoop™ software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.
Dunstan Thomas implements big data solutions using the Microsoft HDInsight framework. HDInsight is based on the Hortonworks Data Platform, which is a 100% open source distribution of Apache™ Hadoop™. HDInsight provides a software framework designed to manage, analyze, and report on data. The HDInsight Service on Windows Azure provides Hadoop as a scalable, on-demand service as part of the Windows Azure Platform.
Dunstan Thomas has developed a platform for analysing electricity market data using HDInsight on the Windows Azure Platform.
For more information on how Dunstan Thomas can help you to develop your big data requirements, Contact Us or ring 02392 822254.