Author: Moshe Segal, Founder and CEO
In recent years, big data has become a hot topic in light of the enormous scale of information of different kinds and from different sources which, only a decade ago, would have sounded like science fiction. As a result, the need has arisen for analytic systems and BI so that organizations can take advantage of the quantities of data to produce practical benefits.
However, the implementation of such systems, no matter how advanced they may be, is still not enough. The question that needs to be asked is whether organizations are actually succeeding in producing value from the information, value that the organization can utilize for business advantage. Do all the organizational applications in fact succeed in benefiting from or utilizing the new insights acquired from the data for practical action? The situation in the field proves otherwise, and most organizations do not exploit these insights or translate them automatically into organizational actions. This means that they are not able to make the transition from the processed information (after it has passed through big data analytics) to organizational applications such as CRM systems, financial systems, and so on.
In order to understand the nature of the challenge, it is important to recognize the situation in the world of business information systems. The first trend is splitting up business systems into specialist vertical applications in the different fields of operation in the organization, such as human resources, finance, customer relations management, and so on. At the same time, the second trend is for most organizations today to operate in a hybrid environment of a private network (LAN) in combination with cloud-based public network systems. As a consequence, it is difficult to support cross-platform business processes, in which it is necessary to distribute the insights and information from the big data systems to the different business applications, some of which are in an on-premise environment, while others are cloud-based systems, and from there, of course, to these same functions in the organization, such as customer service managers, finance managers, marketing, and so on. The result in practice is a Tower of Babel of systems that do not communicate or share common information, and this hampers the organization in making the best use of the valuable information.
Such scenarios involving the transfer of information from big data systems to the relevant business applications are wide-scale. One example is the subject of billing – analyzing millions of data items obtained from the billing system and transferring the information to vertical applications such as customer service, sales, and so on, in order to produce actions that will increase income and customer loyalty.
Although big data identifies the information, analyses it, and processes it into insights, another system is required in order to automate the actions that should certainly be taken in other systems, one that will be able to join up the threads and turn the knowledge into wisdom – an integration platform.
Such an integration platform has to contend with four characteristics of big data, known as the four Vs:
- Volume – the large quantity of information obtained in the organization
- Velocity – the rate of information passed to the different systems
- Variety – the range of sources and the forms in which the information is transferred
- Veracity – uncertainty with regard to the information, “background noises”, bias
The ability to integrate has become no less significant and important to the organization than the big data systems themselves, and they complement each other. As a result, organizations will assimilate an integration platform that can handle the characteristics of the information – on a large scale, transferred at different rates, from a variety of sources, and with varying “background noise”. In this way they can turn the insights from big data into organizational wisdom, expressed in automation of processes and actions in the other organizational systems.
Many organizations today understand that their big data systems can produce value and insights from the data that they accumulate. And indeed, big data systems are a real need, but this application only goes part of the way. In order to obtain and produce the wisdom, there must be integration with the other organizational systems – and it is this that will make the information valuable.
In short, organizations operating in the modern Tower of Babel cannot produce these insights where there is a disconnection between the systems, and it is necessary to add integration systems that are able to synchronize the different systems, and create a uniform language in the organization.
