ASEAN's Premier News and Eco System Portal for Big Data, IoT and AI

Home > Big news > News > Better Decisions For The Community And Economy With Open Data
Better Decisions For The Community And Economy With Open Data
January 15, 2018 News analytics big data open data


There has been a lot of initiatives in Malaysia to push Big Data and the Open Data portal. Open Data is described as data which is publicly available, can be universally and readily accessed, used and redistributed free of charge. It should be structured for usability and computability. This sounds simple but why is it so important to ASEAN nations such as Malaysia, Indonesia and their neighbours?

Big Community spoke to Cloudera, a modern platform for machine learning and analytics optimized for the cloud, on how relevant it is for companies and the government to be on board and in collaboration to make this a success.

Mark Micallef, Vice President, Asia Pacific & Japan, shared his views with us.

Mark Micallef, VP, APJ, Cloudera


Big Community : Does Cloudera have ideas or suggestions on how Malaysia could improve its open data portal and how open data can be utilised in an effective manner?

Mark Micallef : Big data is taking centre stage in many smart city development projects and with businesses today. At the crux of these efforts is the availability and accessibility of data sets, for citizens and businesses to leverage on and learn from – this is open data. With a greater access to information, all parties (the government, private sector, and citizens) can make more educated decisions for the betterment of the community and economy. However, achieving this is easier said than done as organisations, both public and private, face issues like data silos, multiple data sources, and data in proprietary formats. The answer lies in a scalable and secure platform that can make the technical challenges of data collection and formatting simpler and more standardised, resulting in a successful open data model.

Open data models not only accelerate insight-driven decision-making, but also enable Machine Learning (ML) for cybersecurity. This is essential as cyber security has become a machine-scale problem and needs to be tackled with just as big, fast, and complex a solution. Apache Spot, for example, is an open source cyber security project that uses ML to separate bad traffic from benign and characterises the unique behaviour of network traffic, leading to better and faster threat detection. Cloudera’s cybersecurity solution is based on Apache Spot, empowering organisations to build custom security solutions and deploy packaged applications on top of one shared, enriched data set.


Big Community : The current datasets are provided in silos – does Cloudera have aims or ideas for how that could be addressed?

Mark : While the organisational goal is to have all data available in one place, in known formats, that is not the reality. Data sets are often stored across departments and teams (from sales to IT and operations), all in different formats based on specific requirements and with varying security point solutions solving only narrow, specific problems. In fact, the larger the organisation, the more disparate the owners of data.  The pieces of the puzzle need to work together to form a full picture. What organisations need is a scalable solution that can help them centralise data easily, store it safely, and keep the data secure yet accessible to a large group of target users, as needed. An enterprise data hub built on Hadoop, for instance, makes it easier to centralise data across an organisation and have that data available for business intelligence, operations optimisation, and of course, cybersecurity.


Big Community : Do you have examples of private companies and corporations using open data either in Malaysia or elsewhere in the region and will Cloudera start to advise corporate customers to think more about using open data sets in the future?

Mark : The accuracy and security of data are of utmost importance in any open data project. In Malaysia and the region, Cloudera and Tech Data (large wholesale distributor of technology products) are working together to provide advanced ML and analytics capabilities that will help all types of organisations better manage their data and make informed decisions, securely. The goal is to help bolster and accelerate smart city and IoT projects across the ASEAN region.

As the volume of data grows, the opportunity to solve the world’s biggest problems has never been greater. All organisations, big or small and public or private, will benefit from a centralised data hub that merges data sets and insights from across the organisation.  With the right platform and resources, all organisations have much to gain from this.

—End of Interview—

Comment from Big Community Editor: It’s important to note that open data focuses on non-personal data and avoids going into data on specific individuals. Government data also needs to be properly scrutinised for sensitive information and would therefore rely on security restrictions. Open data will help improve government, empower citizens and also build economic value through the opportunities it will create and help enable. Together with big data technology and companies like Cloudera, open data can realise its full potential to make a positive change.