Talk by Assoc. Prof. Rui Carlos Oliveira: "Exascale data management for the IoT"
February 26 (Monday)
National Institute of Informatics
Room 1901-1902, 19th floor
Exascale data management for the IoT
Rui Carlos Oliveira
Member of the Board of Directors of INESC TEC (Institute for Systems and Computer Engineering ,Technology and Science)
Associate Professor at the Informatics Department of University of Minho
The proliferation of connected devices, together with the decreasing cost of storage, led to a huge growth on data being generated and collected every day, which can be leveraged by different organizations. Specifically, the trend to provide connectivity in all segments of devices is currently pushing the IoT trend to grow to unprecedented levels. However, storing and processing this data in a scalable and cost-effective way poses new challenges. It becomes essential to find novel data management solutions that can leverage their computational and storage capabilities.
IoT devices pose interesting challenges, namely related with the mobility often promoted; imposing limitations in processing, storage and connectivity. Recent years introduced research that focus on how to tackle the instability observed in large scale scale systems. The intrinsic dynamism of such systems demands software that is able to cope with continuous failures and high levels of node churn.
We introduce DataFlasks, an epidemic data store tailored for holding and processing data originated across millions to billions of devices. As the main advantage, this system leverages the computational and storage
capabilities of each IoT device, while being able to cope with the high dynamism and limited computational resources found across these devices. In detail, high levels of churn are supported with a pro-active approach to fault tolerance while storage and network resources are spared with a space-efficient design.
In order to extend DataFlasks' capabilities to extract meaningful information for both end-users and companies, this system is being integrated with analytics frameworks such as Apache Spark. Also, analytics are performed over data that may contain sensible information. Such data needs to be protected with proper security/privacy mechanisms.
secretaries (at) nii.ac.jp