Real-time data analysis

The process of obtaining models from data, either for prediction or monitoring, is often difficult or impossible to perform with traditional techniques because the amount of data to be analyzed is too large to be stored before processing. In addition, there is an increasing demand to explain what is happening at the time it is happening, that is, incremental learning is required to keep the models updated and ready for use at all times. The existence of data sources that continuously generate chronologically ordered information and that exceed the usual storage and processing capacities, is already a reality in many applications that grow every day, such as telecommunications, vehicle traffic, energy consumption, commerce, finance, medicine, physiology, sports, robotics or social networks, among others. This aspect also connects with the IoT (Internet of Things) and Edge Computing, where highly efficient real-time data analysis techniques are increasingly needed for direct implementation in sensory devices.

To address this problem, data streams are managed, which are infinite sequences of structured records. The key feature of these systems is that the data produced is not stored permanently but is processed “on the fly”, that is, each data is analyzed, processed and finally forgotten, thus being able to address huge amounts of data in real time even with reduced storage and calculation capacities. Data streams pose new challenges for machine learning and data analysis, such as the use of limited computational resources, the need for rapid response, adaptation to changes in data distribution, or the scarcity of data labeling due to such rapid arrival.

In DaSCI we investigate in the design of machine learning techniques capable of building models in real time according to the data that are being received and we have applied it successfully to problems such as the analysis of electroencephalogram, critical care medicine, Twitter activity, monitoring of cell phone use or energy consumption.

Contact: Jorge Casillas Barranquero

Related Researchers:


  Name Email Area Cat.
Carmona del Jesus, Cristóbal J. Data Science and Big Data Area PhD
Casillas Barranquero, Jorge Data Science and Big Data Area PhD
García Gil, Diego Jesús Data Science and Big Data Area, Computational Intelligence Area PhD
Herrera Triguero, Francisco DaSCI Technology Applications Area, Data Science and Big Data Area, Computational Intelligence Area PhD
Scroll Up