Change search
Link to record
Permanent link

Direct link
BETA
Publications (6 of 6) Show all publications
Holst, A., Bouguelia, M.-R. -., Görnerup, O., Pashami, S., Al-Shishtawy, A., Falkman, G., . . . Soliman, A. (2019). Eliciting structure in data. In: CEUR Workshop Proceedings: . Paper presented at 2019 Joint ACM IUI Workshops, ACMIUI-WS 2019, 20 March 2019.
Open this publication in new window or tab >>Eliciting structure in data
Show others...
2019 (English)In: CEUR Workshop Proceedings, 2019Conference paper, Published paper (Refereed)
Abstract [en]

This paper demonstrates how to explore and visualize different types of structure in data, including clusters, anomalies, causal relations, and higher order relations. The methods are developed with the goal of being as automatic as possible and applicable to massive, streaming, and distributed data. Finally, a decentralized learning scheme is discussed, enabling finding structure in the data without collecting the data centrally. © 2019 Copyright held for the individual papers by the papers’ authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.

Keywords
Anomaly detection, Causal inference, Clustering, Distributed analytics, Higher-order structure, Information visualization, Information systems, User interfaces, Causal inferences, Data acquisition
National Category
Natural Sciences
Identifiers
urn:nbn:se:ri:diva-38261 (URN)2-s2.0-85063227224 (Scopus ID)
Conference
2019 Joint ACM IUI Workshops, ACMIUI-WS 2019, 20 March 2019
Available from: 2019-04-02 Created: 2019-04-02 Last updated: 2019-04-02Bibliographically approved
Abbas, Z., Sigurdsson, T., Al-Shishtawy, A. & Vlassov, V. (2018). Evaluation of the use of streaming graph processing algorithms for road congestion detection. In: Proceedings - 16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018: . Paper presented at 16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018, 11 December 2018 through 13 December 2018 (pp. 1017-1025). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>Evaluation of the use of streaming graph processing algorithms for road congestion detection
2018 (English)In: Proceedings - 16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 1017-1025Conference paper, Published paper (Refereed)
Abstract [en]

Real-time road congestion detection allows improving traffic safety and route planning. In this work, we propose to use streaming graph processing algorithms for road congestion detection and evaluate their accuracy and performance. We represent road infrastructure sensors in the form of a directed weighted graph and adapt the Connected Components algorithm and some existing graph processing algorithms, originally used for community detection in social network graphs, for the task of road congestion detection. In our approach, we detect Connected Components or communities of sensors with similarly weighted edges that reflect different states in the traffic, e.g., free flow or congested state, in regions covered by detected sensor groups. We have adapted and implemented the Connected Components and community detection algorithms for detecting groups in the weighted sensor graphs in batch and streaming manner. We evaluate our approach by building and processing the road infrastructure sensor graph for Stockholm's highways using real-world data from the Motorway Control System operated by the Swedish traffic authority. Our results indicate that the Connected Components and DenGraph community detection algorithms can detect congestion with accuracy up to ? 94% for Connected Components and up to ? 88% for DenGraph. The Louvain Modularity algorithm for community detection fails to detect congestion regions for sparsely connected graphs, representing roads that we have considered in this study. The Hierarchical Clustering algorithm using speed and density readings is able to detect congestion without details, such as shockwaves.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2018
Keywords
Community Detection, Congestion, Connected Components, Graph Processing, Streaming, Acoustic streaming, Big data, Cloud computing, Directed graphs, Population dynamics, Roads and streets, Signal detection, Traffic congestion, Ubiquitous computing, Community detection algorithms, Connected component, Hierarchical clustering algorithms, Road infrastructures, Traffic authorities, Clustering algorithms
National Category
Natural Sciences
Identifiers
urn:nbn:se:ri:diva-38390 (URN)10.1109/BDCloud.2018.00148 (DOI)2-s2.0-85063892833 (Scopus ID)9781728111414 (ISBN)
Conference
16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018, 11 December 2018 through 13 December 2018
Note

Funding details: Fellowships Fund Incorporated; Funding details: VINNOVA; Funding details: 20140221; Funding details: European Commission, FPA 2012-0030; Funding details: Education, Audiovisual and Culture Executive Agency; Funding details: 2015-00677; Funding text 1: ACKNOWLEDGMENT This work was supported by the Erasmus Mundus Joint Doctorate in Distributed Computing (EMJD-DC) funded by the Education, Audiovisual and Culture Executive Agency (EACEA) of the European Commission under FPA 2012-0030, by the project BADA: Big Automotive Data Analytics in the funding program FFI: Strategic Vehicle Research and Innovation (grant 2015-00677) administrated by VINNOVA the Swedish government agency for innovation systems, and by the project BIDAF: Big Data Analytics Framework for a Smart Society (grant 20140221) funded by KKS the Swedish Knowledge Foundation.

Available from: 2019-05-10 Created: 2019-05-10 Last updated: 2019-05-10Bibliographically approved
Abbas, Z., Al-Shishtawy, A., Girdzijauskas, S. & Vlassov, V. (2018). Short-Term Traffic Prediction Using Long Short-Term Memory Neural Networks. In: Proceedings - 2018 IEEE International Congress on Big Data, BigData Congress 2018 - Part of the 2018 IEEE World Congress on Services: . Paper presented at 7th IEEE International Congress on Big Data, BigData Congress 2018, 2 July 2018 through 7 July 2018 (pp. 57-65). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>Short-Term Traffic Prediction Using Long Short-Term Memory Neural Networks
2018 (English)In: Proceedings - 2018 IEEE International Congress on Big Data, BigData Congress 2018 - Part of the 2018 IEEE World Congress on Services, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 57-65Conference paper, Published paper (Refereed)
Abstract [en]

Short-term traffic prediction allows Intelligent Transport Systems to proactively respond to events before they happen. With the rapid increase in the amount, quality, and detail of traffic data, new techniques are required that can exploit the information in the data in order to provide better results while being able to scale and cope with increasing amounts of data and growing cities. We propose and compare three models for short-term road traffic density prediction based on Long Short-Term Memory (LSTM) neural networks. We have trained the models using real traffic data collected by Motorway Control System in Stockholm that monitors highways and collects flow and speed data per lane every minute from radar sensors. In order to deal with the challenge of scale and to improve prediction accuracy, we propose to partition the road network into road stretches and junctions, and to model each of the partitions with one or more LSTM neural networks. Our evaluation results show that partitioning of roads improves the prediction accuracy by reducing the root mean square error by the factor of 5. We show that we can reduce the complexity of LSTM network by limiting the number of input sensors, on average to 35% of the original number, without compromising the prediction accuracy. .

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2018
Keywords
LSTM, neural networks, traffic prediction, Big data, Brain, Forecasting, Intelligent systems, Mean square error, Roads and streets, Traffic control, Evaluation results, Input sensor, Intelligent transport systems, Prediction accuracy, Radar sensors, Root mean square errors, Long short-term memory
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:ri:diva-37285 (URN)10.1109/BigDataCongress.2018.00015 (DOI)2-s2.0-85054887478 (Scopus ID)9781538672327 (ISBN)
Conference
7th IEEE International Congress on Big Data, BigData Congress 2018, 2 July 2018 through 7 July 2018
Note

Funding details: Fellowships Fund Incorporated, FFI; Funding details: VINNOVA; Funding details: 20140221; Funding details: European Commission, EC, FPA 2012-0030; Funding details: Education, Audiovisual and Culture Executive Agency, EACEA; Funding details: Directorate-General for Research and Innovation, 2015-00677; Funding text 1: ACKNOWLEDGMENT This work was supported by the project BADA: Big Automotive Data Analytics in the funding program FFI: Strategic Vehicle Research and Innovation (grant 2015-00677) administrated by VINNOVA the Swedish government agency for innovation systems, by the project BIDAF: Big Data Analytics Framework for a Smart Society (grant 20140221) funded by KKS the Swedish Knowledge Foundation, and by the Erasmus Mundus Joint Doctorate in Distributed Computing (EMJD-DC) programme funded by the Education, Audiovisual and Culture Executive Agency (EACEA) of the European Commission under FPA 2012-0030.

Available from: 2019-01-18 Created: 2019-01-18 Last updated: 2019-03-29Bibliographically approved
Liu, Y., Gureya, D., Al-Shishtawy, A. & Vlassov, V. (2017). OnlineElastMan: self-trained proactive elasticity manager for cloud-based storage services. Cluster Computing
Open this publication in new window or tab >>OnlineElastMan: self-trained proactive elasticity manager for cloud-based storage services
2017 (English)In: Cluster Computing, ISSN 1386-7857, E-ISSN 1573-7543Article in journal (Refereed) In press
Abstract [en]

The pay-as-you-go pricing model and the illusion of unlimited resources in the Cloud initiate the idea to provision services elastically. Elastic provisioning of services allocates/de-allocates resources dynamically in response to the changes of the workload. It minimizes the service provisioning cost while maintaining the desired service level objectives (SLOs). Model-predictive control is often used in building such elasticity controllers that dynamically provision resources. However, they need to be trained, either online or offline, before making accurate scaling decisions. The training process involves tedious and significant amount of work as well as some expertise, especially when the model has many dimensions and the training granularity is fine, which is proved to be essential in order to build an accurate elasticity controller. In this paper, we present OnlineElastMan, which is a self-trained proactive elasticity manager for cloud-based storage services. It automatically evolves itself while serving the workload. Experiments using OnlineElastMan with Cassandra indicate that OnlineElastMan continuously improves its provision accuracy, i.e., minimizing provisioning cost and SLO violations, under various workload patterns.

Place, publisher, year, edition, pages
Springer New York LLC, 2017
Keywords
Cloud storage, Elasticity controller, Online training, SLO, Time series analysis, Workload prediction, Controllers, Costs, Elasticity, Managers, Model predictive control, Cloud storages, Service level objective, Service provisioning, Storage services, Training process, Workload patterns, Workload predictions, Storage management
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:ri:diva-29764 (URN)10.1007/s10586-017-0899-z (DOI)2-s2.0-85019724244 (Scopus ID)
Available from: 2017-06-12 Created: 2017-06-12 Last updated: 2018-07-09Bibliographically approved
Al-Shishtawy, A. (2012). Self-management for large-scale distributed systems (6ed.). (Doctoral dissertation). Sweden: KTH
Open this publication in new window or tab >>Self-management for large-scale distributed systems
2012 (English)Doctoral thesis, monograph (Other academic)
Abstract [en]

Autonomic computing aims at making computing systems self-managing by using autonomic managers in order to reduce obstacles caused by management complexity. This thesis presents results of research on self-management for large-scale distributed systems. This research was motivated by the increasing complexity of computing systems and their management. In the first part, we present our platform, called Niche, for programming self-managing component-based distributed applications. In our work on Niche, we have faced and addressed the following four challenges in achieving self-management in a dynamic environment characterized by volatile resources and high churn: resource discovery, robust and efficient sensing and actuation, management bottleneck, and scale. We present results of our research on addressing the above challenges. Niche implements the autonomic computing architecture, proposed by IBM, in a fully decentralized way. Niche supports a network-transparent view of the system architecture simplifying the design of distributed self-management. Niche provides a concise and expressive API for self-management. The implementation of the platform relies on the scalability and robustness of structured overlay networks. We proceed by presenting a methodology for designing the management part of a distributed self-managing application. We define design steps that include partitioning of management functions and orchestration of multiple autonomic managers. In the second part, we discuss robustness of management and data consistency, which are necessary in a distributed system. Dealing with the effect of churn on management increases the complexity of the management logic and thus makes its development time consuming and error prone. We propose the abstraction of Robust Management Elements, which are able to heal themselves under continuous churn. Our approach is based on replicating a management element using finite state machine replication with a reconfigurable replica set. Our algorithm automates the reconfiguration (migration) of the replica set in order to tolerate continuous churn. For data consistency, we propose a majority-based distributed key-value store supporting multiple consistency levels that is based on a peer-to-peer network. The store enables the tradeoff between high availability and data consistency. Using majority allows avoiding potential drawbacks of a master-based consistency control, namely, a single-point of failure and a potential performance bottleneck. In the third part, we investigate self-management for Cloud-based storage systems with the focus on elasticity control using elements of control theory and machine learning. We have conducted research on a number of different designs of an elasticity controller, including a State-Space feedback controller and a controller that combines feedback and feedforward control. We describe our experience in designing an elasticity controller for a Cloud-based key-value store using state-space model that enables to trade-off performance for cost. We describe the steps in designing an elasticity controller. We continue by presenting the design and evaluation of ElastMan, an elasticity controller for Cloud-based elastic key-value stores that combines feedforward and feedback control.

Place, publisher, year, edition, pages
Sweden: KTH, 2012. p. 266 Edition: 6
Series
SICS dissertation series, ISSN 1101-1335 ; 57
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:ri:diva-24046 (URN)9789175014371 (ISBN)
Available from: 2016-10-31 Created: 2016-10-31 Last updated: 2018-07-09Bibliographically approved
Al-Shishtawy, A., Asif Fayyaz, M., Popov, K. & Vlassov, V. (2010). Achieving Robust Self-Management for Large-Scale Distributed Applications (7ed.). Kista, Sweden
Open this publication in new window or tab >>Achieving Robust Self-Management for Large-Scale Distributed Applications
2010 (English)Report (Other academic)
Abstract [en]

Autonomic managers are the main architectural building blocks for constructing self-management capabilities of computing systems and applications. One of the major challenges in developing self-managing applications is robustness of management elements which form autonomic managers. We believe that transparent handling of the effects of resource churn (joins/leaves/failures) on management should be an essential feature of a platform for self-managing large-scale dynamic distributed applications, because it facilitates the development of robust autonomic managers and hence improves robustness of self-managing applications. This feature can be achieved by providing a robust management element abstraction that hides churn from the programmer. In this paper, we present a generic approach to achieve robust services that is based on finite state machine replication with dynamic reconfiguration of replica sets. We contribute a decentralized algorithm that maintains the set of nodes hosting service replicas in the presence of churn. We use this approach to implement robust management elements as robust services that can operate despite of churn. Our proposed decentralized algorithm uses peer-to-peer replica placement schemes to automate replicated state machine migration in order to tolerate churn. Our algorithm exploits lookup and failure detection facilities of a structured overlay network for managing the set of active replicas. Using the proposed approach, we can achieve a long running and highly available service, without human intervention, in the presence of resource churn. In order to validate and evaluate our approach, we have implemented a prototype that includes the proposed algorithm.

Place, publisher, year, edition, pages
Kista, Sweden: , 2010 Edition: 7
Series
SICS Technical Report, ISSN 1100-3154 ; 2010:02
Keywords
autonomic computing, distributed systems, self-management, replicated state machines, service migration, peer-to-peer
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:ri:diva-23666 (URN)
Available from: 2016-10-31 Created: 2016-10-31 Last updated: 2018-08-14Bibliographically approved
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-9546-4937

Search in DiVA

Show all publications
v. 2.35.7