Change search
Link to record
Permanent link

Direct link
Publications (10 of 76) Show all publications
Corcoran, D., Kreuger, P. & Boman, M. (2022). A Sample Efficient Multi-Agent Approach to Continuous Reinforcement Learning. In: Proceedings of the 2022 18th International Conference of Network and Service Management: Intelligent Management of Disruptive Network Technologies and Services, CNSM 2022. Paper presented at 18th International Conference of Network and Service Management, CNSM 2022, 31 October 2022 through 4 November 2022 (pp. 338-344). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>A Sample Efficient Multi-Agent Approach to Continuous Reinforcement Learning
2022 (English)In: Proceedings of the 2022 18th International Conference of Network and Service Management: Intelligent Management of Disruptive Network Technologies and Services, CNSM 2022, Institute of Electrical and Electronics Engineers Inc. , 2022, p. 338-344Conference paper, Published paper (Refereed)
Abstract [en]

As design, deployment and operation complexity increase in mobile systems, adaptive self-learning techniques have become essential enablers in mitigation and control of the complexity problem. Artificial intelligence and, in particular, reinforcement learning has shown great potential in learning complex tasks through observations. The majority of ongoing reinforcement learning research activities focus on single-Agent problem settings with an assumption of accessibility to a globally observable state and action space. In many real-world settings, such as LTE or 5G, decision making is distributed and there is often only local accessibility to the state space. In such settings, multi-Agent learning may be preferable, with the added challenge of ensuring that all agents collaboratively work towards achieving a common goal. We present a novel cooperative and distributed actor-critic multi-Agent reinforcement learning algorithm. We claim the approach is sample efficient, both in terms of selecting observation samples and in terms of assignment of credit between subsets of collaborating agents. 

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2022
Keywords
Machine learning, Radio resource scheduling, 5G mobile communication systems, Adaptive control systems, Decision making, Learning algorithms, Learning systems, Multi agent systems, Continuous reinforcement, Machine-learning, Mobile systems, Multi-agent approach, Radio resources, Reinforcement learnings, Resource-scheduling, Self-learning, State-space, Reinforcement learning
National Category
Computer Sciences
Identifiers
urn:nbn:se:ri:diva-62621 (URN)10.23919/CNSM55787.2022.9965060 (DOI)2-s2.0-85143886726 (Scopus ID)9783903176515 (ISBN)
Conference
18th International Conference of Network and Service Management, CNSM 2022, 31 October 2022 through 4 November 2022
Note

Funding text 1: Kreuger is partially funded by Ericsson. Corcoran and Boman are partially funded by the WASP (Wallenberg Autonomous Systems and Software Program) research program.

Available from: 2023-01-24 Created: 2023-01-24 Last updated: 2023-01-24Bibliographically approved
Kreuger, P. (2021). A generative mobility model.
Open this publication in new window or tab >>A generative mobility model
2021 (English)Report (Other academic)
Abstract [en]

Many applications in the mobile radio network domain employ simulations to explore e.g. parameter configurations, robustness of protocols and buffer allocation algorithms. User mobility is (together with traffic and radio propagation models), one of the main components of such simulations, and has large impact on e.g. load distribution, cell handover frequency, signal fading and interference. In many simulations, detailed user mobility is as crucial as the physical infrastructure, where the exact position affect fading and reflections, but in others, e.g. load balancing, handover management and radio resource scheduling, coarser models are often sufficient. But even in these cases, properties of the trajectories of individual users will affect the results of the simulation, e.g. where distribution of positions, rest times, and displacement magnitudes and velocities, need to be considered.

Publisher
p. 6
Series
RISE Rapport ; 2021:15
National Category
Telecommunications
Identifiers
urn:nbn:se:ri:diva-59202 (URN)978-91-89167-98-8 (ISBN)
Available from: 2022-05-18 Created: 2022-05-18 Last updated: 2022-05-18Bibliographically approved
Corcoran, D., Kreuger, P. & Boman, M. (2021). Reinforcement Learning for Automated Energy Efficient Mobile Network Performance Tuning. In: 2021 17th International Conference on Network and Service Management (CNSM): . Paper presented at 2021 17th International Conference on Network and Service Management (CNSM). 25-29 Oct. 2021. (pp. 216-224).
Open this publication in new window or tab >>Reinforcement Learning for Automated Energy Efficient Mobile Network Performance Tuning
2021 (English)In: 2021 17th International Conference on Network and Service Management (CNSM), 2021, p. 216-224Conference paper, Published paper (Refereed)
Abstract [en]

Modern mobile networks are increasingly complex from a resource management perspective, with diverse combinations of software, infrastructure elements and services that need to be configured and tuned for correct and efficient operation. It is well accepted in the communications community that appropriately dimensioned, efficient and reliable configurations of systems like 5G or indeed its predecessor 4G is a massive technical challenge. One promising avenue is the application of machine learning methods to apply a data-driven and continuous learning approach to automated system performance tuning. We demonstrate the effectiveness of policy-gradient reinforcement learning as a way to learn and apply complex interleaving patterns of radio resource block usage in 4G and 5G, in order to automate the reduction of cell edge interference. We show that our method can increase overall spectral efficiency up to 25% and increase the overall system energy efficiency up to 50% in very challenging scenarios by learning how to do more with less system resources. We also introduce a flexible phased and continuous learning approach that can be used to train a bootstrap model in a simulated environment after which the model is transferred to a live system for continuous contextual learning.

Keywords
5G mobile communication, Spectral efficiency, System performance, Reinforcement learning, Interference, Energy efficiency, Software, Communication system traffic, Machine learning, Learning systems, System simulation, Self-organization, Radio resource scheduling, Inter-cell interference coordination
National Category
Communication Systems
Identifiers
urn:nbn:se:ri:diva-57473 (URN)10.23919/CNSM52442.2021.9615550 (DOI)
Conference
2021 17th International Conference on Network and Service Management (CNSM). 25-29 Oct. 2021.
Available from: 2021-12-28 Created: 2021-12-28 Last updated: 2022-01-07Bibliographically approved
Corcoran, D., Kreuger, P. & Schulte, C. (2020). Efficient Real-Time Traffic Generation for 5G RAN. In: Proceedings of IEEE/IFIP Network Operations and Management Symposium 2020: Management in the Age of Softwarization and Artificial Intelligence, NOMS 2020. Paper presented at 2020 IEEE/IFIP Network Operations and Management Symposium, NOMS 2020, 20 April 2020 through 24 April 2020. Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>Efficient Real-Time Traffic Generation for 5G RAN
2020 (English)In: Proceedings of IEEE/IFIP Network Operations and Management Symposium 2020: Management in the Age of Softwarization and Artificial Intelligence, NOMS 2020, Institute of Electrical and Electronics Engineers Inc. , 2020Conference paper, Published paper (Refereed)
Abstract [en]

Modern telecommunication and mobile networks are increasingly complex from a resource management perspective, with diverse combinations of software and infrastructure elements that need to be configured and tuned for efficient operation with high quality of service. Increased real-time automation at all levels and time-frames is a critical tool in controlling this complexity. A key component in automation is practical and accurate simulation methods that can be used in live traffic scenarios. This paper introduces a new method with supporting algorithms for sampling key parameters from live or recorded traffic which can be used to generate large volumes of synthetic traffic with very similar rate distributions and temporal characteristics. Multiple spatial renewal processes are used to generate fractional Gaussian noise, which is scaled and transformed into a log-normal rate distribution with discrete arrival events, fitted to the properties observed in given recorded traces. This approach works well for modelling large user aggregates but is especially useful for medium sized and relatively small aggregates, where existing methods struggle to reproduce the most important properties of recorded traces. The technique is demonstrated through experimental comparisons with data collected from an operational LTE network to be highly useful in supporting self-learning and automation algorithms which can ultimately reduce complexity, increase energy efficiency, and reduce total network operation costs. 

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2020
Keywords
Communication system traffic, Fractals, Machine learning, Parametric statistics, System simulation, Aggregates, Automation, Complex networks, Energy efficiency, Gaussian noise (electronic), Mobile telecommunication systems, Quality of service, Automation algorithms, Experimental comparison, Fractional Gaussian noise, Rate distributions, Real time traffics, Real-time automation, Resource management, Temporal characteristics, 5G mobile communication systems
National Category
Natural Sciences
Identifiers
urn:nbn:se:ri:diva-45154 (URN)10.1109/NOMS47738.2020.9110314 (DOI)2-s2.0-85086765703 (Scopus ID)9781728149738 (ISBN)
Conference
2020 IEEE/IFIP Network Operations and Management Symposium, NOMS 2020, 20 April 2020 through 24 April 2020
Available from: 2020-07-13 Created: 2020-07-13 Last updated: 2020-12-01Bibliographically approved
Kreuger, P., Steinert, R., Görnerup, O. & Gillblad, D. (2018). Distributed dynamic load balancing with applications in radio access networks. International Journal of Network Management, 28(2)
Open this publication in new window or tab >>Distributed dynamic load balancing with applications in radio access networks
2018 (English)In: International Journal of Network Management, ISSN 1055-7148, E-ISSN 1099-1190, Vol. 28, no 2Article in journal (Refereed) Published
Abstract [en]

Managing and balancing load in distributed systems remains a challenging problem in resource management, especially in networked systems where scalability concerns favour distributed and dynamic approaches. Distributed methods can also integrate well with centralised control paradigms if they provide high-level usage statistics and control interfaces for supporting and deploying centralised policy decisions. We present a general method to compute target values for an arbitrary metric on the local system state and show that autonomous rebalancing actions based on the target values can be used to reliably and robustly improve the balance for metrics based on probabilistic risk estimates. To balance the trade-off between balancing efficiency and cost, we introduce 2 methods of deriving rebalancing actuations from the computed targets that depend on parameters that directly affects the trade-off. This enables policy level control of the distributed mechanism based on collected metric statistics from network elements. Evaluation results based on cellular radio access network simulations indicate that load balancing based on probabilistic overload risk metrics provides more robust balancing solutions with fewer handovers compared to a baseline setting based on average load.

Place, publisher, year, edition, pages
John Wiley & Sons, 2018
Keywords
Self-organising heterogeneous networks; Distributed dynamic load balancing; Methods/control theories; Network Management/Wireless & mobile networks
National Category
Computer Sciences
Identifiers
urn:nbn:se:ri:diva-32825 (URN)10.1002/nem.2014 (DOI)2-s2.0-85036539033 (Scopus ID)
Funder
Swedish Foundation for Strategic Research , RIT15-0075EU, Horizon 2020, 671639
Available from: 2017-12-05 Created: 2017-12-05 Last updated: 2023-06-02Bibliographically approved
Corcoran, D., Andimeh, L., Ermedahl, A., Kreuger, P. & Schulte, C. (2017). Data driven selection of DRX for energy efficient 5G RAN. In: : . Paper presented at 2017 13th International Conference on Network and Service Management (CNSM). 26-30 Nov. 2017. Tokyo, Japan. Tokyo: IEEE conference proceedings
Open this publication in new window or tab >>Data driven selection of DRX for energy efficient 5G RAN
Show others...
2017 (English)Conference paper, Published paper (Refereed)
Abstract [en]

The number of connected mobile devices is increasing rapidly with more than 10 billion expected by 2022. Their total aggregate energy consumption poses a significant concern to society. The current 3gpp (3rd Generation Partnership Project) LTE/LTE-Advanced standard incorporates an energy saving technique called discontinuous reception (DRX). It is expected that 5G will use an evolved variant of this scheme. In general, the single selection of DRX parameters per device is non trivial. This paper describes how to improve energy efficiency of mobile devices by selecting DRX based on the traffic profile per device. Our particular approach uses a two phase data-driven strategy which tunes the selection of DRX parameters based on a smart fast energy model. The first phase involves the off-line selection of viable DRX combinations for a particular traffic mix. The second phase involves an on-line selection of DRX from this viable list. The method attempts to guarantee that latency is not worse than a chosen threshold. Alternatively, longer battery life for a device can be traded against increased latency. We built a lab prototype of the system to verify that the technique works and scales on a real LTE system. We also designed a sophisticated traffic generator based on actual user data traces. Complementary method verification has been made by exhaustive off-line simulations on recorded LTE network data. Our approach shows significant device energy savings, which has the aggregated potential over billions of devices to make a real contribution to green, energy efficient networks.

Place, publisher, year, edition, pages
Tokyo: IEEE conference proceedings, 2017
Keywords
Software architecture, 5G mobile communication, Adaptive systems, Energy efficiency, Green computing
National Category
Computer Sciences
Identifiers
urn:nbn:se:ri:diva-33288 (URN)10.23919/CNSM.2017.8255972 (DOI)2-s2.0-85046680815 (Scopus ID)
Conference
2017 13th International Conference on Network and Service Management (CNSM). 26-30 Nov. 2017. Tokyo, Japan
Available from: 2018-02-20 Created: 2018-02-20 Last updated: 2018-12-19Bibliographically approved
Kreuger, P., Görnerup, O., Gillblad, D., Lundborg, T., Corcoran, D. & Ermedahl, A. (2015). Autonomous load balancing of heterogeneous networks (11ed.). In: 2015 IEEE 81st Vehicular Technology Conference (VTC Spring): . Paper presented at 81st IEEE Vehicular Technology Conference (VTC Spring 2015), May 11-14, 2015, Glasgow, UK. , Article ID 7145712.
Open this publication in new window or tab >>Autonomous load balancing of heterogeneous networks
Show others...
2015 (English)In: 2015 IEEE 81st Vehicular Technology Conference (VTC Spring), 2015, 11, article id 7145712Conference paper, Published paper (Refereed)
Abstract [en]

This paper presents a method for load balancing heterogeneous networks by dynamically assigning values to the LTE cell range expansion (CRE) parameter. The method records hand-over events online and adapts flexibly to changes in terminal traffic and mobility by maintaining statistical estimators that are used to support autonomous assignment decisions. The proposed approach has low overhead and is highly scalable due to a modularised and completely distributed design that exploits self- organisation based on local inter-cell interactions. An advanced simulator that incorporates terminal traffic patterns and mobility models with a radio access network simulator has been developed to validate and evaluate the method.

Series
IEEE Vehicular Technology Conference, ISSN 1550-2252
Keywords
autonomous network management, self-organising heterogenous networks, distributed algorithms, statistical modelling
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:ri:diva-24457 (URN)10.1109/VTCSpring.2015.7145712 (DOI)2-s2.0-84940399308 (Scopus ID)978-1-4799-8088-8 (ISBN)
Conference
81st IEEE Vehicular Technology Conference (VTC Spring 2015), May 11-14, 2015, Glasgow, UK
Projects
HetNet
Available from: 2016-10-31 Created: 2016-10-31 Last updated: 2023-06-02Bibliographically approved
Kreuger, P., Gillblad, D., Görnerup, O., Corcoran, D., Lundborg, T. & Ermedahl, A. (2015). Methods, Nodes and system for enabling redistribution of cell load (13ed.). .
Open this publication in new window or tab >>Methods, Nodes and system for enabling redistribution of cell load
Show others...
2015 (English)Patent (Other (popular science, discussion, etc.))
Abstract [en]

Patent for distributed load balancing mechanism for LTE, developed by SICS in collaboration with Ericsson DURA

Publisher
p. 49
Keywords
load balancing, cell range expansion, autonomous radio access management, distributed algorithms, LTE
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:ri:diva-24526 (URN)
Projects
HetNet
Available from: 2016-10-31 Created: 2016-10-31 Last updated: 2023-06-02Bibliographically approved
Nemeth, F., Steinert, R., Kreuger, P. & Sköldström, P. (2015). Roles of DevOps tools in an automated, dynamic service creation architecture (11ed.). In: 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM): . Paper presented at 14th IFIP/IEEE International Symposium on Integrated Network Management (IM 2015), May 11-15, 2015, Ottawa, Canada (pp. 1153-1154). , Article ID 7140455.
Open this publication in new window or tab >>Roles of DevOps tools in an automated, dynamic service creation architecture
2015 (English)In: 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM), 2015, 11, p. 1153-1154, article id 7140455Conference paper, Published paper (Refereed)
Abstract [en]

Software Defined Networking (SDN) and Network Functions Virtualization facilitate, with their advanced pro- grammability features, the design of automated dynamic service creation platforms. Applying DevOps principles to service design can further reduce service creation times and support continuous operation. Monitoring, troubleshooting, and other DevOps tools can have different roles within virtualised networks, depending on virtualization level, type of instantiation, and user intent. We have implemented and integrated four key DevOps tools that are useful in their own right, but showcase also an integrated scenario, where they form the basis for a more complete and realistic DevOps toolkit. The current set of tools include a message bus, a rudimentary configuration tool, a probabilistic congestion detector, and a watchpoint mechanism. The demo also presents potential roles and use-cases for the tools.

National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:ri:diva-24504 (URN)10.1109/INM.2015.7140455 (DOI)2-s2.0-84942627031 (Scopus ID)978-1-4799-8241-7 (ISBN)
Conference
14th IFIP/IEEE International Symposium on Integrated Network Management (IM 2015), May 11-15, 2015, Ottawa, Canada
Projects
UNIFY
Available from: 2016-10-31 Created: 2016-10-31 Last updated: 2020-12-01Bibliographically approved
Kreuger, P. & Steinert, R. (2015). Scalable in-network rate monitoring (6ed.). In: 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM): . Paper presented at 14th IFIP/IEEE International Symposium on Integrated Network Management (IM 2015), May 11-15, 2015, Ottawa, Canada (pp. 866-869). , Article ID 7140396.
Open this publication in new window or tab >>Scalable in-network rate monitoring
2015 (English)In: 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM), 2015, 6, p. 866-869, article id 7140396Conference paper, Published paper (Refereed)
Abstract [en]

We propose a highly scalable statistical method for modelling the monitored traffic rate in a network node and suggest a simple method for detecting increased risk of congestion at different monitoring time scales. The approach is based on parameter estimation of a lognormal distribution using the method of moments. The proposed method is computation- ally efficient and requires only two counters for updating the parameter estimates between consecutive inspections. Evaluation using a naive congestion detector with a success rate of over 98% indicates that our model can be used to detect episodes of high congestion risk at 0.3 s using estimates captured at 5 m intervals.

Keywords
probabilistic management, performance monitoring, statistical traffic analysis, link utilization modelling, congestion detection, in-network rate monitoring
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:ri:diva-24458 (URN)10.1109/INM.2015.7140396 (DOI)2-s2.0-84942673092 (Scopus ID)978-1-4799-8241-7 (ISBN)
Conference
14th IFIP/IEEE International Symposium on Integrated Network Management (IM 2015), May 11-15, 2015, Ottawa, Canada
Projects
Unify
Available from: 2016-10-31 Created: 2016-10-31 Last updated: 2020-12-01Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-9331-0352

Search in DiVA

Show all publications