A low cost multi-armed bandit algorithm for dense wireless network

doi:10.7523/j.ucas.2020.0011

Abstract

Abstract: In recent years, people's demand for mobile wireless services has been increasing. In order to meet this challenge, ultra-dense wireless networks are considered to be the infrastructure and important components of the next-generation wireless communication network. Massive deployment of small base stations can reduce the number of network users in each cell, which can in turn provide the users with high-speed and low-latency wireless service. However, the inevitable problem brought with it at the same time is that users will cause frequent network handover when choosing access to ensure that they can access the network with the best service provider. User association problem is often modeled as the online learning model. This paper aims to find an efficient online user association scheme to deal with the additional network performance loss caused by frequent handover. Based on the analysis of the multi-armed bandit (MAB) model, this paper proposes an improved algorithm based on the arm elimination strategy, and demonstrates the effectiveness of the algorithm through rigorous theoretical analysis and numerical simulation experiments.

Key words: online learning, user association, dense network, multi-armed bandit

CLC Number:

TN913

ZHAO Yao, LUO Xiliang. A low cost multi-armed bandit algorithm for dense wireless network[J]. Journal of University of Chinese Academy of Sciences, 2022, 39(3): 403-409.

References

[1] Boccardi F, Heath R W, Lozano A, et al. Five disruptive technology directions for 5G[J]. IEEE Communications Magazine, 1996, 52(2): 74-80.DOI:10.1109/MCOM.2014.6736746.
[2] Lee W, Cho D H. Enhanced group handover scheme in multiaccess networks[J]. IEEE Transactions on Vehicular Technology, 2011, 60(5): 2389-2395.DOI:10.1109/TVT.2011.2140386.
[3] Fischione C, Athanasiou G, Santucci F. Dynamic optimization of generalized least squares handover algorithms[J]. IEEE Transactions on Wireless Communications, 2014,13(3): 1235-1249.DOI:10.1109/TWC.2014.013014.121720.
[4] Guidolin F, Pappalardo I, Zanella A, et al. Context-aware handover policies in HetNets[J]. IEEE Transactions on Wireless Communications, 2016, 15(3): 1895-1906.DOI:10.1109/TWC.2015.2496958.
[5] Ye Q Y, Rong B Y, Chen Y D, et al. User association for load balancing in heterogeneous cellular networks[J]. IEEE Transactions on Wireless Communications, 2013, 12(6): 2706-2716.DOI:10.1109/TWC.2013.040413.120676.
[6] Videv S, Haas H. Energy-efficient scheduling and bandwidth-energy efficiency trade-off with low load [C]//2011 IEEE International Conference on Communications(ICC). June 5-9, 2011,Kyoto, Japan. IEEE, 2011: 1-5.DOI:10.1109/icc.2011.5962571.
[7] 孟庆民,赵媛媛,岳文静,等. 动态超密集网络中的Markov预测切换[J].通信学报, 2018, 39(10): 166-174.DOI:10.11959/j.issn.1000-436x.2018225.
[8] Wang Z, Li L H, Xu Y, et al. Handover control in wireless systems via asynchronous multiuser deep reinforcement learning[J]. IEEE Internet of Things Journal, 2018, 5(6): 4296-4307.DOI:10.1109/jiot.2018.2848295.
[9] Shen C, van der Schaar M. A learning approach to frequent handover mitigations in 3GPP mobility protocols [C]//2017 IEEE Wireless Communications and Networking Conference. March 19-22, 2017, San Francisco, CA, USA. IEEE, 2017: 1-6.DOI:10.1109/WCNC.2017.7925950.
[10] Zhou Y M, Shen C, van der Schaar M. A non-stationary online learning approach to mobility management[J]. IEEE Transactions on Wireless Communications, 2019, 18(2): 1434-1446.DOI:10.1109/TWC.2019.2893168.
[11] Bubeck S. Regret analysis of stochastic and nonstochastic multi-armed bandit problems[M]. Boston: Now Publishers Inc, 2012.DOI:10.1561/9781601986276.
[12] Auer P, Cesa-Bianchi N, Fischer P. Finite-time analysis of the multiarmed bandit problem[J]. Machine Learning, 2002,47: 235-256.DOI:10.1023/A:1013689704352.
[13] Sun Y X, Zhou S, Xu J. EMM: energy-aware mobility management for mobile edge computing in ultra dense networks[J]. IEEE Journal on Selected Areas in Communications, 2017, 35(11): 2637-2646.DOI:10.1109/JSAC.2017.2760160.
[14] Auer P, Ortner R. UCB revisited: improved regret bounds for the stochastic multi-armed bandit problem[J]. Periodica Mathematica Hungarica, 2010, 61(1/2): 55-65.DOI:10.1007/S10998-010-3055.6.