To reduce the costs of task processing, task offloading is put forward as a promising technology in mobile edge computing. In this paper, in order to lessen the burden of long-term tasks of the moving user, we utilize Markov decision process to formulate the task offloading problem as a stochastic programming problem, when taking the user mobility and system dynamics into account. According to the system information state, the problem can be categorized as the one with fully known system information and the other one with limited system information. We provide the optimal and learning-based task offloading algorithms under these two kinds of systems respectively. Furthermore, two learning-based algorithms, one with optimality and another with faster convergence rate, are proposed. The performance is verified in simulations.
LIU Ting
,
LUO Xiliang
. Online task offloading in mobile edge computing[J]. Journal of University of Chinese Academy of Sciences, 2022
, 39(2)
: 267
-274
.
DOI: 10.7523/j.ucas.2020.0012
[1] Li Y T, Cheng Q F, Liu X M, et al. A secure anonymous identity-based scheme in new authentication architecture for mobile edge computing[J]. IEEE Systems Journal, 2021, DOI: 10.1109/JSYST.2020.2979006.
[2] Dinh T Q, Tang J H, La Q D, et al. Offloading in mobile edge computing: task allocation and computational frequency scaling[J]. IEEE Transactions on Communications, 2017, 65(8): 3571-3584.DOI:10.1109/TCOMM.2017.2699660.
[3] Mao Y Y, You C S, Zhang J, et al. A survey on mobile edge computing: the communication perspective[J]. IEEE Communications Surveys & Tutorials, 2017, 19(4): 2322-2358.DOI:10.1109/COMST.2017.2745201.
[4] Mach P, Becvar Z. Mobile edge computing: a survey on architecture and computation offloading[J]. IEEE Communications Surveys & Tutorials, 2017, 19(3): 1628-1656.DOI:10.1109/COMST.2017.26823.
[5] You C S, Huang K B, Chae H, et al. Energy-efficient resource allocation for mobile-edge computation offloading[J]. IEEE Transactions on Wireless Communications, 2017, 16(3): 1397-1411.DOI:10.1109/TWC.2016.2633522.
[6] Mao Y Y, Zhang J, Song S H, et al. Stochastic joint radio and computational resource management for multi-user mobile-edge computing systems[J]. IEEE Transactions on Wireless Communications, 2017, 16(9): 5994-6009.DOI:10.1109/TWC.2017.2717986.
[7] Chen X F, Zhang H G, Wu C, et al. Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning[J]. IEEE Internet of Things Journal, 2019, 6(3): 4005-4018.DOI:10.1109/JIOT.2018.2876279.
[8] Min M H, Xiao L, Chen Y, et al. Learning-based computation offloading for IoT devices with energy harvesting[J]. IEEE Transactions on Vehicular Technology, 2019, 68(2): 1930-1941.DOI:10.1109/TVT.2018.2890685.
[9] Huang L, Bi S Z, Zhang Y J A. Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks[J]. IEEE Transactions on Mobile Computing, 2020,19(11):2581-2593. DOI: 10.1109/TMC.2019.2928811.
[10] 余翀,邱其文. 基于栅格地图的分层式机器人路径规划算法[J]. 中国科学院大学学报, 2013, 30(4): 528-538, 546.DOI:10.7523/j.issn.2095-6134.2013.04.015.
[11] Cheung M H, Huang J W. DAWN: delay-aware Wi-Fi offloading and network selection[J]. IEEE Journal on Selected Areas in Communications, 2015, 33(6): 1214-1223.DOI:10.1109/JSAC.2015.2416989.
[12] Nicholson A J, Noble B D. BreadCrumbs: forecasting mobile connectivity[C]//MobiCom ′08:Proceedings of the 14th ACM International Conference on Mobile Computing and Networking. 2008: 46-57.DOI:10.1145/1409944.1409952.
[13] Gambs S, Killijian M O, del Prado Cortez M N. Next place prediction using mobility Markov chains[C]//MPM ′12:Proceedings of the First Workshop on Measurement, Privacy, and Mobility. 2012: 1-6.DOI:10.1145/2181196.2181199.
[14] Sutton R S, Barto A G. Reinforcement learning: an introduction[M]. 2nd ed.Cambridge,MA:The MIT Press,2018.
[15] Watkins C J C H, Dayan P. Q-learning[J]. Machine Learning, 1992, 8(3/4): 279-292.DOI:10.1007/BF00992698.
[16] Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533.DOI:10.1038/nature14236.
[17] Zhu Z W, Jin S D, Yang Y, et al. Time reusing in D2D-enabled cooperative networks[J]. IEEE Transactions on Wireless Communications, 2018, 17(5): 3185-3200.DOI:10.1109/twc.2018.2808259.
[18] Abadi M, Barham P, Chen J M, et al. Tensorflow: a system for large-scale machine learning[J]. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 2016,abs/1605.08695.
[19] Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks[C]//Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. 2011: 315-323.
