基于Q-learning的飞行自组织网络QoS路由方法

doi:10.7523/j.ucas.2020.0001

中国科学院大学学报 ›› 2022, Vol. 39 ›› Issue (1): 134-143.DOI: 10.7523/j.ucas.2020.0001

• 电子信息与计算机科学 • 上一篇

基于Q-learning的飞行自组织网络QoS路由方法

黄鑫陈^1,2, 陈光祖¹, 郑敏¹, 谭冲¹, 刘洪¹

1. 中国科学院上海微系统与信息技术研究所, 上海;
2. 中国科学院大学微电子学院, 北京 100049

收稿日期:2020-01-02 修回日期:2020-04-29 发布日期:2021-05-31
通讯作者: 黄鑫陈
基金资助:
中国科学院青年创新促进会（2018269）资助

Q-learning based QoS routing for high dynamic flying Ad Hoc networks

HUANG Xinchen^1,2, CHEN Guangzu¹, ZHENG Min¹, TAN Chong¹, LIU Hong¹

1. Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, Shanghai 200050, China;
2. School of Microelectronics, University of Chinese Academy of Sciences, Beijing 100049, China

Received:2020-01-02 Revised:2020-04-29 Published:2021-05-31

摘要/Abstract

摘要： 针对无人机自组网等高动态飞行自组织网络中，网络拓扑的快速变化导致通信链路断裂和路由重建频繁的问题，研究一种基于Q-learning的QoS （quality of service）路由方法。该方法以Q-learning强化学习框架为基础，将邻居节点数量、链路持续时间和链路可用带宽作为路由度量信息，设计一种提供QoS保证的Q-learning奖励函数。网络节点通过广播Hello消息交互各自的本地路由度量信息，邻居节点接收到Hello分组或者数据分组，根据奖励函数计算并更新Q值，待转发数据分组的节点根据其维护的Q值表智能选择下一跳转发节点。EXata无线网络仿真环境中的仿真结果表明，该方法能为高动态飞行自组织网络中的数据传输提供稳定性好、服务质量高的通信链路。

关键词: 飞行自组网, QoS路由, Q-learning, 链路可用带宽, 链路持续时间

Abstract: In high dynamic flying ad hoc networks (FANETs), such as UAV (unmanned aerial vehicle) ad hoc networks, the rapid change of network topology leads to the breakage of communication links and the frequent reconstruction of routes. To solve this problem, a QoS (quality of service) routing method based on Q-learning is studied. Based on the basic Q-learning framework, this method takes the number of neighbor nodes, link duration and link available bandwidth as routing metrics, and designs a Q-learning reward function to provide QoS guarantee. All nodes exchange local routing metrics information with neighbor nodes by broadcasting Hello messages and forwarding data packets. After receiving Hello packets or data packets, neighbor nodes calculate and update the Q value according to the reward function. Then one of neighbor nodes selects a next hop node to forward data packets intelligently according to the Q value table that it maintains. The simulation results in EXata simulator show that this method can provide stable and high QoS communication links for high dynamic flying ad hoc networks.

Key words: FANETs, QoS routing, Q-learning, link available bandwidth, link expiration time

中图分类号:

TN929.5

黄鑫陈, 陈光祖, 郑敏, 谭冲, 刘洪. 基于Q-learning的飞行自组织网络QoS路由方法[J]. 中国科学院大学学报, 2022, 39(1): 134-143.

HUANG Xinchen, CHEN Guangzu, ZHENG Min, TAN Chong, LIU Hong. Q-learning based QoS routing for high dynamic flying Ad Hoc networks[J]. Journal of University of Chinese Academy of Sciences, 2022, 39(1): 134-143.

参考文献

[1] Bekmezci I[DD(-*2]·, Sahingoz O K, Temel Ş. Flying ad-hoc networks (FANETs):a survey[J]. Ad Hoc Networks, 2013, 11(3):1254-1270.
[2] Moussaoui A, Boukeream A. A survey of routing protocols based on link-stability in mobile ad hoc networks[J]. Journal of Network and Computer Applications, 2015, 47:1-10.
[3] Fan X R, Cai W L, Lin J Y. A survey of routing protocols for highly dynamic mobile ad hoc networks[C]//2017 IEEE 17th International Conference on Communication Technology (ICCT). October 27-30, 2017, Chengdu, China. IEEE, 2017:1412-1417.
[4] 陈峥, 单剑锋, 俞能海. 移动Ad hoc网络的分簇后择路由协议[J]. 中国科学技术大学学报, 2008, 38(12):1372-1375.
[5] 张福全, 吴寅, 杨绪兵. 移动性度量的移动时延容忍网络路由策略[J]. 中国科学技术大学学报, 2019, 49(2):132-137.
[6] Chaudhari S S, Biradar R C. Survey of bandwidth estimation techniques in communication networks[J]. Wireless Personal Communications, 2015, 83(2):1425-1476.
[7] Chettibi S, Chikhi S. A survey of reinforcement learning based routing protocols for mobile ad-hoc networks[C]//Recent Trends in Wireless and Mobile Networks. CoNeCo 2011, WiMo 2011. Springer, Berlin, Heidelberg, 2011, 162:1-13.
[8] Alsheikh M A, Lin S W, Niyato D, et al. Machine learning in wireless sensor networks:algorithms, strategies, and applications[J]. IEEE Communications Surveys & Tutorials, 2014, 16(4):1996-2018.
[9] 张德干, 葛辉, 刘晓欢, 等. 一种基于Q-Learning策略的自适应移动物联网路由新算法[J]. 电子学报, 2018, 46(10):2325-2332.
[10] 李荥, 王芳, 景栋盛, 等. 一种基于Q学习的无线传感网络路由方法[J]. 计算技术与自动化, 2017, 36(2):155-160.
[11] 王月娟, 张苏宁, 吴水明, 等. 基于秩的Q-路由选择算法[J]. 计算机与现代化, 2018(10):1-5.
[12] Liang X D, Balasingham I, Byun S S. A reinforcement learning based routing protocol with QoS support for biomedical sensor networks[C]//2008 First International Symposium on Applied Sciences on Biomedical and Communication Technologies. October 25-28, 2008, Aalborg, Denmark. IEEE, 2008:1-5.
[13] Hu T S, Fei Y S. QELAR:A machine-learning-based adaptive routing protocol for energy-efficient and lifetime-extended underwater sensor networks[J]. IEEE Transactions on Mobile Computing, 2010, 9(6):796-809.
[14] 朱明, 雷磊, 朱钢, 等. 多跳无线DCF自组网MAC层可用带宽预测[J]. 小型微型计算机系统, 2016, 37(10):2217-2221.
[15] Chen Y J, Wang G J, Peng S C. Link lifetime-based segment-by-segment routing protocol in MANETs[C]//2008 IEEE International Symposium on Parallel and Distributed Processing with Applications. December 10-12, 2008, Sydney, NSW, Australia. IEEE, 2008:387-392.
[16] Waheed A, Wahid A, Shah M A. LAOD:link aware on demand routing in flying Ad-Hoc networks[C]//2019 IEEE International Conference on Communications Workshops (ICC Workshops). May 20-24, 2019, Shanghai, China. IEEE, 2019:1-5.
[17] Wang H P, Cui L. An enhanced AODV for mobile ad hoc network[C]//2008 International Conference on Machine Learning and Cybernetics. July 12-15, 2008, Kunming, China. IEEE, 2008, 2:1135-1140.

基于Q-learning的飞行自组织网络QoS路由方法

Q-learning based QoS routing for high dynamic flying Ad Hoc networks

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

访问统计

联系我们

[1]	靳晓晗, 岩延, 张宝贤. 基于深度强化学习方法的无线多跳网络能量高效机会路由^*[J]. 中国科学院大学学报, 0, (): 70-70.
[2]	杨特, 洪佩琳, 李润洲. 蜂窝网络下的SWIPT-D2D通信资源分配^*[J]. 中国科学院大学学报, 0, (): 533-533.
[3]	刘梦冰, 邱玲, 梁晓雯. 基于Matern硬核点过程的无人机辅助蜂窝网络吞吐量分析^*[J]. 中国科学院大学学报, 0, (): 106-106.
[4]	王虹, 徐佑宇, 谭冲, 刘洪, 郑敏. 基于改进粒子群的BP神经网络WSN数据融合算法[J]. 中国科学院大学学报, 2020, 37(5): 673-680.
[5]	李颖锐, 徐景, 刘响, 易辉跃. TMRC-Filter:一种基于Filtered OFDM系统的子带滤波器设计方法[J]. 中国科学院大学学报, 2020, 37(1): 120-127.
[6]	金叶奇, 郑敏, 谭冲, 刘洪. 面向蜂窝物联网的RB动态预分配调度方案[J]. 中国科学院大学学报, 2019, 36(6): 825-831.
[7]	李维谦, 邱玲. 支持D2D多播的蜂窝网络分簇策略与资源分配[J]. 中国科学院大学学报, 2019, 36(1): 137-143.
[8]	张剑, 邱玲, 陈正. 超密集网络中基于多连接的用户归属和功率控制联合优化[J]. 中国科学院大学学报, 2018, 35(1): 126-130.
[9]	薛滔, 邱玲. 蜂窝网络中机器类设备突发上行随机接入的优化[J]. 中国科学院大学学报, 2016, 33(5): 669-673.
[10]	房秋生, 洪佩琳. SDN网络中基于业务资源偏好的批路由策略[J]. 中国科学院大学学报, 2016, 33(4): 554-561.
[11]	王鑫, 邱玲. H2H与M2M共存场景的准入控制及资源分配[J]. 中国科学院大学学报, 2016, 33(3): 427-432.
[12]	余林浩, 卢汉成, 洪佩琳. 一种带QoS约束的资源分配和ABS功率控制算法[J]. 中国科学院大学学报, 2015, 32(5): 661-666.
[13]	王川, 邱玲. MIMO下行多用户中继系统的线性收发处理设计[J]. 中国科学院大学学报, 2015, 32(3): 378-383.
[14]	朱士坤, 李冬, 黄奎, 张宝贤. 一种高效的轻量级无线传感器网络文件系统设计与实现[J]. 中国科学院大学学报, 2014, 31(1): 130-134.
[15]	潘军, 张文逸. 家庭基站网络中基于移动和业务的接纳控制算法[J]. 中国科学院大学学报, 2014, 31(1): 139-144.