As heterogeneous VHF data exchange system (VDES) constellations use the same communication frequency and time division multiple access (TDMA) communication mechanism, a large number of co-channel interference is caused by slot conflicts in the overlapping areas of constellations，reducing the communication quality. To tackle this problem, an inter-constellation compatibility strategy for deep Q-network (DQN) is proposed. Based on the VDES communication process, the ship station is set as the resource information transfer node, which enables the satellite to perceive the communication environment. On this basis, the resource allocation problem in the heterogeneous constellation scenario is modeled as a reinforcement learning (RL) problem, and a DQN-based slot resource allocation algorithm is proposed. By reconstructing the historical and current resource information, the optimal slot resource allocation scheme is planned and the algorithm is iteratively optimized according to the results. Simulation results show that the proposed strategy can effectively enhance communication performance.
WANG Xuefan
,
LI Zongwang
,
LIANG Xuwen
. Compatible strategy of VDES heterogeneous constellation based on DQN[J]. Journal of University of Chinese Academy of Sciences, 2024
, 41(4)
: 550
-557
.
DOI: 10.7523/j.ucas.2022.071
[1] ITU-R. Recommendation ITU-R M.2092-1-Technical characteristics for a VHF data exchange system in the VHF maritime mobile band [S/OL]. Geneva: International Telecommunication Union:(2022-02-23)[2022-04-20].https://www.itu.int/rec/R-REC-M.2092-1-202202-I/en/.
[2] IALA. Guideline G1139 the technical specification of VDES [S/OL]. Saint Germain en Laye: International Association of Marine Aids to Navigation and Lighthouse Authorities:(2019-06-21)[2022-04-20].https://www.iala-aism.org/product/g1139-technical-specification-vdes/.
[3] 姚治萱. VDES通信技术应用及其发展趋势[J]. 世界海运, 2019, 42(2): 34-38. DOI:10.16176/j.cnki.21-1284.2019.02.007.
[4] 胡旭, 林彬, 王珍. 基于VDES的空天地海通信网络架构与关键技术[J]. 移动通信, 2019, 43(5): 1-8. DOI:10.3969/j.issn.1006-1010.2019.05.001.
[5] 王福斋, 胡青, 姚高乐, 等. 甚高频数字交换系统发展现状及推进工作建议[J]. 中国海事, 2021(2): 18-21. DOI:10.16831/j.cnki.issn1673-2278.2021.02.005.
[6] Wang Y F, Ding X J, Zhang G X. A novel dynamic spectrum-sharing method for GEO and LEO satellite networks[J]. IEEE Access, 2020, 8: 147895-147906. DOI:10.1109/ACCESS.2020.3015487.
[7] Gu P, Li R, Hua C Q, et al. Dynamic cooperative spectrum sharing in a multi-beam LEO-GEO co-existing satellite system[J]. IEEE Transactions on Wireless Communications, 2022, 21(2): 1170-1182. DOI:10.1109/TWC.2021.3102704.
[8] Gu P, Li R, Hua C Q, et al. Cooperative spectrum sharing in a co-existing LEO-GEO satellite system[C]//GLOBECOM 2020:2020 IEEE Global Communications Conference. December 7-11, 2020, Taipei, China. IEEE, 2020:1-6. DOI:10.1109/GLOBECOM42002.2020.9347950.
[9] Jia M, Li Z, Gu X M, et al. Joint multi-beam power control for LEO and GEO spectrum-sharing networks[C]//2021 IEEE/CIC International Conference on Communications in China (ICCC). July 28-30, 2021, Xiamen, China. IEEE, 2021: 841-846. DOI:10.1109/ICCC52777.2021.9580210.
[10] 李壮. 基于干扰控制的LEO和GEO频谱共享方法[D]. 哈尔滨: 哈尔滨工业大学, 2021.
[11] Bu G J, Jiang J. Reinforcement learning-based user scheduling and resource allocation for massive MU-MIMO system[C]//2019 IEEE/CIC International Conference on Communications in China (ICCC). August 11-13, 2019, Changchun, China. IEEE, 2019: 641-646. DOI:10.1109/ICCChina.2019.8855949.
[12] Li Z W, Xie Z C, Liang X W. Dynamic channel reservation strategy based on DQN algorithm for multi-service LEO satellite communication system[J]. IEEE Wireless Communications Letters, 2021, 10(4): 770-774. DOI:10.1109/LWC.2020.3043073.
[13] 李子煜. 多波束低轨卫星通信系统切换与资源管理算法研究[D].重庆:重庆邮电大学,2021.
[14] Seo J, Cho K, Cho W, et al. A discovery scheme based on carrier sensing in self-organizing bluetooth low energy networks[J]. Journal of Network and Computer Applications,2016,65:72-83. DOI:10.1016/j.jnca.2015.09.015.
[15] 韩存武, 周慧, 刘蕾, 等. 基于数据的无线通信网络功率和速率控制[J]. 计算机仿真, 2022, 39(2): 375-379, 511. DOI:10.3969/j.issn.1006-9348.2022.02.072.
[16] 曾欢, 张灿, 陈德元. 空间通信网中音视频传输的应用层QoS控制与测试方法[J]. 中国科学院研究生院学报, 2011, 28(1): 108-115. DOI:10.7523/j.issn.2095-6134.2011.1.016.
[17] Strehl A L, Li L H, Wiewiora E, et al. PAC model-free reinforcement learning[C]//ICML’06: Proceedings of the 23rd international conference on Machine learning. June 25-29, 2006, Pittsburgh, USA. New York: ACM, 2006: 881-888. DOI:10.1145/1143844.1143955.
[18] Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533. DOI:10.1038/nature14236.
