异构VDES(VHF data exchange system)星座采用相同的通信频率和时分多址通信机制，使得异构星座重复覆盖区域内存在大量由时隙冲突造成的同频干扰，严重影响通信质量。针对此问题，提出一种基于深度Q网络(DQN)的星座间兼容策略。基于VDES通信流程，设置船站作为资源信息中转节点，赋予卫星对通信环境的感知能力。在此基础上，将异构星座场景下的资源分配问题建模为强化学习问题，提出一种基于DQN的时隙资源分配算法。通过重构历史资源信息和当前资源信息，规划最优时隙资源分配方案，并根据结果对算法迭代优化。仿真结果表明，所提出的策略可以有效提高通信性能。
As heterogeneous VHF data exchange system (VDES) constellations use the same communication frequency and time division multiple access (TDMA) communication mechanism, a large number of co-channel interference is caused by slot conflicts in the overlapping areas of constellations，reducing the communication quality. To tackle this problem, an inter-constellation compatibility strategy for deep Q-network (DQN) is proposed. Based on the VDES communication process, the ship station is set as the resource information transfer node, which enables the satellite to perceive the communication environment. On this basis, the resource allocation problem in the heterogeneous constellation scenario is modeled as a reinforcement learning (RL) problem, and a DQN-based slot resource allocation algorithm is proposed. By reconstructing the historical and current resource information, the optimal slot resource allocation scheme is planned and the algorithm is iteratively optimized according to the results. Simulation results show that the proposed strategy can effectively enhance communication performance.
[1] ITU-R. Recommendation ITU-R M.2092-1-Technical characteristics for a VHF data exchange system in the VHF maritime mobile band [S/OL]. Geneva: International Telecommunication Union:(2022-02-23)[2022-04-20].https://www.itu.int/rec/R-REC-M.2092-1-202202-I/en/.
[2] IALA. Guideline G1139 the technical specification of VDES [S/OL]. Saint Germain en Laye: International Association of Marine Aids to Navigation and Lighthouse Authorities:(2019-06-21)[2022-04-20].https://www.iala-aism.org/product/g1139-technical-specification-vdes/.
[3] 姚治萱. VDES通信技术应用及其发展趋势[J]. 世界海运, 2019, 42(2): 34-38. DOI:10.16176/j.cnki.21-1284.2019.02.007.
[4] 胡旭, 林彬, 王珍. 基于VDES的空天地海通信网络架构与关键技术[J]. 移动通信, 2019, 43(5): 1-8. DOI:10.3969/j.issn.1006-1010.2019.05.001.
[5] 王福斋, 胡青, 姚高乐, 等. 甚高频数字交换系统发展现状及推进工作建议[J]. 中国海事, 2021(2): 18-21. DOI:10.16831/j.cnki.issn1673-2278.2021.02.005.
[6] Wang Y F, Ding X J, Zhang G X. A novel dynamic spectrum-sharing method for GEO and LEO satellite networks[J]. IEEE Access, 2020, 8: 147895-147906. DOI:10.1109/ACCESS.2020.3015487.
[7] Gu P, Li R, Hua C Q, et al. Dynamic cooperative spectrum sharing in a multi-beam LEO-GEO co-existing satellite system[J]. IEEE Transactions on Wireless Communications, 2022, 21(2): 1170-1182. DOI:10.1109/TWC.2021.3102704.
[8] Gu P, Li R, Hua C Q, et al. Cooperative spectrum sharing in a co-existing LEO-GEO satellite system[C]//GLOBECOM 2020:2020 IEEE Global Communications Conference. December 7-11, 2020, Taipei, China. IEEE, 2020:1-6. DOI:10.1109/GLOBECOM42002.2020.9347950.
[9] Jia M, Li Z, Gu X M, et al. Joint multi-beam power control for LEO and GEO spectrum-sharing networks[C]//2021 IEEE/CIC International Conference on Communications in China (ICCC). July 28-30, 2021, Xiamen, China. IEEE, 2021: 841-846. DOI:10.1109/ICCC52777.2021.9580210.
[10] 李壮. 基于干扰控制的LEO和GEO频谱共享方法[D]. 哈尔滨: 哈尔滨工业大学, 2021.
[11] Bu G J, Jiang J. Reinforcement learning-based user scheduling and resource allocation for massive MU-MIMO system[C]//2019 IEEE/CIC International Conference on Communications in China (ICCC). August 11-13, 2019, Changchun, China. IEEE, 2019: 641-646. DOI:10.1109/ICCChina.2019.8855949.
[12] Li Z W, Xie Z C, Liang X W. Dynamic channel reservation strategy based on DQN algorithm for multi-service LEO satellite communication system[J]. IEEE Wireless Communications Letters, 2021, 10(4): 770-774. DOI:10.1109/LWC.2020.3043073.
[13] 李子煜. 多波束低轨卫星通信系统切换与资源管理算法研究[D].重庆:重庆邮电大学,2021.
[14] Seo J, Cho K, Cho W, et al. A discovery scheme based on carrier sensing in self-organizing bluetooth low energy networks[J]. Journal of Network and Computer Applications,2016,65:72-83. DOI:10.1016/j.jnca.2015.09.015.
[15] 韩存武, 周慧, 刘蕾, 等. 基于数据的无线通信网络功率和速率控制[J]. 计算机仿真, 2022, 39(2): 375-379, 511. DOI:10.3969/j.issn.1006-9348.2022.02.072.
[16] 曾欢, 张灿, 陈德元. 空间通信网中音视频传输的应用层QoS控制与测试方法[J]. 中国科学院研究生院学报, 2011, 28(1): 108-115. DOI:10.7523/j.issn.2095-6134.2011.1.016.
[17] Strehl A L, Li L H, Wiewiora E, et al. PAC model-free reinforcement learning[C]//ICML’06: Proceedings of the 23rd international conference on Machine learning. June 25-29, 2006, Pittsburgh, USA. New York: ACM, 2006: 881-888. DOI:10.1145/1143844.1143955.
[18] Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533. DOI:10.1038/nature14236.
