Reinforcement Learning Method for Beam Management in Millimeter-Wave Networks

With the rapid growth of mobile data demand, the fifth generation (5G) mobile network must exploit the large amount of spectrum in the millimeter wave (mmWave) band to increase the network capacity. Due to the limitation of propagation distance, line-of-sight (LOS) link is highly desirable for mmWave systems. However, LOS channel is not feasible all the time and mmWave is also impacted significantly by the surrounding environment. The LOS signal can be easily blocked by surrounding buildings. Based on this issue, in this paper, we propose to use reinforcement learning to manage the non line of sight (NLOS) scenario. Specifically, we build a model simulating blocked LOS signal for the user equipment (UE) with only NLOS channel available for the UE. Q-Learning is used to select the NLOS beam that meets the UE's quality of service requirements. Simulation results show that Q-Learning can be used to manage the beam selection. In particular, at initial training stage the Q-Learning explores in the environment. However, with the training process, Q-Learning learns from experience and the received power increases significantly and converges to an excellent level.