Vision-Assisted Beam Prediction for Real World 6G Drone Communication

The rapid evolution of drone communication systems necessitates the development of novel approaches for optimal beam management in millimetre wave (mmWave) 6G networks. Beamforming is used to improve signal quality and enhance the signal-to-noise ratio (SNR); however, the existing beam management performs an exhaustive search over the pre-defined codebook, resulting in higher latency due to training overhead that makes it impractical for high-mobility applications. Therefore, this paper introduces an innovative technique for mmWave beam prediction, considering practical visual and communication scenarios. The approach proposed in this study utilizes computer vision (CV) and ensemble learning via stacking, combining multi-modal vision sensing and positional data to achieve accurate estimations of drone positions and orientations. The developed framework first fine-tunes "you look only once"version 5 (YOLO-v5), a CV model to obtain the bounding box (location) of the drone from RGB images. This filtered vision sensing information and position data are used to train two different sets of neural networks, and the output of each model is stacked to train a meta-learner, used for the prediction of K-beams from a pre-defined codebook. The proposed method outperforms with the top-1 accuracy of approximately 90% compared to 86% and 60% for vision and position models, respectively. Furthermore, top-3 and top-5 accuracies are approximately 100%, resulting in a significant receive signal strength.