End-to-End Optimized Non-Orthogonal Multicarrier Waveform Design via Deep Learning

This paper proposes a novel joint transceiver optimization framework for multi-carrier (MC) waveform design. Unlike conventional orthogonal frequency division multiplexing, which employs memoryless modulation and fixed inverse discrete Fourier transform-based waveform generation, our approach utilizes neural network (NN)-based modulation with memory and NN-driven waveform generation at the transmitter. On the receiver side, a large-kernel attention-based NN replaces the traditional demodulation process, effectively mitigating large-span inter-carrier interference. This architecture provides enhanced flexibility for MC waveform optimization, allowing better adaptation to spectral emission mask constraints and maximizing the utilization of allocated spectrum resources. Additionally, it achieves significant spectral efficiency gains across diverse channel conditions, including additive white Gaussian noise (AWGN) and linear time-varying (LTV) channels with delay and Doppler spread. Numerical evaluations demonstrate significant bit error rate performance improvements, with up to 10 dB signal-to-noise ratio gain in LTV channels and approximately 6 dB gain in AWGN channels, underscoring the superiority of the proposed framework over state-of-the-art schemes.