In the proposed 5G architecture where cell densification is expected to be used for network capacity enhancement, the deployment of millimetre wave (mmWave) massive multiple-input multiple-output (MIMO) in urban microcells located outdoor is expected to be used for high channel capacity small cell wireless traffic backhauling as the use of copper and optic-fibre cable becomes infeasible owing to the high cost and issues with right of way. The high cost of radio frequency (RF) chain and its prohibitive power consumption are big drawbacks for mmWave massive MIMO transceiver implementation and the complexity of using optimal detection algorithm as a result of inter-channel interference (ICI) as the base station antenna approaches large numbers. Spatial modulation (SM) and Generalized Spatial Modulation (GSM) are new novel techniques proposed as a low-complexity, low cost and low-power-consumption MIMO candidate with the ability to further reduce the RF chain for mmWave massive MIMO hybrid beamforming systems. In this work, we present the principles of generalized spatial modulation aided hybrid beamforming (GSMA-HBF) and its use for cost-effective, high energy efficient mmWave massive MIMO transceiver for small cell wireless backhaul in a 5G ultra-dense network.