Requirements of compressed sensing (CS) techniques targeted at miniaturized hyperspectral satellite applications include lightweight onboard hardware, high-speed sensing, low sampling rate for compressing the massive volume of typical hyperspectral data, and noise robustness for reliable data transmission to the ground station. We achieve all these aims via deep learning, and neural networks resulted from which can be implemented on-chip, thereby allowing light hardware implementation.

Our neural networks were trained from small-scaled data, but, even so, the resulting encoder used in the CS stage achieves a very low sampling rate and very high speed. Unlike typical network training, the input/output pairs are not square but stripe-like images, partly because compressed acquisition does not allow performing compression after obtaining complete data cube, and partly because stripe-like acquisition well matches the widely adopted pushbroom hyperspectral sensing schemes. Even with such hard restriction caused by nontraditional training pairs, the resulting decoder still reconstructs the hyperspectral image with high accuracy. To match the requirement of pushbroom sensing, a lightweight encoder is proposed to compress the stripelike images immediately. Meanwhile, Multi-scale Feature fusion Block (MFB) and Aggregation (MFA) modules are proposed to form our decoder for enhancing the feature representation of the compressed acquisitions. Furthermore, we achieve joint spatial and spectral super-resolution progressively, ensuring accurate hyperspectral reconstruction via a low-rank-driven decoder. The encoder and decoder are trained in an end-to-end manner, where the noise-robustness is forced during the training stage.

Comprehensive experiments demonstrate the superiority of the proposed hyperspectral compressed sensing method, as well as its one-shot transfer learning (OTL) based extension, both quantitatively and qualitatively.

The overall flow is to compressively sense the hyperspectral data by our encoder (in real-time), and then to decode the compressed signal by our decoder. In order to be used on miniaturized satellite, a lightweight encoder with a stripe-like data sensing strategy is proposed. The network architecture of the proposed encoder is illustrated in left part of Fig.1. The encoder only stacked three convolutional layers consisting of one 3 x 3 and two 3 x 1 kernels with LeakyReLU activation function.

Instead of putting such complicated network components on the encoder, we remain most parts of feature representation learning on the decoder to boost the efficiency of the communication between the miniaturized satellite and the ground station.

Our decoder shown as right part of Fig.1, and it comprises four parts to progressively restore the spatial and spectral information from the noise-corrupted compressed data, to achieve joint spatial and spectral resolution enhancement. First, an effective backbone is selected to enrich the feature representation for the compressed signal. Next, we upscale the spatial resolution using transposed convolutional layers. Then, an extra convolutional layer is adopted to learn how to decode the spectral information from the output of the previous stage. Finally, the full spectral information can be super-resolved by the last convolutional layer.

In our MFB, each feature map will be squeezed by a convolutional layer named squeeze layer, followed by concatenating it to the next feature map. An example is illustrated in Fig. 2, in the second layer (say, green and blue blocks), the input feature map (green and blue) will be convoluted by a 3 x 3 kernel and followed by concatenating them to the next layers (brown, blue, and green blocks), where the feature map indicated in brown is equivalent to have a larger receptive field as it has passed through two 3 x 3 kernels. Therefore, we achieve the multi-scale feature aggregation and fusion module, by the nature of the equivalence between the large receptive field-wise feature map with one larger kernel size and the several stacked convolution operations with smaller kernel size.

Fig. 1: Network architecture of the proposed DCSN for hyperspectral compressed sensing.

 

Fig. 2: Network architecture of the proposed MFB block.