Artificial intelligence (AI) is paving the way for a new era of algorithms focusing directly on the information contained in the data, autonomously extracting relevant features for a given application. While the initial paradigm was to have these applications run by a server hosted processor, recent advances in microelectronics provide hardware accelerators with an efficient ratio between computation and energy consumption, enabling the implementation of AI algorithms “at the edge.” In this way only the meaningful and useful data are transmitted to the end-user, minimizing the required data bandwidth, and reducing the latency with respect to the cloud computing model. In recent years, European Space Agency (ESA) is promoting the development of disruptive innovative technologies on-board earth observation (EO) missions. In this field, the most advanced experiment to date is the Φ -sat-1, which has demonstrated the potential of artificial intelligence (AI) as a reliable and accurate tool for cloud detection on-board a hyperspectral imaging mission. The activities involved included demonstrating the robustness of the Intel Movidius Myriad 2 hardware accelerator against ionizing radiation, developing a Cloudscout segmentation neural network (NN), run on Myriad 2, to identify, classify, and eventually discard on-board the cloudy images, and assessing the innovative Hyperscout-2 hyperspectral sensor. This mission represents the first official attempt to successfully run an AI deep convolutional NN (CNN) directly inferencing on a dedicated accelerator on-board a satellite, opening the way for a new era of discovery and commercial applications driven by the deployment of on-board AI.

With the adoption of edge AI processors for space, on-orbit inference on EO data has become a possibility. This enables a range of new applications for space-based EO systems. Since the development of on-orbit AI applications requires rarely available raw data, training of these AI networks remains a challenge. To address this issue, we investigate the effects of varying two key image parameters between training and testing data on a ship segmentation network: Ground Sampling Distance and band misalignment magnitude. Our results show that for both parameters the network exhibits degraded performance if these parameters differ in testing data with respect to training data. We show that this performance drop can be mitigated with appropriate data augmentation. By preparing models at the training stage for the appropriate feature space, the need for additional computational resources on-board for e.g. image scaling or band-alignment of camera data can be mitigated.

Deep Space missions can benefit from onboard image analysis. We demonstrate deep learning inference to facilitate future mission adoption of said algorithms. Traditional space flight hardware provides modest compute when compared to today’s laptop and desktop computers. New generations of commercial off the shelf (COTS) processors designed for embedded applications, such as the Qualcomm Snapdragon and Movidius Myriad X, deliver significant compute in small Size Weight and Power (SWaP) packaging and offer direct hardware acceleration for deep neural networks. We deploy neural network models on these processors hosted by Hewlett Packard Enterprise’s Spaceborne Computer-2 onboard the International Space Station (ISS). We benchmark a variety of algorithms trained on imagery from Earth or Mars, as well as some standard deep learning models for image classification.

Future space missions can benefit from processing imagery onboard to detect science events, create insights, and respond autonomously. This capability can enable the discovery of new science. One of the challenges to this mission concept is that traditional space flight hardware has limited capabilities and is derived from much older computing in order to ensure reliable performance in the extreme environments of space, particularly radiation. Modern Commercial Off The Shelf (COTS) processors, such as the Movidius Myriad X and the Qualcomm Snapdragon, provide significant improvements in small Size Weight and Power (SWaP) packaging. They offer direct hardware acceleration for deep neural networks, which are state-of-the art in computer vision. We deploy neural network models on these processors hosted by Hewlett Packard Enterprise’s Spaceborne Computer-2 onboard the International Space Station (ISS).We benchmark a variety of algorithms on these processors. The models are run multiple times on the ISS to see if any errors develop. In addition, we run a memory checker to detect radiation effects on the embedded processors.