World breakthrough in onboard AI model training presented by Φ-lab at IGARSS

Illustration of the training data for the model (left) and the resulting predictions from new data (right). Detected clouds are indicated with red dots.

At the International Geoscience and Remote Sensing Symposium (IGARSS) on 21 July, ESA Φ-lab presented the results of groundbreaking research in artificial intelligence (AI) aboard Earth observation satellites. Carried out by Oxford University and Trillium Technologies in collaboration with Φ-lab, the research successfully trained a cloud-detection Machine Learning model while in flight on a D-Orbit ION mission.

ESA has given considerable attention in recent years to Cognitive Cloud Computing in Space (3CS), with such initiatives as the Φsat missions and the 3CS Call for Ideas helping to push the boundaries of enabling ever-increasing computational power onboard Earth observation (EO) satellites. The D-Orbit Dashing Through the Stars mission was launched in January 2022 and included a number of service tests funded by ESA. Among these was a customisable Machine Learning (ML) payload which allows models to be uploaded, updated and run directly on the satellite.

Despite the models running on the satellite, model training would normally take place on the ground due to the computational load and the enormous datasets required. In a world first however, Trillium Technologies and Oxford University’s Department of Computer Science have trained one such model directly onboard during the D-Orbit mission. This development has momentous implications for remote sensing, not only in Earth Observation but also in deep space exploration, as onboard training allows models to adapt autonomously to both changing conditions in space and sensor calibration drift. This is all the more important when the communication channel between the spacecraft and ground is limited due to distance and/or bandwidth.

To overcome the resource constraints of onboard training for ML models, the research group adopted an innovative approach involving an initial ML model known as RaVAEn. This model specialises in efficiently compressing EO images to just a few kilobytes, reducing the data to a ‘latent space’ representation. The key advantage of this methodology lies in the ease with which the constrained satellite hardware can handle the compact latent space. Unlike the original large and data-intensive imagery, the compressed representation is significantly smaller and more manageable.

The full paper on the research will be published soon at ieeeigarss.org.