Making satellite imagery easy-to-use: the MOSAIKS algorithm

Satellite imagery has become a valuable tool in global development: from environmental monitoring and disaster response to urban planning and agriculture. With more and more high-resolution satellite imagery available as open-source datasets, information about land usage and populations have become widely accessible. But this data also needs advanced analytical techniques to make sense of it.

Machine learning is one of these tools: we can use deep learning models to extract information from lots of satellite images, and quickly. In particular, we can even use them to spot complex patterns in the satellite images, and use them to make predictions. For example: we can predict land cover by training a model to classify images into forest, water or urban areas, which can then be used to track changes in land use patterns or environmental conditions.

While machine learning applied to satellite imagery can result in enormous impact in global development, it is difficult for stakeholders to actually use these tools, for several reasons: (1) satellite imagery is high volume – difficult to fetch from open source databases, store and analyse in reasonable time, (2) using machine learning to analyze them requires expert knowledge and (3) applying machine learning to satellite imagery requires expensive computing and storage capacity. This makes it prohibitively resource-intensive for most players in the development sector to use, despite how useful it could be for their work.

But Esther Rolf and colleagues came up with a method they call MOSAIKS, to solve this problem, along with pre-computed features and resources at MOSAIKS.org. With the help of IDinsight’s lean innovation grant, and a talented team of data scientists and engineers, we extend MOSAIKS to make it easier to use in international development (see this post for a discussion of potential uses of this tool). We’ll unpack this in two parts and explain what we learnt along the way: in this post, we explain how Rolf et. al set up a machine learning model that doesn’t require expert knowledge to use, and in a follow-up post, how we sped up fetching and analyzing satellite images and how we set up compute infrastructure and documentation that makes it easy for anyone to use our solution.

How to extract information from satellite images?

The challenge with using machine learning to extract information from satellite imagery is that you need a lot of compute power and expertise, and also that these models do not transfer knowledge across prediction problems well. What this means is that for example, if we train a machine learning model to predict forest cover from satellite imagery, we cannot easily re-use it to predict urban population density. This is precisely why ML for satellite imagery is so resource intensive: you have to pay the cost of training the model every time there is a new prediction problem!

Under the hood, in order to extract useful information (features) from the images, a machine learning model performs convolutions. We can think of convolutions as applying different kinds of filters to an image: so to get information about forest cover, the model would apply a “greenness” filter to a satellite image, and return a “negative” that shows which parts of the image are green and which parts are not. The model then computes the amount of greenness in the negative, and this is the feature or information from the image that is now relevant to predict forest cover. We usually train the ML model with labeled satellite image data, so that it learns the best greenness filter (and maybe even to compute the amount of greenness). But if we now wanted to use it to predict population density instead, we would have to train it to learn a different filter, since a greenness filter would no longer work so well in providing information about populations.

A simple model for featurizing satellite imagery

Rolf et al. show that there is a better way to do this: instead of training the best filter for each model/problem and extracting a feature from it, they instead sample a whole host of random filters (either from a Gaussian distribution, or by taking patches of the satellite images). Not surprisingly, the name of the featurization algorithm that provides the foundation of the MOSAIKS approach is called “random convolutional features.” These filters are chosen before they ever look at a specific prediction problem. They then compute features for each of these random filters, given a satellite image. Now, we can just use these features directly for any prediction problem!

Note that at no point in this process do we have to train a ML model. What’s more, if we know ahead of time that we’re going to work with a certain set of satellite images (e.g. because we are interested in projects in a certain geographical region), then we can just compute these filters / features and store them – this is a lot easier than storing the satellite images. The best part is that this works just as well as a trained machine learning model, and generalizes much better!

Why does it work though? – well, because of the power of statistics. If we choose enough random features, at least a subset of them will generate “negatives” that are relevant to a specific prediction problem. By replacing costly feature optimization with randomization, you get close to the same performance at much lower computational cost.

Is this all we need to work with satellite imagery?

This still leaves the problem of fetching, performing convolutions and storing the data while keeping time and compute resources limited, which we will cover in a subsequent post.

To sum up, we’ve shown you how we can use satellite imagery for problem solving in the development sector that doesn’t require heavy computing. The MOSAIKS algorithm is particularly useful in cases where data and computing resources are scarce, a problem that is ubiquitous in this sector. However, we look forward to satellite imagery (potentially helped along by the MOSAIKS algorithm and our open source package) becoming a dominant tool in helping improve people’s lives across different sectors in the development space.