What do you call an awesome project that is never shared with the world? — You don’t. Unless the project is yours, you won’t ever know about its existence. Research is not only about having ideas and writing scientific articles. Research is about sharing those ideas, is about solving the everyday problems and aiming towards a better world and an inspiring future.

您如何称呼一个从未与世界共享的优秀项目？ —不。 除非项目是您的，否则您永远不会知道它的存在。 研究不仅涉及有想法和撰写科学文章。 研究是分享这些想法，是解决日常问题，并致力于建立一个更美好的世界和鼓舞人心的未来。

Over the past two weeks, I embraced this mission of putting a Deep Learning project into production and share it with the world. This project is a model to map regions affected by wildfires on a daily basis, using satellite images. How difficult can it be to put in production? Well, there are some considerations to keep in mind.

在过去的两个星期中，我接受了将深度学习项目投入生产并与世界分享的使命。 该项目是一个使用卫星图像每天绘制受野火影响区域的地图的模型。 投入生产有多困难？ 好吧，要记住一些注意事项。

表中的内容 (Table of content)

• Some background about the project有关项目的一些背景
• Downloading the data下载数据
• Preprocessing the data and generating the model outputs预处理数据并生成模型输出
• Creating and deploying a website with Django使用Django创建和部署网站

有关该项目的一些背景 (Some background about the project)

As I mentioned above, this project consists of a model to generate daily maps of regions burned by wildfires using satellite images as inputs. The research led to the publication of a scientific paper that I describe in the following two stories in case you are interested to find more details about the topic.

如上所述，该项目由一个模型组成，该模型使用卫星图像作为输入来生成野火燃烧区域的每日地图。 该研究导致发表了一篇 科学论文 ，如果您有兴趣查找有关该主题的更多详细信息，我将在以下两个故事中进行介绍。

下载数据 (Downloading the data)

Any model needs some input data to generate the outputs. In my case, the input data — the daily satellite images — need to be downloaded from a web server. This consists of using an API to make requests, wait for the requests to be processed and finally download the images. It’s very easy when everything goes well but often there are unexpected problems.

任何模型都需要一些输入数据来生成输出。 就我而言，需要从Web服务器下载输入数据(每日卫星图像)。 这包括使用API​​发出请求，等待请求被处理并最终下载图像。 当一切顺利时，这很容易，但是经常会出现意想不到的问题。

• The request to the server may take some time to be processed and there’s a limit of files per request;发送到服务器的请求可能要花一些时间，每个请求的文件数是有限的。
• The connection may fail and as a result, the download interrupted — leading to corrupted files;连接可能会失败，结果，下载将中断，从而导致文件损坏；
• The data may be delayed or missing for some unpredictable reason.由于某些不可预测的原因，数据可能会延迟或丢失。

To make the request/download process work robustly I had to write a Python package to manage the download process with features including:

为了使请求/下载过程稳健地工作，我不得不编写一个 Python软件包 来管理具有以下功能的下载过程：

• Splitting the request to the API in case the number of files is too large for a single request;如果单个请求的文件数量太大，则将请求拆分为API；
• Taking note of which files have been already downloaded;注意哪些文件已经下载；
• Checking the hash of the downloaded files to make sure they are not corrupted;检查下载文件的哈希值，以确保它们没有损坏；
• Being able to restart the process without having to make a new request in case the connection is lost.万一失去连接，就可以重新启动进程而不必发出新的请求。

It is time-consuming to test and debug all this code but a necessity when moving to production. The process needs to be as reliable as possible and requiring the least human intervention.

测试和调试所有这些代码非常耗时，但是在投入生产时则是必不可少的。 该过程必须尽可能可靠，并且需要最少的人工干预。

预处理数据并生成模型输出 (Preprocessing the data and generating the model outputs)

Fortunately, this step is more straight forward as the preprocessing functions used during research to prepare the data can be used. However, there are often some modifications required:

幸运的是，由于可以使用在研究中准备数据的预处理功能，因此这一步骤更为直接。 但是，通常需要进行一些修改：

• The code should be fast and memory-efficient to reduce the computing costs and processing time;该代码应快速且内存有效，以减少计算成本和处理时间；
• The code should be robust to problems such as missing data;该代码应对诸如数据丢失之类的问题具有鲁棒性。

• The code should be able to run either on CPU or GPU depending on the machine specifications — something that, fortunately, Pytorch makes very easy.

  该代码应能够根据计算机规格在CPU或GPU上运行-幸运的是， Pytorch使此操作非常容易。

After the preprocessing of the data, it’s time to load the model, generate the predictions and save the results. The results, however, need to be saved in a format that is friendly to the website. In my case this led to some additional steps:

在对数据进行预处理之后，该加载模型，生成预测并保存结果了。 但是，结果必须以对网站友好的格式保存 。 就我而言，这导致了一些附加步骤：

• Converting the image data to vectorial format and separating the individual burned regions.将图像数据转换为矢量格式，并分离各个刻录区域。
• Saving also GeoTiff images corresponding to each individual event.还保存与每个事件对应的GeoTiff图像。

The data can then be added to a database. I opted for using PostGIS that can handle geographical data types such as polygons and rasters.

然后可以将数据添加到数据库。 我选择使用可以处理地理数据类型(例如多边形和栅格)的PostGIS 。

In practice, I’m running the download and preprocessing steps on a Google Cloud Platform virtual machine. To reduce costs the instance is only turned on once every day to compute the outputs and send them to the server where the website is hosted.

实际上，我正在Google Cloud Platform虚拟机上运行下载和预处理步骤。 为了降低成本，该实例每天仅打开一次以计算输出并将其发送到托管网站的服务器。

使用Django创建和部署网站 (Creating and deploying a website with Django)

Django is a web framework for rapid development using Python. The problem is, two weeks ago I didn’t know how to use it. I read about it before but I never found the right opportunity to use it. I figured it could be a good and useful challenge. After all, learning is fun! Or at least it is fun after you finally understand how things work. The learning process was something like this:

Django是一个使用Python快速开发的网络框架。 问题是，两个星期前我不知道如何使用它。 我之前曾读过它，但从未找到使用它的正确机会。 我认为这可能是一个很好且有用的挑战。 毕竟，学习很有趣！ 至少在您最终了解事物的工作原理之后，这才是有趣的。 学习过程是这样的：

• On the first two days, I started by looking at some tutorials on YouTube;在头两天，我首先看了YouTube上的一些教程；
• As I learned the basics, I started to create my website project;在学习基础知识之后，我开始创建我的网站项目。
• The next few days I gradually added more features, as I started to understand how the framework works.接下来的几天，随着我开始了解框架的工作原理，我逐渐添加了更多功能。

This is a process characterized by a lot of Google searches, an unreasonable amount of open tabs in the browser, and patience — a lot of it.

这个过程的特点是Google搜索很多，浏览器中打开标签的数量不合理，而且耐心-很多。

The good thing is that the most I learn the more I realise the potential Django brings. Over time, the progress starts to accelerate — climbing the learning curve!

好消息是，我学到的知识越多，我就越能体会Django带来的潜力。 随着时间的流逝，进度开始加速- 攀登学习曲线！

After having a working example I decided it was time to deploy it. I found this process quite more challenging than anticipated. Often, tutorials on YouTube only cover the development part and lack an explanation about the deploying process. It took me about three days just to make this step work. I had to figure out what are Nginx and Gunicorn and how to make everything work. But well, like in everything, when the task is more challenging is when we learn the most.

在有了一个可行的示例之后，我决定是时候部署它了。 我发现此过程比预期的更具挑战性。 YouTube上的教程通常只涵盖开发部分，而缺乏对部署过程的解释。 我花了大约三天时间才使此步骤生效。 我必须弄清楚什么是Nginx和Gunicorn，以及如何使一切正常 。 但是，就像在所有事情中一样， 当任务更具挑战性时，就是我们学到最多的东西。

Enough talk! Let me show you the webpage (meteo-ai.com) and describe the main functionality.

聊够了！ 让我向您展示该网页( meteo-ai.com )并描述主要功能。

Nice features the website has:

网站的不错功能：

• Burned areas data and active fires data are stored in a PostGIS database that plays well with GeoDjango;

  燃烧区数据和活跃火灾数据存储在PostGIS数据库中，该数据库可与GeoDjango配合使用 ；

• The user can select the time range for the visualization that will result in a query for the database;用户可以选择可视化的时间范围，以查询数据库。

• Results on the website are displayed on Leaflet maps, making use of the nice marker cluster extension that allows to aggregate close markers in a clean and efficient way (see the map in Figure 2).

  网站上的结果显示在Leaflet地图上，利用了不错的标记簇扩展 ，该扩展允许以干净有效的方式聚合关闭标记(请参见图2中的映射)。

• The burned area maps are displayed by rendering each pixel as a polygon, using the dates of burning or the confidence level of the model to colour the regions. This allows me to have a colourmap spanning the range selected by the user (see the map in Figure 1);

  通过使用燃烧日期或模型的置信度为每个区域着色，将每个像素渲染为多边形来显示燃烧区域图。 这使我可以得到一个色彩图，该色彩图跨越了用户选择的范围 (请参见图1中的图)。

• Statistics of the selected time period or fire event are shown with dynamical plots made with Plotly.js (see the bar plots in both Figures 1 and 2).

  使用Plotly.js绘制的动态图显示了所选时间段或火灾事件的统计信息 (请参见图1和2中的条形图)。

Being able to aggregate all this information in a visual and dynamical form is quite useful for monitoring and studying the fires.

能够以视觉和动态的形式汇总所有这些信息对于监视和研究火灾非常有用。

There are webpages showing a similar type of information, however, the burned areas produced by this Deep Learning model are state-of-the-art for this type of product. By analysing the spatio-temporal correlations in sequences of input images with 3D convolutions and an LSTM layer, the trained model is very good at identifying burned regions and determining at which day the pixel burned — with particular emphasis for the latter.

有些网页显示了类似的信息，但是，这种深度学习模型产生的烧伤区域是此类产品的最新技术。 通过使用3D卷积和LSTM层分析输入图像序列中的时空相关性， 训练后的模型非常擅长识别燃烧的区域并确定像素在哪一天燃烧 -特别是后者。

结束？ (The end?)

Not at all. There are many aspects to improve in the current state of the website. Furthermore, there are many more research paths to further explore in the burned areas field. One hot topic is higher resolution (30m pixel size or less) burned area maps using publicly available satellite images. The high spatial resolution trades off with less frequent images. This is a problem I’m working on. Working with such big volumes of data is a challenge by itself! I will make sure to write about the results in the future.

一点也不。 网站的当前状态有许多方面需要改进。 此外， 在燃烧区领域还有更多的研究途径可进一步探索 。 一个热门话题是使用公开提供的卫星图像，以更高的分辨率(小于或等于30m像素大小)刻录区域图。 高空间分辨率与频率较低的图像权衡。 这是我正在解决的问题。 处理如此大量的数据本身就是一个挑战！ 我一定会在以后写结果。

Maybe this is just the beginning.

也许这仅仅是开始。

关于我 (About me)

Thanks for reading! Have a great day!

谢谢阅读！ 祝你有美好的一天！