Many inventories of street trees are recorded manually by an expert walking through cities and recording each tree with different attributes. This process is obviously expensive and doesn’t scale very well.

专家们在城市中穿行并记录具有不同属性的每棵树，从而手动记录了许多行道树的清单。 这个过程显然很昂贵，而且扩展性很差。

To address this problem, Wegner et al. (2016) proposed a way of automating the creation of urban tree inventories by using a deep learning model that detects and assigns geo-coordinates to trees from Street View images. However, in order to perform long-term tree health studies, with time-series starting before street-level imagery was available, we still rely on these traditional inventories.

为了解决这个问题，Wegner等人。 (2016)提出了一种通过使用深度学习模型自动创建城市树木清单的方法，该模型可以根据街景图像检测并为树木分配地理坐标。 但是，为了进行长期的树木健康研究，我们需要从获得街道级图像之前​​的时间序列开始， 我们仍然依赖这些传统清单 。

Most of the existing old databases have one crucial deficiency: the tree locations are recorded as street addresses instead of geographic coordinates.

大多数现有的旧数据库都有一个关键缺陷： 将树的位​​置记录为街道地址，而不是地理坐标。

This makes them much less accurate and harder to use in practice. The goal of our project is to address this challenge: we provide an automated way to assign geo-coordinates to old tree inventories using Street View images.

这使它们的准确性大大降低，在实践中更难使用。 我们项目的目标是解决这一挑战： 我们提供了一种自动方式，可以使用街景视图图像将地理坐标分配给老树清单。

The hope is that our approach on retrofitting existing street tree inventories with geographic coordinates will enable large-scale longitudinal studies, where data of the same population of trees can be analysed over decades.

希望我们采用地理坐标翻新现有街道树木清单的方法将能够进行大规模的纵向研究，在该研究中，可以对数十年内相同树木的数据进行分析。

为什么街上的树木很重要？ (Why are street trees important?)

Although street trees make up a small proportion of the urban forest, they provide considerable ecosystem services to the urban landscape. These benefits include for example improvement in air quality, a reduction of the heat island effect and increased carbon capture and storage. Additionally, they can contribute to rising property values and an improvement in individual and community wellbeing.

尽管街头树木仅占城市森林的一小部分，但它们为城市景观提供了可观的生态系统服务。 这些好处包括例如改善空气质量 ， 减少热岛效应以及增加碳捕获和储存 。 此外，它们还可以促进财产价值的增长以及个人和社区福祉的改善。

如何解决这样的问题？ (How to approach such a problem?)

Our idea: We want to use the existing tool developed by Wegner et al. (2016) to automatically find trees on Street View images and then match them to the trees in the old databases based on their addresses.

我们的想法：我们想使用Wegner等人开发的现有工具。 (2016)自动在街景图像上查找树木，然后根据其地址将其与旧数据库中的树木匹配。

Our general workflow is the following:

我们的一般工作流程如下：

1. The first step is to find some approximate coordinates for each tree by using a geocoding API. This means each address is converted from a textual address to a lat/long coordinate pair. We used the Geocoding API service provided by Google, which returns the geo-coordinates centered on the building of a particular parcel. This gives us a rough idea of where those trees are located.

   第一步是使用地理编码API 为每棵树找到一些近似坐标 。 这意味着每个地址都将从文本地址转换为经纬度坐标对。 我们使用了Google提供的Geocoding API服务，该服务返回以特定宗地建筑物为中心的地理坐标。 这使我们对这些树的位置有了一个大概的了解。

2. Next, we download the Street View images in the vicinity of those coordinates.

   接下来，我们下载这些坐标附近的街景图像 。

3. And detect the trees in these images using our trained deep neural network.

   并使用我们训练有素的深度神经网络检测这些图像中的树木 。

4. To get their coordinates, the detections are projected into geographical space.

   为了获得其坐标 ，将检测结果投影到地理空间中。

5. As a final step, we need to match the trees found in the images with the existing ones corresponding to the addresses, using a global optimization approach. This assignment enhances the old databases with the coordinate information from the detected trees.

   最后，我们需要使用全局优化方法将图像中找到的树与对应于地址的现有树进行匹配 。 这项任务利用来自检测到的树的坐标信息增强了旧数据库。

在街景图像中查找现有树木 (Finding existing trees in Street View images)

This part of the workflow follows the ideas proposed by Wegner et al. (2016) and Branson et al. (2018), with a simplified algorithm in order to make it easier to scale to huge datasets.

工作流的这一部分遵循Wegner等人提出的想法。 (2016)和Branson等人。 (2018)，采用简化算法，以便更轻松地扩展到庞大的数据集。

Trees in Street View panorama images are detected using Faster R-CNN (Ren et al., 2015), one of the most popular CNNs for object detection. The output of the tree detector is one bounding box per detected tree in each Street View panorama.

使用Faster R-CNN(Ren等人，2015)检测街景全景图像中的树木，Raster C-NN是用于对象检测的最受欢迎的CNN之一。 树木检测器的输出是每个“街景”全景图中每个检测到的树木的一个边界框。

Like all supervised learning algorithms, the network has to be trained with sufficient ground truth data. We have manually labeled 6783 individual tree instances in 718 Street View panoramas to get accurate and reliable results.

像所有监督学习算法一样，必须用足够的地面真实数据来训练网络。 我们在718 Street View全景图中手动标记了6783个单独的树实例，以获取准确可靠的结果。

After downloading all the images near an address, they are fed into the tree detector which outputs several bounding boxes per image.

在将所有图像下载到某个地址附近后，它们被馈送到树检测器，该检测器为每个图像输出几个边界框。

The metadata of the Street View images includes the camera position and heading, which lets us calculate the geographic coordinates of the detected trees by projecting the bottom center point of the bounding boxes.

街景图像的元数据包括摄像头的位置和航向 ，这使我们可以通过投影边界框的底部中心点来计算检测到的树木的地理坐标 。

使用全局优化匹配检测到的树和现有地址 (Matching detected trees and existing addresses using global optimization)

Now we have got a database of geocoded trees. The last step of the puzzle is to match them with the geocoded addresses from the existing inventories. We cast this matching task as an optimization problem where the total distance between all pairs of trees is minimized across an entire municipality. We want to solve the following problem:

现在我们有了一个地理编码树数据库。 难题的最后一步是将它们与现有库存中的地理编码地址进行匹配。 我们将此匹配任务视为优化问题，其中整个城市中所有树木对之间的总距离最小。 我们要解决以下问题：

• x_ij is our binary decision variable, indicating if an address i is matched to a tree j (1 if i is matched to j, 0 otherwise),

  x _ ij是我们的二进制决策变量，指示地址i是否与树j匹配(如果i与j匹配则为1，否则为0)，

• dist_ij is simply the geographical distance between address i and tree j,

  dist _ ij只是地址i与树j之间的地理距离，

• M is a distance threshold to avoid matching trees to addresses that are too far away (50 meters in our experiments),

  M是距离阈值，可避免将树与太远的地址(在我们的实验中为50米)匹配，

• We also introduce two constraints to ensure that only one street address is assigned per tree and that the number of assigned detected trees per street address does not exceed K, the total number of trees at that address given in the database.

  我们还引入了两个约束条件，以确保每棵树仅分配一个街道地址，并且确保每个街道地址分配的检测到的树木数量不超过K ，即数据库中给定该地址的树木总数。

By minimizing the sum over the misalignment between the geocoded trees and the geocoded addresses, we seek a globally optimal solution per city. The mathematical problem is solved using Linear Programming.

通过最小化地理编码树和地理编码地址之间未对齐的总和，我们寻求每个城市的全球最佳解决方案。 使用线性编程可以解决数学问题。

加利福尼亚城市 约有 1棵Mio树的结果 (Results for around ~1 Mio trees in Californian cities)

We ran experiments with tree inventories from five cities in California, which come with already existing coordinates such that we can validate our results. In total, our approach correctly assigned geographic coordinates to 22’363 individual street trees out of 57’938 input trees, which represents 38%. Results for the full processing pipeline are shown in the following figure:

我们对来自加利福尼亚五个城市的树木清单进行了实验，这些清单具有已经存在的坐标，因此我们可以验证我们的结果。 总体而言，我们的方法将正确的地理坐标分配给了57,938棵输入树中的22,363棵单独的街道树，占38％。 下图显示了整个处理流程的结果：

The green symbols denote the correctly assigned trees, the rest is categorized into the different error reasons. Those include erroneous geocoding of the addresses, missing Google Street View images, and incorrect assignments.

绿色符号表示正确分配的树，其余的分为不同的错误原因。 这些包括地址的错误地理编码，丢失的Google Street View图片以及错误的分配。

Particularly difficult cases are street addresses of large buildings with a high number of trees such as these two examples:

尤其困难的情况是具有大量树木的大型建筑物的街道地址，例如以下两个示例：

Our system comes at virtually no cost if we put aside costs for running computers and downloading Google images. Given appropriate hardware and a fast internet connection for downloading images, the method scales to arbitrarily large data sets.

如果我们撇开运行计算机和下载Google图片的费用，我们的系统几乎是免费的。 给定适当的硬件和用于下载图像的快速Internet连接，该方法可以扩展到任意大的数据集。

To demonstrate this, we ran our processing pipeline on 48 municipalities containing altogether 1’100’952 input trees. Our method assigned geographic coordinates to 56% of the individual street trees.

为了证明这一点， 我们在48个包含1,100'952输入树的城市上运行了处理管道。 我们的方法将地理坐标分配给了56％的街道树。

结论 (Conclusion)

We have presented a novel approach to assign geographic coordinates to street-trees given street addresses and Street View panorama images. We were able to assign correct geographic coordinates to 38% of the street trees, which is a good starting point for long-term studies on the ecosystem services value of street trees at large scale.

我们提供了一种新颖的方法，可以在给定街道地址和街景视图全景图像的情况下，将地理坐标分配给街道树。 我们能够为38％的街道树分配正确的地理坐标，这是长期研究街道树的生态系统服务价值的良好起点。

There are different reasons, which account for this percentage being relatively low. One could be the varying geocoding quality provided by the API. Also, the detection of trees can fail due to missing panoramas, occlusions or an error by the detector. The incorrect assignment is another major reason for mismatches, which is generally difficult to fix. It is due to mixing up trees close to parcel boundaries or also when several trees are on the same property, which is very often the case.

有不同的原因，导致该百分比相对较低。 一种可能是API提供的不同的地理编码质量。 此外，由于缺少全景图，遮挡或检测器错误，树木的检测可能会失败。 错误的分配是不匹配的另一个主要原因，通常不容易解决。 这是由于混合了靠近地块边界的树木，或者是由于几棵树木在同一属性上，通常是这种情况。

We hope this work encourages further research to generate longer time series (over several decades) of street-tree inventories with the ultimate goal of better understanding street tree population dynamics and corresponding changes in ecosystem services at a very large scale.

我们希望这项工作能够鼓励进行进一步的研究，以产生更长的时间序列(超过几十年)的行道树清单，其最终目标是更好地了解行道树种群动态以及大规模的生态系统服务的相应变化。

For the interested reader, we refer to our full research paper: here

对于感兴趣的读者，请参阅我们的完整研究论文： 此处

Laumer, D., Lang, N., van Doorn, N., Aodha, O.M., Perona, P., Wegner, J.D. “Geocoding of trees from street addresses and street-level images” ISPRS Journal of Photogrammetry and Remote Sensing 162(2020)

洛默(D.)，朗(N.)，朗(N.)，范多恩(N.)，密西西比州奥达(Aodha)，佩罗纳(P.)，韦格纳(Wegner)，京东(JD)“对街道地址和街道图像中的树木进行地理编码”， ISPRS摄影测量与遥感杂志 162 ( 2020)

Branson, S., Wegner, J., Hall, D., Lang, N., Schindler, K., Perona, P., 2018. From google maps to a fine-grained catalog of street trees. ISPRS J. Photogramm. Remote Sens. 135, 13–30.

Branson，S.，Wegner，J.，Hall，D.，Lang，N.，Schindler，K.，Perona，P.，2018年。从谷歌地图到街树的细粒度目录。 ISPRS J. Photogramm。 遥感135，13-30。

Ren, S., He, K., Girshick, R., Sun, J., 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems (NIPS).

Ren，S.，He，K.，Girshick，R.，Sun，J.，2015。更快的R-CNN：通过区域提议网络实现实时对象检测。 于：神经信息处理系统(NIPS)的进展。

Wegner, J.D., Branson, S., Hall, D., Schindler, K., Perona, P., 2016. Cataloging public objects using aerial and street-level images — urban trees. In: IEEE Conference on Computer Vision and Pattern Recognition.

韦格纳，法学博士，布兰森，S。，霍尔，D。，辛德勒，K。，佩罗纳，体育，2016年。使用空中和街道图像(城市树木)对公共对象进行分类。 在：IEEE计算机视觉和模式识别会议上。