使用OSM数据(Working with OSM data)

This article will show you how you can set up your own custom styled OpenStreetMap tile server with Docker. We will also set up a build script that will automatically rebuild our custom Docker image each time we make changes and deploy the changes directly to our Kubernetes cluster.

Ť他的文章将告诉你如何建立与码头工人自己的自定义风格的瓷砖OpenStreetMap的服务器。 我们还将设置一个构建脚本，该脚本将在每次进行更改并将其直接部署到Kubernetes集群时自动重建我们的自定义Docker映像。

In this example, we will be using Google Cloud, Kubernetes, Docker, and Bitbucket pipelines, but the process is the same for all environments that can run Docker.

在此示例中，我们将使用Google Cloud，Kubernetes，Docker和Bitbucket管道，但是对于可以运行Docker的所有环境，该过程都是相同的。

最终产品 (The final product)

When we are done with this tutorial, we will have a website that shows our beautifully custom styled OSM map. By hosting an OSM map in our own cluster, we can potentially save some money compared to using a map provider like Mapbox.

完成本教程后，我们将拥有一个网站，该网站显示了我们漂亮的自定义样式的OSM映射。 通过在我们自己的集群中托管OSM地图，与使用诸如Mapbox之类的地图提供程序相比，我们可以节省一些钱。

0.您需要什么(0. What you need)

Before we can start, here is what you will need:

在开始之前，您需要满足以下条件：

• A running Kubernetes cluster on Google Cloud with an existing workload, where we will run our future Docker containerGoogle Cloud上具有现有工作负载的Kubernetes集群正在运行，我们将在此运行未来的Docker容器
• Access to a repository that can be connected to a build system like Bitbucket Pipelines访问可以连接到Bitbucket Pipelines等构建系统的存储库

1.项目设置(1. Project setup)

First, we need to add some files to our existing repository.

首先，我们需要向现有存储库中添加一些文件。

1. We will be building a Docker image, so we need a Dockerfile.我们将构建一个Docker映像，因此我们需要一个Dockerfile。
2. We will add some files to the Docker image, that will define our style for the map.我们将向Docker映像添加一些文件，这些文件将定义地图的样式。

3. What would our map be without map data? In this example we will directly add the map data to our repository. That way we avoid build errors if our map download source is temporarily offline.

   没有地图数据，我们的地图将会是什么？ 在此示例中，我们将直接将地图数据添加到我们的存储库中。 这样，如果我们的地图下载源暂时处于离线状态，我们将避免构建错误。

Let’s work through the list:

让我们来研究一下清单：

1.1添加Dockerfile (1.1 Add the Dockerfile)

Let's add an empty file called “Dockerfile” to our repo. We will take care of the content later.

让我们在仓库中添加一个名为“ Dockerfile”的空文件。 稍后我们将处理内容。

1.2添加地图样式基础文件 (1.2 Add the map style base files)

To find the correct files for defining our style, we need to start digging a bit.

为了找到正确的文件来定义我们的样式，我们需要进行一些挖掘。

The Docker container that we will later extend from is called overv/openstreetmap-tile-server:1.4.0. If you check the Dockerfile contents of this Docker container, you can see that it uses a framework called “carto” to compile all .mss style files into one XML called mapnik.xml. The same line references a file called project.mmlthat seems to tie all the .mss files together.

我们稍后将扩展的Docker容器称为overv/openstreetmap-tile-server:1.4.0 。 如果检查此Docker容器的Dockerfile内容，则可以看到它使用名为“ carto”的框架将所有.mss样式文件编译为一个名为mapnik.xml XML。 同一行引用了一个名为project.mml的文件，该文件似乎将所有.mss文件捆绑在一起。

overv/openstreetmap-tile-server:1.4.0 internally uses carto version 4.23.0. When you check the GitHub repository for that tag, you can see that all the files we need are just sitting on the root of that project. Download the whole project and add all these .mss files and the one .mmlfiles to our repo. We will modify those 19 files later to suit our needs. Do not copy anything else from that repository, we only need the .mss files and the one .mml file.

overv/openstreetmap-tile-server:1.4.0内部使用carto版本4.23.0。 当您在GitHub存储库中查看该标签时，您会看到我们需要的所有文件都位于该项目的根目录下。 下载整个项目，然后将所有这些.mss文件和一个.mml文件添加到我们的.mml 。 我们将在以后修改这19个文件以适合我们的需求。 不要从该存储库复制任何其他内容，我们只需要.mss文件和一个.mml文件。

In this example, we will put all of them into a sub-folder called “carto-style”. The path of our style files will therefore be:

在此示例中，我们将所有这些文件放入一个名为“ carto-style”的子文件夹中。 因此，样式文件的路径为：

• [PROJECT ROOT]/carto-style/*[项目根] / carto-style / *

1.3添加地图数据 (1.3 Add map data)

It is possible with overv/openstreetmap-tile-server:1.4.0 to automatically download and import map files, but we will add the map files directly to our custom Docker container, so that we can skip the download step when the container boots up. Having the map data directly in the repo also makes our build more robust in case the download source is offline while we are building a new Docker container.

overv/openstreetmap-tile-server:1.4.0可以自动下载和导入地图文件，但是我们会将地图文件直接添加到自定义Docker容器中，以便在容器启动时可以跳过下载步骤。 如果在构建新的Docker容器时下载源处于离线状态，则将地图数据直接存储在存储库中也使我们的构建更加可靠。

Let’s add the needed files. We need a .poly file and a .odf file of the map that we want to display. In this example, we will be rendering a map of Iceland.The needed files can be found on the website of geofabrik.de. Here are e.g. all files that cover Europe.

让我们添加所需的文件。 我们需要一个.poly文件和.odf ，我们要显示的地图文件。 在这个例子中，我们将绘制地图Iceland.The需要的文件都可以在网站上找到geofabrik.de 。 例如，这是涉及欧洲的所有文件。

Download the files for the area you want to cover and put them into the repository. We will save them in a sub-folder called “map-data”. So the path for the files in this example will be:

下载要覆盖的区域的文件，然后将其放入存储库中。 我们将它们保存在一个名为“ map-data”的子文件夹中。 因此，此示例中文件的路径为：

• [PROJECT ROOT]/map-data/iceland-latest.osm.pbf[项目根目录] /map-data/iceland-latest.osm.pbf
• [PROJECT ROOT]/map-data/iceland.poly[项目根] /map-data/iceland.poly

2.让我们构建一个新的Docker映像 (2. Let’s build a new Docker image)

Now that we have the project setup done, we can start building our future Docker container and then modify it’s content as we need them.

现在我们已经完成了项目设置，我们可以开始构建将来的Docker容器，然后根据需要修改其内容。

If you are not familiar with Docker, no worries. We will extend an existing Docker image and overwrite some files inside the parent’s file system.

如果您不熟悉Docker，请不用担心。 我们将扩展现有的Docker映像并覆盖父级文件系统中的某些文件。

What we are doing in our Dockerfile is basically redoing some of the steps from the original Dockerfile from overv/openstreetmap-tile-server:1.4.0. We just replace all the style files with our own style files, add our own map data, and then compile the style files again.

我们在Dockerfile中所做的基本上是从overv/openstreetmap-tile-server:1.4.0重做原始Dockerfile中的一些步骤。 我们只是将所有样式文件替换为我们自己的样式文件，添加我们自己的地图数据，然后再次编译样式文件。

See the final Dockerfile bellow and have a look at the explanatory comments inside.

请参阅最后的Dockerfile波纹管，并查看其中的解释性注释。

FROM overv/openstreetmap-tile-server:1.4.0
EXPOSE 80
# Step 1: remove all original style files
RUN rm -rf /home/renderer/src/openstreetmap-carto/*.mss
RUN rm -rf /home/renderer/src/openstreetmap-carto/project.mml# Step 2: add our custom style files
ADD carto-style /home/renderer/src/openstreetmap-carto# Step 3: recompile the stylesheet
RUN cd /home/renderer/src/openstreetmap-carto \&& carto project.mml > mapnik.xml \&& scripts/get-shapefiles.py# Step 4: See https://github.com/Overv/openstreetmap-tile-server
ADD map-data/iceland.poly /data.poly
ADD map-data/iceland-latest.osm.pbf /data.osm.pbf

After the docker build ran successfully, you will have a ready-to-go Docker file with your map data and your style.

成功运行docker构建之后，您将拥有一个随时可用的Docker文件，其中包含您的地图数据和样式。

3.自动部署切片服务器 (3. Deploy the tile server automatically)

We are going to set up a simple script in bitbucket pipelines that builds our Docker image and deploys it directly into our existing Kubernetes cluster in Google Cloud.

我们将在bitbucket管道中建立一个简单的脚本，该脚本可构建Docker映像并将其直接部署到Google Cloud中现有的Kubernetes集群中。

In this example, I am using the public Stokkur Build Environment container. It’s a container that already has gcloud, docker and kubectl installed. You can of course also use your own container or install gcloud, docker, and kubectl during the build.

在此示例中，我使用了公共的Stokkur Build Environment容器。 这是一个已经有一个容器gcloud ，码头工人和kubectl安装。 您当然也可以在构建过程中使用自己的容器或安装gcloud，docker和kubectl。

The build script will look somewhat like this:

构建脚本将看起来像这样：

image: stokkur/backend-build-environment:latestoptions:docker: truepipelines:branches:master:- step:name: Build and Deployscript:# Prepare Cloud Login- echo ${GOOGLE_CLIENT_SECRET} > client-secret.json# Define name for the new Docker container- export IMAGE_NAME=eu.gcr.io/myProject/osm-tile-server:v$BITBUCKET_BUILD_NUMBER- gcloud auth configure-docker --quiet# Build the container- docker build -t $IMAGE_NAME .# Google Cloud login- gcloud auth activate-service-account --key-file client-secret.json# Push the new Docker image into the private container registry in Google Cloud- docker push $IMAGE_NAME# Login into the target kubernetes cluster- gcloud container clusters get-credentials my-cluster --zone europe-west1-d --project my-project# Change the Docker image for the existing workload "osm-tile-server"- kubectl set image deployment/osm-tile-server osm-tile-server=$IMAGE_NAME

Please note that GOOGLE_CLIENT_SECRET contains the JSON key for a service account that you need to create in your google cloud project. The GOOGLE_CLIENT_SECRET variable is an environment variable defined for the repository on bitbucket.

请注意， GOOGLE_CLIENT_SECRET包含您需要在Google云端项目中创建的服务帐户的JSON密钥。 GOOGLE_CLIENT_SECRET变量是为bitbucket上的存储库定义的环境变量。

Also, note that we deploy the container into an existing cluster and an existing workload. See chapter 3.1 in this article for more info about that.

另外，请注意，我们将容器部署到现有集群和现有工作负载中。 有关更多信息，请参见本文中的第3.1章。

3.1 Kubernetes工作负载配置 (3.1 Kubernetes Workload configuration)

If you don’t know how to create a cluster and a workload, check this help page.

如果您不知道如何创建集群和工作负载，请查看此帮助页面。

The most important part of our workload in Kubernetes is the startup command. Without the correct command, our container won’t do anything.

Kubernetes中我们工作量中最重要的部分是启动命令。 没有正确的命令，我们的容器将无法执行任何操作。

On start-up, we want our container to import our custom map data and then serve the map. It is possible to define multiple startup commands for a container. In our case, we need to run import and after that run.

在启动时，我们希望容器导入我们的自定义地图数据，然后提供地图。 可以为一个容器定义多个启动命令。 在我们的例子中，我们需要运行import ，然后run 。

In this example, we will also configure to use 20 threads for rendering map tiles and allow CORS. The part of the workloads YAML file that defines the crucial parts will look like this:

在此示例中，我们还将配置为使用20个线程来渲染地图图块并允许CORS。 工作负载YAML文件中定义关键部分的部分如下所示：

spec:containers:- command:- /bin/sh- -c- |/run.sh import/run.sh runenv:- name: ALLOW_CORSvalue: enabled- name: THREADSvalue: "20"

Also, don’t forget to set up an Ingress and allow traffic to reach your workload, otherwise, you can’t access your map.

另外，别忘了设置Ingress并允许流量到达您的工作量，否则，您将无法访问地图。

Now that we have the build and the deployment process ready, we can start customizing our map.

现在我们已经准备好构建和部署过程，我们可以开始自定义地图了。

4.给地图一些样式 (4. Give the map some styling)

It’s finally time to have a look at how we can draw our map as we want it.

现在是时候看看我们如何根据需要绘制地图了。

From the carto GitHub README:

从carto GitHub README ：

These stylesheets can be used in your own cartography projects, and are designed to be easily customised. They work with Kosmtik and also with the command-line CartoCSS processor.

这些样式表可以在自己的制图项目中使用，并且可以轻松自定义。 它们与Kosmtik以及命令行CartoCSS处理器一起使用。

I won’t go too deep into this topic, as this topic would probably deserve its own article. So, here are the basics:

我不会对此主题做过深入的介绍，因为该主题可能值得您自己撰写。 因此，这里是基础知识：

• The .mss files define the look of the map just like CSS files. This is the place where you define e.g. what colors your forests and waters will have.

  .mss文件与CSS文件一样，定义了地图的外观。 在这里您可以定义例如森林和水的颜色。

• The .mml file ties all .mss files together and defines what layers your map will have. So if you e.g. want to hide all forests, go into this file and remove the part that renders forests.

  .mml文件将所有.mss文件.mss在一起，并定义了地图将具有的图层。 因此，例如，如果您想隐藏所有森林，请进入该文件并删除渲染森林的部分。

After you changed some styles, just commit them and bitbucket pipelines will rebuild and redeploy your container to the cloud.

更改某些样式后，只需提交它们，bitbucket管道将重新构建并将容器重新部署到云中。

5.想法和改进空间 (5. Thoughts and room for improvement)

5.1样式(5.1 Styling)

We covered a lot in this article, and especially the part about using carto and doing the actual styling of the map requires the most time. You will see that the styling itself seems easy at first, but gets more and more complex the further you differ from the default styling.

我们在本文中介绍了很多内容，尤其是有关使用carto和进行地图实际样式化的部分最多。 您会发现样式本身乍一看似乎很容易，但是与默认样式的不同之处在于样式变得越来越复杂。

I also did not find any good documentation about how to work with the carto styles. Maybe there is a generator somewhere that could take most of the manual work away from the developer and just generate those .mss files based on your wishes.

我也没有找到任何有关如何使用carto样式的好的文档。 也许某个地方有一个生成器，可以使开发人员摆脱大部分手动工作，而只是根据您的意愿生成那些.mss文件。

5.1容器启动 (5.1 Container Startup)

As you have probably already noticed by now, the startup duration of the container can take a long time. Importing the map data over and over again takes time and it took between 10 and 15 minutes each time I changed some styling for my own styled map until I saw the changes live on the web. I don’t know how much time it would take to import the map for bigger countries than Iceland, like Germany or France.

您可能已经注意到，容器的启动时间可能会花费很长时间。 一遍又一遍地导入地图数据需要花费时间，每次我更改自己的样式地图的样式时，都要花10到15分钟，直到我看到更改实时在网络上为止。 我不知道将地图导入到比冰岛更大的国家(例如德国或法国)需要多少时间。

There is still a lot of room for improvement here, maybe it is possible to upload a pre-initialized docker image to google cloud. If that can be done, then our container would only need to be run inside google cloud and we could save the importing time.

这里仍然有很多改进的空间，也许可以将预先初始化的docker映像上传到Google Cloud。 如果可以做到，那么我们的容器只需要在Google Cloud中运行，就可以节省导入时间。

5.2永久存储 (5.2 Persistent Storage)

As the setup is right now, we store the rendered map tiles inside the running Docker image. As a consequence of that, the image will blow up in memory and storage over time. To avoid this, I recommend using persistent storage for the OSM database and for the map tiles. Instructions about that can be found here.

由于设置现在已经完成，我们将渲染的地图图块存储在正在运行的Docker映像中。 结果，随着时间的推移，图像将在内存和存储中爆炸。 为避免这种情况，我建议对OSM数据库和地图图块使用永久性存储。 有关说明，请参见此处。

资料和进一步阅读 (Sources & Further Reading)

• Map Data: https://www.openstreetmap.org/

  地图数据： https : //www.openstreetmap.org/

• Map Data download: http://download.geofabrik.de/

  地图数据下载： http : //download.geofabrik.de/

• OpenStreetMap Tile Server Docker: https://github.com/Overv/openstreetmap-tile-server/tree/v1.4.0

  OpenStreetMap Tile Server Docker： https : //github.com/Overv/openstreetmap-tile-server/tree/v1.4.0

• Google Cloud: https://cloud.google.com/

  Google云端： https : //cloud.google.com/

• Google Cloud CLI: https://cloud.google.com/sdk/gcloud

  Google Cloud CLI： https ： //cloud.google.com/sdk/gcloud

• Kubernetes Project: https://kubernetes.io/

  Kubernetes项目： https ://kubernetes.io/

• Docker Docs: https://docs.docker.com/

  Docker文档： https : //docs.docker.com/

• Carto: https://github.com/gravitystorm/openstreetmap-carto/tree/v4.23.0

  卡托： https : //github.com/gravitystorm/openstreetmap-carto/tree/v4.23.0