Estimating depth maps of images is of vital importance in computer vision and robotics. Benefiting from the development of deep learning, many methods have been proposed to estimate the depth map using a single input image. These methods can be deployed easily and used in a variety of applications such as visual odometry [1], [2], sensor fusion [3], and augmented reality [4].

Although generating impressive results, learning-based methods need a large amount of data for training. Per-pixel depth annotating of real-world images is almost impossible as LiDAR only provides sparse depth measurements, and time-of-flight (ToF) cameras have limited ranges. The KITTI stereo dataset [5] uses CAD models to densify depth measurements of cars but only contains hundreds of images. Yang et al. [6] propose DrivingStereo with more than 180k frames fused from multi-frame LiDAR measurements. Despite the high accuracy depth measurement in DrivingStereo, the density of annotated pixels is less than 15%. Recently, many self-supervised works [7]–[16] have been proposed to train networks using calibrated stereo images or monocular videos. These methods are built on the assumption that images from nearby views can be synthesized correctly if the scene geometry and camera motion are estimated correctly.

Compared with using active sensors (e.g., LiDAR and ToF), training with stereo images or monocular videos offers a number of advantages. First and foremost, training data can be captured much more easily without depth labeling, thus a larger variety of data can be used for training. Second, the depth of every pixel can be supervised by minimizing photometric errors. Despite the success of recent selfsupervised methods, the geometric view synthesize process needs camera intrinsic parameters, which requires offline calibration. Because of this, current self-supervised methods cannot utilize the almost unlimited videos from the internet and train a ‘universal’ monocular depth estimation model. To overcome the training data limitations, using networks pretrained on ImageNet [17] classification tasks as training initialization becomes a default choice.

The success of transferring ImageNet-pretrained models to other classification tasks has been widely demonstrated and studied [19], [20]. However, transferring networks trained on ImageNet classification tasks for dense depth map estimation has several potential problems. First, classification tasks emphasize semantic feature maps, while the spatial information is ignored. When transferred to spatial-sensitive tasks, e.g. keypoint detection, He et. al. [20] show that ImageNetpretrained networks have limited benefits. Secondly, images from the ImageNet dataset focus on object detection and are very different from images for depth estimation.

In this paper, we propose a novel pretraining task that uses wild videos from the internet to capture both semantic and spatial information. Based on the fact that the optical flow between two images is determined solely by the geometric structure and motions (both camera and object motion), we use a conditional encoder-decoder to separate structure information and motion information from uncalibrated videos.The structure information of the scene is encoded from a single image, and motion information is estimated from two adjacent images. The optical flow between two images is finally reconstructed by using the estimated structure information conditioned on the motion vector. Since optical flow can be supervised without camera intrinsic or extrinsic parameters, the pretraining task can utilize unlimited videos from the internet and learn a variety of structures. After the pretraining stage, the encoder network can be easily transferred to depth estimation tasks.

Extensive experiments are used to demonstrate the transferred performance of the proposed geometric-pretrained networks. In Figure 1, we show estimated depth maps of ImageNet-pretrained networks and geometric-pretrained networks. Compared with ImageNet-pretrained networks, the proposed networks generate more accuracy and sharper depth estimations and can be generalized to other scenes or datasets.

估计图像的深度图在计算机视觉和机器人技术中至关重要。受益于深度学习的大环境，许多方法已经被提出，通过单张图片的输入去估计其深度图。这些方法可以轻松部署并用于各种应用程序，例如视觉测程、传感融合以及增强现实。

尽管产生了令人印象深刻的结果，基于学习的方法需要大量的数据用于训练。对于现实生活中的图像的逐像素的深度标注几乎是不可能的，现有的方法有以下问题：(1)稀疏 (2)有范围限制。有人对车辆的深度测量进行增稠，但只有几百张图像；有人对图像进行高精度标注，但是标注像素的稠密度低于15%。最近，研究者们提出了许多自监督的工作使用带标注的立体图像或单目视频去训练网络。这些方法都建立在，如果场景几何和相机运动都能被正确估计，那么临近视图的图像可以被正确估计的假定之下。

相较于使用传感器，使用立体图像集或单目视频提供了一些列优势。(1) 训练数据可以不适用深度标注就被被轻松地获取，因此可以使用多种数据进行训练。(2) 每个像素的深度可以通过最小化光度误差被监督。尽管最近的自监督方法很成功，但几何视图合成过程需要相机内在参数，这需要离线标注， 因此目前的自监督方法无法利用无限制的来自互联网的视频来训练“通用”的单目深度估计模型。 为了克服训练数据的限制，使用在ImageNet分类任务上预训练的网络作为初始化成为了大家的默认选择。

ImageNet Pretraining针对其他分类任务的迁移学习已经被证实了是很大的成功。但是，ImageNet针对单目深度估计的迁移学习有几个潜在的问题。(1) 分类任务强调语义特征图，但是空间上的信息被忽略了。针对空间敏感的任务，ImageNet的预训练效果不显著。 (2) ImageNet上的图片集中在对象检测上，这和深度估计所使用的图片非常不一样。

本文中，我们提出了一个新奇的与训练任务，它使用来自互联网的视频去抓取语义信息和空间信息。由于两张图象之间的光流仅仅由几何结构和运动（相机运动和对象运动）决定，我们使用了一种条件编码-解码器来从未标注的视频中分离结构信息和运动信息。结构信息由单张图片编码得到，运动信息则由两个相邻的图像估计得到。两个图像之间的光流最终会以运动向量为条件使用估计的结构信息去重建。由于光流可以不需要相机内置或外置的参数被监督，预训练任务可以使用无限制的来自互联网的视频并且学习到多种结构类型。在预训练阶段结束后，编码网络可以被轻松的迁移到深度估计任务中。

实验证明，我们的模型....，我们提供了更准确、更锋利的深度估计，并且可以泛化到其他的场景或者数据集上。

对象在第  帧和第  帧间的运动齐次坐标

第  帧图像

We propose a simple but effective pretraining task, called geometric pretraining, that is designed for monocular depth learning. The pretraining task only requires uncalibrated monocular image sequences and thus can utilize unlimited resources from the Internet. Extensive experiments are used to prove that, in terms of depth transfer learning, the proposed geometric pretrained model performs better in accuracy, few-short learning, and generalization ability compared to ImageNet-pretrained networks. The pretraining task can also be used to generalize depth learning into specific scenes without the calibration or synchronization of cameras.

In the future, we plan to improve the pretraining tasks that can further decouple each independent moving object. The driving videos dataset will also be further expanded due to the observation that a larger dataset benefits the performance of transferred models.

我们为单目深度学习提出了而一个简单但有效的预训练任务，称为Geometric Pretraining。预训练任务只需要未标注的弹幕图像序列，因此可以利用无限制的来自于互联网的资源。大量的实验已经证明了我们提出的几何预训练模型，在深度估计迁移学习中，在精确度、小样本学习、泛化能力上相较于ImageNet Pretraining表现的更好。这个预训练任务还可以将深度学习泛化到具体的场景，而不需要相机的校准和同步。

未来，我们计划改进这个预训练任务去进一步将每个独立的移动对象解耦。行车记录视频数据集也将被进一步扩大，因为更大的数据集会带来更好的迁移学习效果。