gepc 骨架图算法Graph Embedded Pose Clustering

骨架图算法

Graph Embedded Pose Clustering for Anomaly Detection

paper	code
https://arxiv.org/abs/1912.11850	https://github.com/amirmk89/gepc

我们提出了一种用于人类行为异常检测的新方法。我们的方法直接适用于可以从输入视频序列计算的人体姿势图。这使得分析独立于扰动参数，如视点或照明。我们将这些图映射到一个潜在空间并将它们聚类。然后，每个操作都由其对每个聚类的软赋值来表示。这为数据提供了一种“词袋”表示，其中每个动作都由其与一组基本动作词的相似性来表示。然后，我们使用基于狄利克雷过程的混合物，这对于处理比例数据（例如我们的软赋值向量）很有用，以确定一个动作是否正常。

首先，我们对输入数据使用人体姿态检测器。这抽象了问题，并防止下一步处理诸如视点或照明变化等有害参数。人的行为被表示为时空图，我们将其嵌入（第3.1、3.2小节）并聚类（第3.3小节）到一些潜在空间中。现在，每个动作都表示为一组基本动作的软分配向量。这抽象了动作的基本类型（即细粒度或粗粒度），从而进入学习其分布的最后阶段。我们用于学习软分配向量分布的工具是Dirichlet过程混合（第3.4小节），我们将模型拟合到数据中。然后使用该模型确定动作是否正常。

图的每个节点对应于一个关键点、一个身体关节，每个边表示两个节点之间的某种关系。 存在许多"关键点关系"，如解剖学上定义的物理关系（例如，左手腕和肘部连接）和由运动定义的动作关系，这些运动往往在特定动作的上下文中高度相关（例如，跑步时左右膝盖倾向于朝相反方向移动）。图的方向来自于这样一个事实，即一些关系是在优化过程中学习的，并且不是对称的。这种表示的一个好处是紧凑，这对于高效的视频分析非常重要。
为了在时间上扩展，将从视频序列中提取的姿势关键点表示为姿势图的时间序列。 时间姿势图是人体关节位置的时间序列。时域邻接可以类似地通过连接连续帧中的关节来定义，允许我们利用姿势图序列的空间和时间维度执行图卷积运算

我们提出了一种基于深度时态图自动编码器的结构，用于嵌入时态姿态图。 基于图2所示ST-GCN的基本块设计，我们将基本GCN算子替换为新的空间注意力图卷积，如下所示。

3.2. Spatial Attention Graph Convolution

我们提出了一个新的图算子，如图3所示，它使用三种类型的邻接矩阵：静态、全局学习和推断（基于注意力）。每个邻接类型使用单独的权重应用其自己的GCN。

GCN的输出按通道维度堆叠。采用1×1卷积作为加权叠加输出的可学习缩减度量，并提供所需的输出信道数。

三个邻接矩阵捕捉了模型的不同方面：
（i）使用身体部位连通性作为优先于节点关系，使用静态邻接矩阵表示。
（ii）由全局邻接矩阵捕获的数据集级关键点关系，以及
（iii）由推断邻接矩阵获取的样本特定关系。最后，可学习约简度量对不同的输出进行加权

后续段落介绍了静态、全局学习和推断的邻接矩阵的设置方法，即图3中的A,B和C，在此略过。

3.3. Deep Embedded Clustering

为了构建我们的底层动作词典，我们采用训练集样本，并将它们联合嵌入和聚类到一些潜在空间中。然后，每个样本由其分配给每个底层聚类的概率表示。选择目标是为了提供不同的潜在集群，这些集群上存在动作。

我们采用了深嵌入聚类的概念[32]，用我们的ST-GCAE架构对时间图进行聚类。所提出的聚类模型由编码器、解码器和软聚类层三部分组成。

具体地说，我们的ST-GCAE模型保持了图的结构，但使用了较大的时间步长和不断增加的通道数来将输入序列压缩为潜在向量。解码器使用时间上采样层和额外的图卷积块，用于逐渐恢复原始信道计数和时间维度。

ST-GCAE的嵌入是数据聚类的起点。在我们的聚类优化阶段，对基于重构的初始嵌入进行微调，以达到最终的聚类优化嵌入。

符号	表示
$x_i$	输入示例
$z_i$	编码器的潜在嵌入
$y_i$	使用聚类层计算的软聚类分配
$Θ$	聚类层的参数
$p_{ik}$	probability for the i-th sample to be assigned to the k-th cluster

我们采用[32]提出的聚类目标和优化算法。聚类目标是最小化当前模型概率聚类预测P和目标分布Q之间的KL散度：

目标分布旨在通过标准化和将每个值推到更接近0或1的值来加强当前的群集分配。反复应用将P转换为Q的函数将最终导致硬分配向量。使用以下等式计算目标分布的每个成员：

聚类层由为编码训练集计算的K均值质心初始化。优化以期望最大化（EM）的方式进行。
在期望步骤期间，整个模型是固定的，并且目标分布Q被更新。在最大化阶段，优化模型以最小化聚类损失Lcluster。

    @staticmethoddef target_distribution(q):weight = q ** 2 / q.sum(0)w = weight.t() / weight.sum(1)w = w.t()return w

3.4. Normality Scoring

该模型支持两种类型的多模分布。一个是集群分配级别；另一个是在软分配向量级别。例如，一个动作可能被分配给多个集群（集群级分配），导致多模式软分配向量。
软分配向量本身（捕获动作）也可以通过多模态分布建模。

Dirichlet过程混合模型（DPMM）是评估比例数据分布的一种有效方法。它满足我们所需的设置：（i）估计（拟合）阶段，在此阶段，一组分布参数为评估，和（ii）推理阶段，为每个嵌入样本使用拟合模型。彻底的Blei和Jordan[4]给出了该模型的概述。

Dirichlet过程混合模型（DPMM）是评估比例数据分布的有效方法。它符合我们要求的设置：
（i）估计（拟合）阶段，在此期间评估一组分布参数，以及
（ii）推理阶段，使用拟合模型为每个嵌入样本提供分数。Blei和Jordan[4]对模型进行了全面概述。

DPMM是单峰Dirichlet分布的常见混合扩展，并使用Dirichllet过程，这DirichletDistribution的无限维扩展。该模型是多模态的，能够将每个模式捕获为混合成分。拟合模型具有多个模式，每个模式表示对应于一个正常行为的一组比例。在测试时，使用拟合模型通过其对数概率对每个样本进行评分。[4,8]中提供了关于DPMM使用的进一步解释和讨论。

3.5. Training

该模型的训练阶段包括两个阶段，一个是自动编码器的预训练阶段，其中网络的聚类分支保持不变，另一个是微调阶段，其中嵌入和聚类都得到优化。具体而言：

Pre-Training: 该模型通过最小化重建损失（表示为Lrec）来学习编码和重建序列，Lrec是原始瞬时位姿图和ST-GCAE重建的位姿图之间的L2损失

Fine-Tuning:
该模型优化了由重建损失和聚类损失组成的组合损失函数。
进行优化，使得聚类层优化为w.r.t.Lcluster，解码器优化为w.r.t.Lrec，编码器优化为w.r.t.两者。
集群层的初始化是通过Kmeans完成的。如[9]所示，当编码器针对这两种损失进行优化时，解码器保持不变，并充当正则化器，以保持编码器的嵌入质量。
本阶段的综合损失为：

结果

实现细节

def calc_reg_loss(model, reg_type='l2', avg=True):reg_loss = Noneparameters = list(param for name, param in model.named_parameters() if 'bias' not in name)num_params = len(parameters)if reg_type.lower() == 'l2':for param in parameters:if reg_loss is None:reg_loss = 0.5 * torch.sum(param ** 2)else:reg_loss = reg_loss + 0.5 * param.norm(2) ** 2if avg:reg_loss /= num_paramsreturn reg_losselse:return torch.tensor(0.0, device=model.device)

PatchModel((patch_fe): Identity()(gcae): GCAE((data_bn): BatchNorm1d(54, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)(st_gcn_enc): ModuleList((0): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(3, 8, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(3, 8, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(3, 8, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(3, 8, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(3, 8, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(3, 8, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(down): Sequential((0): Conv2d(3, 32, kernel_size=(1, 1), stride=(1, 1))(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True))(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(3, 288, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(96, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(32, 32, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0.3, inplace=True)))(1): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(32, 288, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(96, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(32, 32, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0.3, inplace=True)))(2): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(32, 288, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(96, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(32, 32, kernel_size=(9, 1), stride=(2, 1), padding=(4, 0))(3): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0.3, inplace=True))(residual): Sequential((0): Conv2d(32, 32, kernel_size=(1, 1), stride=(2, 1))(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)))(3): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(32, 12, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(32, 12, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(32, 12, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(32, 12, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(32, 12, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(32, 12, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(down): Sequential((0): Conv2d(32, 48, kernel_size=(1, 1), stride=(1, 1))(1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True))(bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(32, 432, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(144, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(48, 48, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0.3, inplace=True))(residual): Sequential((0): Conv2d(32, 48, kernel_size=(1, 1), stride=(1, 1))(1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)))(4): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(48, 432, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(144, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(48, 48, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0.3, inplace=True)))(5): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(48, 432, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(144, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(48, 48, kernel_size=(9, 1), stride=(3, 1), padding=(4, 0))(3): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0.3, inplace=True))(residual): Sequential((0): Conv2d(48, 48, kernel_size=(1, 1), stride=(3, 1))(1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)))(6): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(48, 16, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(48, 16, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(48, 16, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(48, 16, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(48, 16, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(48, 16, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(down): Sequential((0): Conv2d(48, 64, kernel_size=(1, 1), stride=(1, 1))(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True))(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(48, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(192, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(64, 64, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0.3, inplace=True))(residual): Sequential((0): Conv2d(48, 64, kernel_size=(1, 1), stride=(1, 1))(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)))(7): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(64, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(192, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(64, 64, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0.3, inplace=True)))(8): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(64, 8, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(64, 8, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(64, 8, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(64, 8, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(64, 8, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(64, 8, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(down): Sequential((0): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1))(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True))(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(64, 288, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(96, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(32, 32, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0.3, inplace=True))(residual): Sequential((0): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1))(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True))))(dec_final_gcn): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): ConvTemporalGraphical((conv): Conv2d(48, 9, kernel_size=(1, 1), stride=(1, 1))))(tcn): Sequential((0): BatchNorm2d(3, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(3, 3, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): Identity()(4): Dropout(p=0.3, inplace=True)))(st_gcn_dec): ModuleList((0): Upsample(scale_factor=(3.0, 1.0), mode=bilinear)(1): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(down): Sequential((0): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1))(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True))(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(32, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(192, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(64, 64, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0, inplace=True))(residual): Sequential((0): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1))(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)))(2): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(64, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(192, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(64, 64, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0, inplace=True)))(3): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(64, 12, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(64, 12, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(64, 12, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(64, 12, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(64, 12, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(64, 12, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(down): Sequential((0): Conv2d(64, 48, kernel_size=(1, 1), stride=(1, 1))(1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True))(bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(64, 432, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(144, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(48, 48, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0, inplace=True))(residual): Sequential((0): Conv2d(64, 48, kernel_size=(1, 1), stride=(1, 1))(1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)))(4): Upsample(scale_factor=(2.0, 1.0), mode=bilinear)(5): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(48, 432, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(144, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(48, 48, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0, inplace=True)))(6): ConvBlock((act): ReLU(inplace=True)(gcn): PyGeoConv((g_conv): SAGC((conv_a): ModuleList((0): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1)))(conv_b): ModuleList((0): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(1): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))(2): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1)))(gconv): ModuleList((0): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(1): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True))(2): GraphConvBR((bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(act): ReLU(inplace=True)))(bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(soft): Softmax(dim=-2)(relu): CELU(alpha=0.01)(expanding_conv): Conv2d(48, 432, kernel_size=(1, 1), stride=(1, 1), bias=False)(reduction_conv): Conv2d(144, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)))(tcn): Sequential((0): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(1): ReLU(inplace=True)(2): Conv2d(48, 48, kernel_size=(9, 1), stride=(1, 1), padding=(4, 0))(3): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)(4): Dropout(p=0, inplace=True)))))
)