Microsoft COCO: Common Objects in Context - 目标检测评估指标(Detection Evaluation)

参考博客
coco官网

coco目标检测评估指标

以下12个指标用于描述COCO上的对象检测器的性能:

Average Precision (AP):
AP
% AP at IoU=.50:.05:.95 (primary challenge metric)
APIoU=.50
% AP at IoU=.50 (PASCAL VOC metric)
APIoU=.75
% AP at IoU=.75 (strict metric)AP Across Scales:
APsmall
% AP for small objects: area < 32^2
APmedium
% AP for medium objects: 32^2 < area < 96^2
APlarge
% AP for large objects: area > 96^2Average Recall (AR):
ARmax=1
% AR given 1 detection per image
ARmax=10
% AR given 10 detections per image
ARmax=100
% AR given 100 detections per imageAR Across Scales:
ARsmall
% AR for small objects: area < 32^2
ARmedium
% AR for medium objects: 32^2 < area < 96^2
ARlarge
% AR for large objects: area > 96^2

除非另有说明，否则 AP 和 AR 在多个交并比 (IoU) 值上取平均值。具体来说，我们使用 10 个 IoU 阈值 0.50 : 0.05 : 0.95。这是对传统的一个突破，传统的 AP 是在一个单一的 0.50 的 IoU 上计算的 (这对应于我们的度量AP^IoU=.50)。超过均值的 IoUs 奖励更好定位的探测器 (Averaging over IoUs rewards detectors with better localization.)。
AP是所有类别的平均值。传统上，这被称为“平均准确度” (mAP，mean average precision)。我们没有区分AP 和 mAP (同样是 AR 和 mAR)，并假定从上下文中可以清楚地看出差异。
AP (所有 10 个 IoU 阈值和所有 80 个类别的平均值) 将决定赢家。在考虑 COCO 性能时，这应该被认为是最重要的一个指标。
在 COCO 中，与大物体相比有更多的小物体。具体地说，大约 41％的物体很小 (面积 < 32^2)，34％是中等 (32^2 < area < 96^2)，24％大 (area > 96^2)。测量的面积是分割掩码 (segmentation mask) 中的像素数量。
AR 是在每个图像中检测到固定数量的最大召回 (recall)，在类别和 IoU 上平均。AR 与提案评估 (proposal evaluation) 中使用的同名度量相关，但是按类别计算。
所有度量标准允许每个图像 (在所有类别中) 最多 100 个最高得分检测进行计算。
除了 IoU 计算 (分别在框 (box) 或掩码 (mask) 上执行) 之外，用边界框和分割掩码检测的评估度量在所有方面是相同的。

目标检测提交结果格式

Object Detection
对于带有包围框的检测，请使用以下格式:

[{"image_id": int, "category_id": int, "bbox": [x,y,width,height], "score": float,
}]

[{"image_id":42,"category_id":18,"bbox":[258.15,41.29,348.26,243.78],"score":0.236},{"image_id":73,"category_id":11,"bbox":[61,22.75,504,609.67],"score":0.318},{"image_id":73,"category_id":4,"bbox":[12.66,3.32,268.6,271.91],"score":0.726},{"image_id":74,"category_id":18,"bbox":[87.87,276.25,296.42,103.18],"score":0.546},{"image_id":74,"category_id":2,"bbox":[0,3.66,142.15,312.4],"score":0.3},{"image_id":74,"category_id":1,"bbox":[296.55,93.96,18.42,58.83],"score":0.407},{"image_id":74,"category_id":1,"bbox":[328.94,97.05,13.55,25.93],"score":0.611},{"image_id":74,"category_id":1,"bbox":[356.62,95.47,15.71,52.08],"score":0.335},{"image_id":74,"category_id":1,"bbox":[464.08,105.09,31.66,41.9],"score":0.805},{"image_id":74,"category_id":1,"bbox":[276.11,103.84,15.33,46.88],"score":0.953},{"image_id":74,"category_id":1,"bbox":[281.65,103.41,11.69,24.25],"score":0.087},{"image_id":133,"category_id":65,"bbox":[0,2.87,601,418.65],"score":0.699},{"image_id":133,"category_id":88,"bbox":[521.34,20.93,47.95,28.72],"score":0.423},{"image_id":136,"category_id":16,"bbox":[8.84,114.71,53.54,257.7],"score":0.473},{"image_id":136,"category_id":1,"bbox":[9,61.35,68.92,312.65],"score":0.796},{"image_id":136,"category_id":38,"bbox":[101,116.38,185.74,252.68],"score":0.127},{"image_id":136,"category_id":25,"bbox":[315.72,131.83,128.04,164.25],"score":0.326},{"image_id":139,"category_id":64,"bbox":[236.98,142.51,24.7,69.5],"score":0.221},{"image_id":139,"category_id":37,"bbox":[20.03,167.76,149.32,94.87],"score":0.849},{"image_id":139,"category_id":72,"bbox":[559.21,209.19,80.79,78.73],"score":0.518},{"image_id":139,"category_id":13,"bbox":[352.98,218.05,56,102.83],"score":0.477},{"image_id":139,"category_id":62,"bbox":[292.69,218,61.83,98.48],"score":0.38},{"image_id":139,"category_id":62,"bbox":[410.2,223.01,30.17,81.36],"score":0.864},{"image_id":139,"category_id":62,"bbox":[317.4,219.24,21.58,11.59],"score":0.401},{"image_id":139,"category_id":1,"bbox":[413.8,157.61,53.05,138.01],"score":0.157},{"image_id":139,"category_id":1,"bbox":[386.43,172.21,15.12,35.74],"score":0.71},{"image_id":139,"category_id":78,"bbox":[513.22,205.75,14.74,15.97],"score":0.075},{"image_id":139,"category_id":82,"bbox":[492.1,174.34,20.29,108.31],"score":0.26},{"image_id":139,"category_id":85,"bbox":[446.77,121.12,13.97,21.88],"score":0.164},{"image_id":139,"category_id":86,"bbox":[539.06,309.43,36.68,89.67],"score":0.806},{"image_id":139,"category_id":86,"bbox":[349.76,208.84,11.37,22.55],"score":0.474},{"image_id":139,"category_id":62,"bbox":[413.25,219.02,9.63,12.52],"score":0.375},{"image_id":139,"category_id":11,"bbox":[337.79,199.5,9.73,16.73],"score":0.532},{"image_id":139,"category_id":61,"bbox":[305.21,231.22,125.56,88.93],"score":0.106},{"image_id":143,"category_id":16,"bbox":[441.88,192.47,131.18,176.35],"score":0.829},{"image_id":143,"category_id":1,"bbox":[124.19,104.46,95.77,167.42],"score":0.523},{"image_id":143,"category_id":16,"bbox":[275.59,126.88,100,162.77],"score":0.755},{"image_id":143,"category_id":35,"bbox":[129.08,319.22,92.28,161.25],"score":0.42},{"image_id":143,"category_id":16,"bbox":[45.96,20.22,95.5,133.71],"score":0.144},{"image_id":143,"category_id":16,"bbox":[123.18,237.43,114.24,134.32],"score":0.878},{"image_id":143,"category_id":16,"bbox":[468.71,320.79,94.24,154.59],"score":0.756},{"image_id":164,"category_id":44,"bbox":[388.84,183.92,8.02,26.87],"score":0.923},{"image_id":164,"category_id":44,"bbox":[374.4,189.64,6.87,20.93],"score":0.46},{"image_id":164,"category_id":44,"bbox":[366.79,184.72,7.05,27.54],"score":0.168},{"image_id":164,"category_id":44,"bbox":[383.16,187.56,7.21,24.24],"score":0.084},{"image_id":164,"category_id":44,"bbox":[428.79,280.36,18.32,31.37],"score":0.145},{"image_id":164,"category_id":82,"bbox":[439.7,171.07,96.72,138.04],"score":0.178},{"image_id":164,"category_id":44,"bbox":[463.1,282.68,9.92,36.55],"score":0.263},{"image_id":164,"category_id":44,"bbox":[401.87,161.68,9.4,17.1],"score":0.917},{"image_id":164,"category_id":44,"bbox":[374.61,135.8,10.49,15.16],"score":0.805},{"image_id":164,"category_id":57,"bbox":[404.56,132.62,9.78,15.75],"score":0.155},{"image_id":164,"category_id":44,"bbox":[413.11,131.22,10.65,16.86],"score":0.863},{"image_id":164,"category_id":62,"bbox":[173.04,409.16,151.36,70.84],"score":0.3},{"image_id":164,"category_id":46,"bbox":[154.96,189.95,12.83,28.26],"score":0.125},{"image_id":164,"category_id":28,"bbox":[140.01,190.76,16.1,36.54],"score":0.134},{"image_id":164,"category_id":46,"bbox":[108.42,196.32,14.31,33.4],"score":0.631},{"image_id":164,"category_id":46,"bbox":[57.59,196.12,19.66,37.63],"score":0.04},{"image_id":164,"category_id":46,"bbox":[81.76,195.46,15.44,39.6],"score":0.048},{"image_id":164,"category_id":46,"bbox":[94.89,196.13,11.83,36.93],"score":0.679},{"image_id":164,"category_id":47,"bbox":[148.61,233.85,26.78,20.8],"score":0.269},{"image_id":164,"category_id":58,"bbox":[113.56,236.81,23.23,19.52],"score":0.062},{"image_id":164,"category_id":47,"bbox":[176.77,232.66,20.14,16.13],"score":0.616},{"image_id":164,"category_id":47,"bbox":[195.65,228.88,19.99,15.51],"score":0.845},{"image_id":164,"category_id":51,"bbox":[513.25,221.5,56.15,10.53],"score":0.347},{"image_id":164,"category_id":78,"bbox":[380.72,227.45,53.28,30.36],"score":0.856},{"image_id":164,"category_id":79,"bbox":[273.61,263.38,46.82,34.22],"score":0.5},{"image_id":164,"category_id":46,"bbox":[77.17,194.99,10.84,39.68],"score":0.883},{"image_id":164,"category_id":46,"bbox":[134.3,192.93,10.75,35.37],"score":0.154},{"image_id":164,"category_id":62,"bbox":[158.47,231.64,21.72,21.24],"score":0.438},{"image_id":164,"category_id":47,"bbox":[512.83,134.18,15.4,21.38],"score":0.231},{"image_id":164,"category_id":47,"bbox":[529.63,139.85,15.19,13.71],"score":0.71},{"image_id":164,"category_id":47,"bbox":[505.11,146.68,6.26,8.2],"score":0.301},{"image_id":164,"category_id":47,"bbox":[172.77,203.22,17.75,18.85],"score":0.302},{"image_id":164,"category_id":47,"bbox":[135.77,233.33,15.32,20.21],"score":0.96},{"image_id":164,"category_id":51,"bbox":[503.95,211.57,29.18,4.68],"score":0.364},{"image_id":164,"category_id":51,"bbox":[505.14,219.24,56.89,4.21],"score":0.089},{"image_id":164,"category_id":22,"bbox":[514.26,215.43,52.03,5.24],"score":0.853},{"image_id":164,"category_id":51,"bbox":[511.83,212.69,55.62,6.07],"score":0.084},{"image_id":164,"category_id":74,"bbox":[362.88,171.41,13.36,9.05],"score":0.704},{"image_id":164,"category_id":67,"bbox":[364.78,459.12,137.14,14.96],"score":0.236},{"image_id":192,"category_id":1,"bbox":[341.48,253.48,131.6,220.05],"score":0.123},{"image_id":192,"category_id":1,"bbox":[448.21,218.33,78.41,248.03],"score":0.864},{"image_id":192,"category_id":1,"bbox":[0.93,274.23,36.53,205.51],"score":0.919},{"image_id":192,"category_id":1,"bbox":[255.11,179.46,111.35,235.68],"score":0.222},{"image_id":192,"category_id":39,"bbox":[9.2,381.39,36,92.98],"score":0.054},{"image_id":196,"category_id":50,"bbox":[285.73,200.54,210.81,48.65],"score":0.574},{"image_id":196,"category_id":57,"bbox":[443.17,363.72,22.98,42.51],"score":0.405},{"image_id":196,"category_id":57,"bbox":[392.24,374.67,24.62,53.82],"score":0.948},{"image_id":196,"category_id":57,"bbox":[411.55,404.5,55.98,30.74],"score":0.876},{"image_id":196,"category_id":50,"bbox":[548.65,250.15,73.72,122.35],"score":0.851},{"image_id":196,"category_id":46,"bbox":[232.52,42.78,71.44,43.44],"score":0.536},{"image_id":196,"category_id":50,"bbox":[470.49,390.81,121.54,88.15],"score":0.138},{"image_id":196,"category_id":51,"bbox":[277.91,58.21,128.58,62.76],"score":0.213},{"image_id":196,"category_id":10,"bbox":[421.99,236.14,144.88,113.72],"score":0.888},{"image_id":196,"category_id":51,"bbox":[219.69,197.88,169.4,86.23],"score":0.227},{"image_id":196,"category_id":51,"bbox":[376.02,337.26,161.24,139.17],"score":0.111},{"image_id":196,"category_id":51,"bbox":[321.22,157.3,132.68,69.04],"score":0.122},{"image_id":196,"category_id":51,"bbox":[349.69,115.74,151.25,64.57],"score":0.47},{"image_id":196,"category_id":51,"bbox":[183.83,72.85,88.39,33.07],"score":0.621},{"image_id":196,"category_id":51,"bbox":[358.33,47.29,92.99,62.87],"score":0.43},{"image_id":196,"category_id":56,"bbox":[200.46,254.35,18.25,31.59],"score":0.958},{"image_id":196,"category_id":56,"bbox":[281.51,393.55,37.68,28.9],"score":0.892},{"image_id":196,"category_id":56,"bbox":[172.36,269.84,21.54,29.09],"score":0.108},{"image_id":196,"category_id":57,"bbox":[459.6,401.48,26.37,31.69],"score":0.361},{"image_id":196,"category_id":57,"bbox":[412.04,417.75,13.64,31.75],"score":0.963},{"image_id":196,"category_id":70,"bbox":[0,41.94,555,438.06],"score":0.539},{"image_id":196,"category_id":6,"bbox":[214.7,407.69,84.78,66.08],"score":0.03},{"image_id":196,"category_id":51,"bbox":[105.26,88.64,96.13,39.34],"score":0.62},{"image_id":196,"category_id":16,"bbox":[25.84,109.71,117.18,60.2],"score":0.896},{"image_id":196,"category_id":51,"bbox":[0,145.79,71.35,73.52],"score":0.587},{"image_id":196,"category_id":56,"bbox":[119.3,340.83,40,25.45],"score":0.593},{"image_id":196,"category_id":57,"bbox":[577.34,377.01,12.18,15.03],"score":0.972},{"image_id":196,"category_id":57,"bbox":[368.9,388.96,31.37,27.25],"score":0.183},{"image_id":196,"category_id":57,"bbox":[165.75,359.55,41.76,41.77],"score":0.036},{"image_id":196,"category_id":57,"bbox":[435.1,446.58,20.64,10.6],"score":0.033},{"image_id":196,"category_id":50,"bbox":[524.71,129.89,19.64,9.29],"score":0.081},{"image_id":196,"category_id":56,"bbox":[250.3,325.7,24.42,20.53],"score":0.479},{"image_id":196,"category_id":56,"bbox":[288.99,290.28,17.52,33.3],"score":0.767},{"image_id":196,"category_id":56,"bbox":[150.98,262.69,16.43,13.92],"score":0.277},{"image_id":196,"category_id":43,"bbox":[6,188.89,172.64,92.01],"score":0.624},{"image_id":208,"category_id":81,"bbox":[52.08,108.94,587.92,366.75],"score":0.476},{"image_id":208,"category_id":49,"bbox":[0,150.36,299.55,109.76],"score":0.287},{"image_id":208,"category_id":90,"bbox":[133.81,170.21,191.76,88.89],"score":0.495},{"image_id":208,"category_id":90,"bbox":[42.35,122.18,158.34,82.29],"score":0.126},{"image_id":241,"category_id":63,"bbox":[333.46,250.92,146.54,296.02],"score":0.674},{"image_id":241,"category_id":63,"bbox":[0,278.19,142.45,260.43],"score":0.434},{"image_id":241,"category_id":1,"bbox":[147.7,6.57,200.43,626.17],"score":0.857},{"image_id":241,"category_id":1,"bbox":[4.44,263.55,205.66,348.05],"score":0.25},{"image_id":241,"category_id":1,"bbox":[307.71,263.06,107.15,257.15],"score":0.622},{"image_id":241,"category_id":47,"bbox":[106.72,595.29,50.72,44.71],"score":0.986},{"image_id":241,"category_id":47,"bbox":[0,596.49,39.42,43.51],"score":0.097},{"image_id":241,"category_id":75,"bbox":[116.1,321.7,44.12,22.16],"score":0.703},{"image_id":241,"category_id":1,"bbox":[407.51,337.98,72.49,230.11],"score":0.319},{"image_id":241,"category_id":64,"bbox":[88.85,71.11,109.84,201.2],"score":0.748},{"image_id":241,"category_id":64,"bbox":[135.45,350.26,49.78,72.27],"score":0.521},{"image_id":257,"category_id":10,"bbox":[50.27,199.13,25.09,23.76],"score":0.923},{"image_id":257,"category_id":27,"bbox":[177.16,362.21,50.4,55.49],"score":0.683},{"image_id":257,"category_id":8,"bbox":[286.17,277.96,204.94,122.97],"score":0.912},{"image_id":257,"category_id":1,"bbox":[144.57,345.58,71.52,134.42],"score":0.7},{"image_id":257,"category_id":65,"bbox":[231.74,326.46,28.38,79.72],"score":0.412},{"image_id":257,"category_id":1,"bbox":[213.62,332.49,22.28,60.49],"score":0.646},{"image_id":257,"category_id":1,"bbox":[265.09,330.65,16.37,72.88],"score":0.23},{"image_id":257,"category_id":14,"bbox":[291.83,337.14,33.15,77],"score":0.427},{"image_id":257,"category_id":1,"bbox":[568.98,327.79,14.75,42.24],"score":0.044},{"image_id":257,"category_id":1,"bbox":[124.19,325.45,10.13,37.03],"score":0.765},{"image_id":257,"category_id":1,"bbox":[314.72,334.13,20.17,74.8],"score":0.055},{"image_id":257,"category_id":1,"bbox":[78.65,326.41,20.3,48.17],"score":0.278},{"image_id":257,"category_id":1,"bbox":[138.11,325.59,15.55,38.45],"score":0.868},{"image_id":257,"category_id":1,"bbox":[226.17,327.33,15.21,69.54],"score":0.648},{"image_id":257,"category_id":10,"bbox":[227.37,296.5,11.3,11.39],"score":0.526},{"image_id":257,"category_id":10,"bbox":[26.58,303.1,5.69,7.11],"score":0.789},{"image_id":257,"category_id":10,"bbox":[27.93,276.6,5.82,11.76],"score":0.96},{"image_id":257,"category_id":27,"bbox":[208.47,340.87,11.3,19.58],"score":0.116},{"image_id":257,"category_id":1,"bbox":[190.83,330.6,20.63,39.77],"score":0.626},{"image_id":257,"category_id":8,"bbox":[513.96,323.81,120.04,156.19],"score":0.881},{"image_id":257,"category_id":8,"bbox":[25.24,301.11,141.23,59.84],"score":0.563},{"image_id":257,"category_id":10,"bbox":[89.93,177.97,15.03,14.65],"score":0.956},{"image_id":257,"category_id":31,"bbox":[94.13,344.24,5.88,12.48],"score":0.514},{"image_id":257,"category_id":10,"bbox":[301.98,260.82,7.09,14.18],"score":0.489},{"image_id":257,"category_id":10,"bbox":[295.6,261.4,6.39,12.2],"score":0.511},{"image_id":257,"category_id":10,"bbox":[55.4,299.17,4.88,4.3],"score":0.605},{"image_id":257,"category_id":31,"bbox":[209.91,341.48,11.21,29.05],"score":0.455},{"image_id":257,"category_id":31,"bbox":[472.6,340.88,6.14,4.6],"score":0.292},{"image_id":283,"category_id":44,"bbox":[49.18,42.58,82.4,311.75],"score":0.676},{"image_id":283,"category_id":62,"bbox":[10.23,142.61,59.56,86.35],"score":0.691},{"image_id":283,"category_id":62,"bbox":[149.65,67.02,278.35,289.03],"score":0.914},{"image_id":283,"category_id":70,"bbox":[144.76,156.24,88.26,214.22],"score":0.418},{"image_id":285,"category_id":23,"bbox":[0,68.81,559,563.94],"score":0.487},{"image_id":294,"category_id":50,"bbox":[414.57,349.85,15.31,23.55],"score":0.071},{"image_id":294,"category_id":78,"bbox":[508.6,312.81,131.4,109.39],"score":0.895},{"image_id":294,"category_id":48,"bbox":[607.51,251.83,7.67,40.75],"score":0.798},{"image_id":294,"category_id":49,"bbox":[447.73,329.05,17.33,34.67],"score":0.346},{"image_id":294,"category_id":49,"bbox":[452.37,320.69,15.89,45.17],"score":0.014},{"image_id":294,"category_id":50,"bbox":[398.49,323.29,23.1,25.32],"score":0.192},{"image_id":294,"category_id":50,"bbox":[387.89,341.86,35.58,40.04],"score":0.055},{"image_id":294,"category_id":50,"bbox":[421.95,114.4,13.43,103.27],"score":0.643},{"image_id":294,"category_id":49,"bbox":[460.6,335.52,16.28,33.28],"score":0.378},{"image_id":294,"category_id":49,"bbox":[465.52,325.24,13.79,32.5],"score":0.153},{"image_id":294,"category_id":49,"bbox":[466.28,335.38,15.32,33.19],"score":0.438},{"image_id":294,"category_id":85,"bbox":[0,0,47.87,53.96],"score":0.746},{"image_id":294,"category_id":50,"bbox":[404.99,367.62,32.19,21.11],"score":0.256},{"image_id":294,"category_id":50,"bbox":[394.56,299.36,16.54,32.35],"score":0.201},{"image_id":294,"category_id":50,"bbox":[400.36,355.24,21.6,25.14],"score":0.937},{"image_id":294,"category_id":50,"bbox":[623.23,50.49,4.79,22.57],"score":0.234},{"image_id":328,"category_id":77,"bbox":[438.75,283.15,35.01,18.64],"score":0.663},{"image_id":328,"category_id":1,"bbox":[240.91,90.52,205.13,389.34],"score":0.19},{"image_id":328,"category_id":1,"bbox":[358.87,58.48,249.91,426.04],"score":0.191},{"image_id":328,"category_id":23,"bbox":[49.6,104.76,587.67,374.74],"score":0.08},{"image_id":328,"category_id":84,"bbox":[232.65,220.95,86.44,37.42],"score":0.749},{"image_id":328,"category_id":3,"bbox":[349.46,77.31,101.02,65.61],"score":0.078},{"image_id":328,"category_id":32,"bbox":[461.06,177.54,18.31,43.81],"score":0.406},{"image_id":328,"category_id":32,"bbox":[317.37,171.71,22.5,57.98],"score":0.448},{"image_id":328,"category_id":28,"bbox":[24.03,171.36,119.45,79.63],"score":0.057},{"image_id":328,"category_id":27,"bbox":[46.45,160.85,85.28,47.84],"score":0.712},{"image_id":338,"category_id":82,"bbox":[216.59,149.74,95.78,177.26],"score":0.919},{"image_id":338,"category_id":85,"bbox":[461.17,88.69,23.24,24.61],"score":0.333},{"image_id":338,"category_id":28,"bbox":[325,142.07,90.46,183.7],"score":0.397},{"image_id":338,"category_id":1,"bbox":[436.35,173.72,56.91,153.28],"score":0.154},{"image_id":338,"category_id":79,"bbox":[28.25,243.96,138.88,78.63],"score":0.502},{"image_id":338,"category_id":31,"bbox":[463.15,231.65,25.66,39.34],"score":0.787},{"image_id":357,"category_id":39,"bbox":[540.09,121.4,4,27.24],"score":0.407},{"image_id":357,"category_id":40,"bbox":[567.96,186.44,8.91,6.69],"score":0.621},{"image_id":357,"category_id":40,"bbox":[77.74,97.16,6.97,9.21],"score":0.52},{"image_id":357,"category_id":40,"bbox":[279.48,25.03,5.46,4.72],"score":0.069},{"image_id":357,"category_id":1,"bbox":[370.19,25.03,17.25,27.63],"score":0.34},{"image_id":357,"category_id":1,"bbox":[275.1,18.4,27.02,37.16],"score":0.437},{"image_id":357,"category_id":1,"bbox":[208.34,22.61,22.2,31.01],"score":0.752},{"image_id":357,"category_id":1,"bbox":[72.62,79.12,18.53,48.08],"score":0.997},{"image_id":357,"category_id":1,"bbox":[527.83,136.21,29.04,54.59],"score":0.888},{"image_id":357,"category_id":1,"bbox":[570.61,160.02,26.76,38.03],"score":0.294},{"image_id":357,"category_id":1,"bbox":[592.37,154.71,27.18,46.6],"score":0.344},{"image_id":357,"category_id":40,"bbox":[557.03,23.96,13.89,9.67],"score":0.406},{"image_id":357,"category_id":1,"bbox":[266.33,6.43,17.69,27.34],"score":0.59},{"image_id":357,"category_id":1,"bbox":[567.15,29.27,11.87,23.74],"score":0.057},{"image_id":357,"category_id":1,"bbox":[580.99,32.19,6.72,23.92],"score":0.804},{"image_id":359,"category_id":36,"bbox":[155.87,100.44,65.37,34.25],"score":0.822},{"image_id":359,"category_id":10,"bbox":[245.76,99.94,51,22.4],"score":0.773},{"image_id":359,"category_id":10,"bbox":[278.48,293.48,4.46,4.11],"score":0.324},{"image_id":359,"category_id":46,"bbox":[2,290.11,123.72,37.58],"score":0.914},{"image_id":360,"category_id":18,"bbox":[258.02,138.76,52.1,36.95],"score":0.054},{"image_id":360,"category_id":36,"bbox":[248.45,155.01,52.19,41.49],"score":0.428},{"image_id":387,"category_id":73,"bbox":[176.38,50.29,463.62,312.38],"score":0.328},{"image_id":387,"category_id":77,"bbox":[355.5,111.88,210,93.75],"score":0.064},{"image_id":395,"category_id":77,"bbox":[293.15,289.29,19.84,22.97],"score":0.037},{"image_id":395,"category_id":1,"bbox":[177.61,138.16,432.72,435.32],"score":0.607},{"image_id":395,"category_id":1,"bbox":[0,207.32,72.81,278.92],"score":0.043},{"image_id":395,"category_id":1,"bbox":[81.02,270.92,82.55,204.66],"score":0.465},{"image_id":395,"category_id":77,"bbox":[618.05,305.74,15.83,11.59],"score":0.297},{"image_id":395,"category_id":1,"bbox":[111.58,232.54,38.86,46.9],"score":0.252},{"image_id":395,"category_id":1,"bbox":[132.76,239.82,82.12,109.48],"score":0.961},{"image_id":395,"category_id":1,"bbox":[413.44,194.77,32.94,105.97],"score":0.58},{"image_id":395,"category_id":1,"bbox":[420.93,206.74,43.9,113.15],"score":0.632},{"image_id":395,"category_id":1,"bbox":[153.8,332.19,60.39,66.97],"score":0.76},{"image_id":395,"category_id":1,"bbox":[58.7,214.92,48.39,126.94],"score":0.803},{"image_id":395,"category_id":1,"bbox":[451.16,253.08,70.79,89.66],"score":0.239},{"image_id":397,"category_id":62,"bbox":[469.94,0,143.48,98.06],"score":0.935},{"image_id":397,"category_id":47,"bbox":[7,294.57,36.75,71.88],"score":0.157},{"image_id":397,"category_id":48,"bbox":[596.24,404.53,43.76,52.1],"score":0.63},{"image_id":397,"category_id":54,"bbox":[85.25,349.09,503.69,130.91],"score":0.657},{"image_id":397,"category_id":1,"bbox":[0,0,205.4,205.33],"score":0.089},{"image_id":400,"category_id":18,"bbox":[430.5,148.97,97.62,78.77],"score":0.534},{"image_id":400,"category_id":9,"bbox":[0,64.72,616,477.48],"score":0.136},{"image_id":415,"category_id":10,"bbox":[62.13,399.09,41.97,44.41],"score":0.045},{"image_id":415,"category_id":1,"bbox":[1,257.44,122.96,301.62],"score":0.373},{"image_id":428,"category_id":32,"bbox":[367.56,200.49,94.65,154.52],"score":0.097},{"image_id":428,"category_id":1,"bbox":[218.94,32.65,355.92,323.27],"score":0.912},{"image_id":428,"category_id":61,"bbox":[399.41,234.73,175.13,121.62],"score":0.374},{"image_id":459,"category_id":32,"bbox":[170.57,278.99,61.63,239.81],"score":0.97},{"image_id":459,"category_id":1,"bbox":[0,84.85,457.88,547.96],"score":0.16},{"image_id":472,"category_id":5,"bbox":[390.86,56.68,87.21,31.46],"score":0.656},{"image_id":474,"category_id":1,"bbox":[23.37,42.7,309.63,429.21],"score":0.163},{"image_id":474,"category_id":40,"bbox":[22.86,234.5,88.41,71.31],"score":0.962},{"image_id":486,"category_id":44,"bbox":[621.9,233.27,16.02,38.17],"score":0.286},{"image_id":486,"category_id":50,"bbox":[223.65,192.28,6.92,51.96],"score":0.24},{"image_id":486,"category_id":44,"bbox":[496.82,278.46,23.05,25.31],"score":0.004},{"image_id":486,"category_id":44,"bbox":[82.06,219.17,5.12,29.08],"score":0.14},{"image_id":486,"category_id":49,"bbox":[234.42,200.87,3.9,16.53],"score":0.182},{"image_id":486,"category_id":51,"bbox":[265.43,299.84,131.39,88.22],"score":0.915},{"image_id":488,"category_id":1,"bbox":[183.76,210.3,104.63,131.03],"score":0.596},{"image_id":488,"category_id":39,"bbox":[211.91,225.8,63.45,45.93],"score":0.379},{"image_id":488,"category_id":40,"bbox":[562.51,253.32,10.5,7.99],"score":0.984},{"image_id":488,"category_id":1,"bbox":[553.32,216.45,44.13,49.26],"score":0.661},{"image_id":488,"category_id":1,"bbox":[37.67,208.02,91.24,152.36],"score":0.566},{"image_id":502,"category_id":23,"bbox":[190.96,88,273.13,257.74],"score":0.205},{"image_id":520,"category_id":16,"bbox":[297.04,140.73,32.01,20.26],"score":0.519},{"image_id":520,"category_id":16,"bbox":[230.03,131.19,34.17,13.96],"score":0.373},{"image_id":520,"category_id":1,"bbox":[244.02,307.84,15.21,34.27],"score":0.371},{"image_id":520,"category_id":48,"bbox":[451.26,217.6,18.56,17.79],"score":0.874},{"image_id":520,"category_id":16,"bbox":[0,187.56,22.33,15.48],"score":0.564},{"image_id":520,"category_id":16,"bbox":[444.38,174.06,21.97,16.09],"score":0.608},{"image_id":520,"category_id":16,"bbox":[92.33,194.51,35.82,22.84],"score":0.937},{"image_id":520,"category_id":44,"bbox":[156.5,184.44,25.5,11.87],"score":0.507},{"image_id":520,"category_id":1,"bbox":[508.05,323.61,8.1,13.7],"score":0.446},{"image_id":536,"category_id":62,"bbox":[285.41,138.98,156.33,195.53],"score":0.82},{"image_id":536,"category_id":1,"bbox":[286.86,133.95,155.14,198.27],"score":0.838},{"image_id":536,"category_id":1,"bbox":[185.41,79.52,108.42,255.62],"score":0.584},{"image_id":536,"category_id":1,"bbox":[89.34,68.58,98.16,263.52],"score":0.769},{"image_id":536,"category_id":31,"bbox":[23.1,258.04,44.05,77.96],"score":0.176},{"image_id":536,"category_id":31,"bbox":[193.07,185.98,69.48,63.22],"score":0.071},{"image_id":536,"category_id":31,"bbox":[350.78,235.32,45.57,12.3],"score":0.619},{"image_id":536,"category_id":77,"bbox":[203.43,106.55,13.68,21.14],"score":0.324},{"image_id":536,"category_id":77,"bbox":[101.62,87.68,28.93,39.35],"score":0.44},{"image_id":536,"category_id":62,"bbox":[3.68,141.91,82.48,194.09],"score":0.481},{"image_id":536,"category_id":57,"bbox":[361.35,224.65,8.04,12.47],"score":0.124},{"image_id":544,"category_id":37,"bbox":[596.57,293.78,13.81,7.44],"score":0.954},{"image_id":544,"category_id":1,"bbox":[273.32,243.73,113.23,161.2],"score":0.921},{"image_id":544,"category_id":1,"bbox":[114.47,301.3,129.54,111.31],"score":0.381},{"image_id":544,"category_id":1,"bbox":[36.53,269.29,101.67,142.79],"score":0.186},{"image_id":544,"category_id":1,"bbox":[75.23,191.66,28.84,44.78],"score":0.785},{"image_id":544,"category_id":1,"bbox":[248.29,185.44,29.17,50.01],"score":0.739},{"image_id":544,"category_id":1,"bbox":[191.14,209.98,30.04,27.53],"score":0.17},{"image_id":544,"category_id":1,"bbox":[144.91,193.7,31.15,41.98],"score":0.76},{"image_id":544,"category_id":1,"bbox":[3.07,221.17,33.48,77.9],"score":0.274},{"image_id":544,"category_id":1,"bbox":[546.81,211.75,21.74,21],"score":0.23},{"image_id":544,"category_id":38,"bbox":[224.14,300.79,52.74,7.6],"score":0.941},{"image_id":544,"category_id":40,"bbox":[217.39,324.92,20.27,30.58],"score":0.466},{"image_id":544,"category_id":1,"bbox":[19.46,237.09,42.72,59.44],"score":0.439},{"image_id":544,"category_id":1,"bbox":[17.77,194.87,25.64,45.63],"score":0.913},{"image_id":544,"category_id":54,"bbox":[514.31,203.13,16.39,29.72],"score":0.066},{"image_id":564,"category_id":1,"bbox":[225.51,335.3,147.48,279.2],"score":0.292},{"image_id":564,"category_id":1,"bbox":[433.78,203.68,86.22,93.9],"score":0.17},{"image_id":564,"category_id":43,"bbox":[282.89,432.81,105.48,40.82],"score":0.658},{"image_id":564,"category_id":62,"bbox":[58.9,234.39,43.46,22.27],"score":0.842},{"image_id":564,"category_id":62,"bbox":[103.71,232,45.58,25.44],"score":0.721},{"image_id":564,"category_id":62,"bbox":[197.82,230.33,47.5,26.02],"score":0.807},{"image_id":564,"category_id":62,"bbox":[254.16,307.81,56.56,28.28],"score":0.666},{"image_id":564,"category_id":62,"bbox":[140.73,258.53,50.13,30.25],"score":0.051},{"image_id":564,"category_id":62,"bbox":[84.79,256.37,50.46,25.81],"score":0.785},{"image_id":564,"category_id":62,"bbox":[47.44,260.53,46.92,23.68],"score":0.136},{"image_id":564,"category_id":62,"bbox":[18.52,235.62,47.74,22.44],"score":0.776},{"image_id":564,"category_id":62,"bbox":[397.26,304.21,61.51,46.54],"score":0.75},{"image_id":564,"category_id":62,"bbox":[331.82,279.27,51.53,21.61],"score":0.44},{"image_id":564,"category_id":62,"bbox":[307.39,307.56,51.38,27.27],"score":0.684},{"image_id":564,"category_id":62,"bbox":[229.06,228.6,66.46,77.41],"score":0.565},{"image_id":564,"category_id":62,"bbox":[149.8,232.51,48.71,41.53],"score":0.493},{"image_id":569,"category_id":1,"bbox":[100.1,269.14,80.18,145.38],"score":0.253},{"image_id":569,"category_id":38,"bbox":[168.12,216.94,16.66,32.58],"score":0.839},{"image_id":569,"category_id":38,"bbox":[142.5,229.28,22.99,23.7],"score":0.761},{"image_id":569,"category_id":38,"bbox":[134.91,239.77,25.66,23.29],"score":0.369},{"image_id":589,"category_id":34,"bbox":[465.86,229.41,18.75,6.3],"score":0.729},{"image_id":589,"category_id":1,"bbox":[359.91,184.57,145.43,184.29],"score":0.882},{"image_id":590,"category_id":81,"bbox":[86.87,295.01,85.2,11.57],"score":0.151},{"image_id":590,"category_id":51,"bbox":[80,292.99,92.87,27.6],"score":0.79},{"image_id":599,"category_id":64,"bbox":[545.37,5.48,94.63,237.26],"score":0.751},{"image_id":599,"category_id":17,"bbox":[18.92,37.13,412.5,365.75],"score":0.952},{"image_id":599,"category_id":63,"bbox":[56,1.83,584,400.9],"score":0.314},{"image_id":599,"category_id":75,"bbox":[481.01,198.19,158.99,87.8],"score":0.438},{"image_id":599,"category_id":75,"bbox":[382.58,167.29,198,60.5],"score":0.492},{"image_id":623,"category_id":15,"bbox":[283.29,72.62,91.71,238.38],"score":0.823},{"image_id":623,"category_id":88,"bbox":[29.12,31.46,345.88,462.92],"score":0.872},{"image_id":623,"category_id":62,"bbox":[277.62,62.92,97.38,239.84],"score":0.872},{"image_id":623,"category_id":1,"bbox":[144.34,1.3,211.69,490.91],"score":0.735},{"image_id":626,"category_id":85,"bbox":[314.52,124.36,41.32,42.47],"score":0.378},{"image_id":626,"category_id":73,"bbox":[398.15,129.71,20.88,46.91],"score":0.701},{"image_id":632,"category_id":65,"bbox":[0,266.85,376.5,208.25],"score":0.144},{"image_id":632,"category_id":64,"bbox":[187.36,136.56,60.78,92.39],"score":0.344},{"image_id":632,"category_id":84,"bbox":[455.98,192.5,8.57,35.59],"score":0.088},{"image_id":632,"category_id":84,"bbox":[453.31,252.97,8.02,33.93],"score":0.589},{"image_id":632,"category_id":84,"bbox":[444.76,297.63,5.32,39.68],"score":0.902},{"image_id":632,"category_id":84,"bbox":[504.95,191.02,12.15,36.78],"score":0.405},{"image_id":632,"category_id":84,"bbox":[487.51,199.33,7.48,28.05],"score":0.032},{"image_id":632,"category_id":62,"bbox":[234.82,230.45,104.72,87.69],"score":0.051},{"image_id":632,"category_id":84,"bbox":[527.02,248.57,24.4,40.43],"score":0.026},{"image_id":632,"category_id":84,"bbox":[524.32,97.38,2.8,37.77],"score":0.084},{"image_id":632,"category_id":84,"bbox":[454.66,245.27,48.55,12.14],"score":0.704},{"image_id":632,"category_id":84,"bbox":[461.77,253.68,8.24,33.31],"score":0.451},{"image_id":636,"category_id":70,"bbox":[122.72,89.37,265.23,536.22],"score":0.283},{"image_id":641,"category_id":2,"bbox":[563.25,265.02,17.49,21.82],"score":0.031},{"image_id":641,"category_id":6,"bbox":[100.35,100.03,417.42,228.9],"score":0.329},{"image_id":641,"category_id":1,"bbox":[580.29,233.52,11.19,13.08],"score":0.993},{"image_id":641,"category_id":1,"bbox":[569.24,240.92,17.46,34.38],"score":0.18},{"image_id":641,"category_id":1,"bbox":[595.87,232.88,9.74,13.22],"score":0.969},{"image_id":641,"category_id":1,"bbox":[556.69,231.63,7.03,22.85],"score":0.25},{"image_id":641,"category_id":1,"bbox":[463.68,225.62,5.04,26.32],"score":0.583},{"image_id":641,"category_id":1,"bbox":[606.61,231.89,7.13,14.45],"score":0.591},{"image_id":641,"category_id":1,"bbox":[25.91,207.55,29.39,23],"score":0.355},{"image_id":641,"category_id":1,"bbox":[3.82,209.9,20.4,22.96],"score":0.751},{"image_id":641,"category_id":1,"bbox":[0,220.97,6.32,18.94],"score":0.862},{"image_id":661,"category_id":77,"bbox":[90.11,4.42,111.24,44.67],"score":0.843},{"image_id":661,"category_id":5,"bbox":[17.15,45.37,337.07,289.04],"score":0.734},{"image_id":675,"category_id":17,"bbox":[325.71,81.3,314.29,391.6],"score":0.248},{"image_id":675,"category_id":77,"bbox":[99.84,232.35,34.79,25.58],"score":0.102},{"image_id":675,"category_id":47,"bbox":[243.34,209.84,69.65,98.5],"score":0.616},{"image_id":675,"category_id":47,"bbox":[138.74,231.28,56.88,73.37],"score":0.789},{"image_id":675,"category_id":60,"bbox":[169.37,179.04,46.55,90.39],"score":0.817},{"image_id":692,"category_id":90,"bbox":[315.39,263.78,37.77,33.79],"score":0.255},{"image_id":692,"category_id":1,"bbox":[288.42,30.26,351.58,428.34],"score":0.608},{"image_id":692,"category_id":1,"bbox":[188.87,129.27,307.8,343.32],"score":0.073},{"image_id":692,"category_id":90,"bbox":[383,338.04,22.31,29.38],"score":0.757},{"image_id":693,"category_id":41,"bbox":[429.64,193.67,125.51,198.6],"score":0.884},{"image_id":693,"category_id":41,"bbox":[0,190.22,212.1,137.01],"score":0.577},{"image_id":693,"category_id":41,"bbox":[14.96,220.25,236.6,190.44],"score":0.815},{"image_id":693,"category_id":41,"bbox":[163.19,19.83,125.4,153.27],"score":0.668},{"image_id":693,"category_id":41,"bbox":[211.24,48.93,91.56,89.91],"score":0.278},{"image_id":693,"category_id":41,"bbox":[236.35,0,88.94,77.36],"score":0.93},{"image_id":693,"category_id":41,"bbox":[71.7,92.27,192.32,124.51],"score":0.532},{"image_id":693,"category_id":80,"bbox":[0,122.61,190.86,142.72],"score":0.404},{"image_id":693,"category_id":41,"bbox":[223.46,36.1,71.24,78.42],"score":0.152},{"image_id":693,"category_id":42,"bbox":[213.87,0,82.98,64.55],"score":0.435},{"image_id":699,"category_id":10,"bbox":[31.32,74.58,66.07,95.58],"score":0.647},{"image_id":699,"category_id":10,"bbox":[23.1,171.18,85.54,70.98],"score":0.32},{"image_id":699,"category_id":10,"bbox":[44.63,0.69,46.46,74.21],"score":0.38},{"image_id":711,"category_id":17,"bbox":[451.41,164.9,45.52,34.77],"score":0.138},{"image_id":711,"category_id":64,"bbox":[349.66,148.34,41.34,20.79],"score":0.708},{"image_id":711,"category_id":75,"bbox":[208.59,342.4,26.64,28.5],"score":0.394},{"image_id":711,"category_id":49,"bbox":[288.32,314.87,53.95,21.46],"score":0.49},{"image_id":711,"category_id":62,"bbox":[554.21,165.91,35.93,105.94],"score":0.162},{"image_id":711,"category_id":64,"bbox":[280.08,175.71,18.23,27.81],"score":0.296},{"image_id":711,"category_id":86,"bbox":[404.93,65.59,8.78,20.64],"score":0.793},{"image_id":711,"category_id":86,"bbox":[330.66,118.24,8.79,25.07],"score":0.847},{"image_id":711,"category_id":64,"bbox":[557.89,126.82,23.63,29.7],"score":0.281},{"image_id":711,"category_id":86,"bbox":[45.69,87.06,26.69,34.26],"score":0.982},{"image_id":711,"category_id":86,"bbox":[418.67,91.88,16.87,21.9],"score":0.035},{"image_id":711,"category_id":84,"bbox":[237.28,352.6,38.7,21.15],"score":0.339},{"image_id":711,"category_id":63,"bbox":[9,164.54,302.22,241.78],"score":0.438},{"image_id":715,"category_id":52,"bbox":[237.81,243.61,62.21,79.23],"score":0.576},{"image_id":715,"category_id":52,"bbox":[461.66,324.67,56.09,81.98],"score":0.555},{"image_id":715,"category_id":52,"bbox":[244.25,244.91,283.06,188.98],"score":0.524},{"image_id":715,"category_id":47,"bbox":[357.38,121.03,40.84,42.26],"score":0.962},{"image_id":715,"category_id":82,"bbox":[473.35,114.89,43.93,43.2],"score":0.812},{"image_id":715,"category_id":47,"bbox":[515.91,110.3,43.77,36.36],"score":0.277},{"image_id":715,"category_id":47,"bbox":[282.12,88.59,27.93,33.33],"score":0.813},{"image_id":715,"category_id":47,"bbox":[440.09,106.55,41.2,33.46],"score":0.34},{"image_id":715,"category_id":47,"bbox":[397.82,109.4,41.38,29.87],"score":0.344},{"image_id":715,"category_id":53,"bbox":[33.65,141.22,16.4,19.03],"score":0.746},{"image_id":715,"category_id":53,"bbox":[49.96,146.83,19,18.43],"score":0.871},{"image_id":715,"category_id":53,"bbox":[6.48,140.32,16.95,14.84],"score":0.522},{"image_id":715,"category_id":54,"bbox":[75,108.53,35.91,18.24],"score":0.322},{"image_id":715,"category_id":54,"bbox":[127.53,60.89,37.18,18.9],"score":0.881},{"image_id":715,"category_id":54,"bbox":[46.93,119.59,77.47,35.69],"score":0.249},{"image_id":715,"category_id":54,"bbox":[64.04,81.55,30.36,18.58],"score":0.161},{"image_id":715,"category_id":55,"bbox":[128.2,270.78,32.69,33.61],"score":0.627},{"image_id":715,"category_id":55,"bbox":[291.13,154.15,37.18,44.92],"score":0.799},{"image_id":715,"category_id":22,"bbox":[211.31,146.7,377.53,135.91],"score":0.356},{"image_id":715,"category_id":55,"bbox":[149.11,142.74,27.79,26.37],"score":0.112},{"image_id":715,"category_id":55,"bbox":[120.2,146.07,42.61,37.28],"score":0.473},{"image_id":715,"category_id":55,"bbox":[87.2,285.29,34.6,31.79],"score":0.959},{"image_id":715,"category_id":55,"bbox":[235.1,337.96,42.86,42.35],"score":0.407},{"image_id":715,"category_id":55,"bbox":[459.66,415.28,47.46,51.78],"score":0.753},{"image_id":715,"category_id":55,"bbox":[480.16,373.21,48.54,44.23],"score":0.833},{"image_id":715,"category_id":47,"bbox":[312.85,78.82,32.21,42.18],"score":0.877},{"image_id":715,"category_id":47,"bbox":[183.7,90.85,31.08,29.98],"score":0.308},{"image_id":715,"category_id":53,"bbox":[86.77,156.3,16.45,15.84],"score":0.871},{"image_id":715,"category_id":55,"bbox":[168.64,317.06,38.04,34.34],"score":0.31},{"image_id":715,"category_id":55,"bbox":[131.22,303.17,36.59,32.39],"score":0.556},{"image_id":715,"category_id":55,"bbox":[480.72,346.82,39.77,35.88],"score":0.433},{"image_id":715,"category_id":47,"bbox":[256.88,98.33,29.03,38.79],"score":0.787},{"image_id":715,"category_id":64,"bbox":[90.72,98.04,32.83,16.77],"score":0.638},{"image_id":715,"category_id":58,"bbox":[54.94,49.87,36.92,19.98],"score":0.313},{"image_id":724,"category_id":13,"bbox":[123.07,71.83,134.49,153.08],"score":0.132},{"image_id":724,"category_id":8,"bbox":[121.39,279.44,21.74,30.17],"score":0.449},{"image_id":724,"category_id":47,"bbox":[127.26,267.21,12.88,8.15],"score":0.141},{"image_id":724,"category_id":13,"bbox":[202.38,259.9,19.01,26.06],"score":0.388},{"image_id":730,"category_id":3,"bbox":[29.45,275.25,88.35,76.3],"score":0.304},{"image_id":730,"category_id":3,"bbox":[430.55,288.02,54.48,57.65],"score":0.611},{"image_id":730,"category_id":36,"bbox":[122.58,109.89,328.93,279.88],"score":0.094},]

注:框坐标是从图像左上角开始测量的浮点数(并且是0索引)。我们建议将坐标舍入到最接近十分之一像素的位置，以减少生成的JSON文件大小。

使用对象段(实例段)进行检测，请使用以下格式:

[{"image_id": int, "category_id": int, "segmentation": RLE, "score": float,
}]

评估参数如下 (括号中的默认值，一般不需要改变)

params{"imgIds"
: [all] N img ids to use for evaluation
"catIds"
: [all] K cat ids to use for evaluation
"iouThrs"
: [.5:.05:.95] T=10 IoU thresholds for evaluation
"recThrs"
: [0:.01:1] R=101 recall thresholds for evaluation
"areaRng"
: [all,small,medium,large] A=4 area ranges for evaluation
"maxDets"
: [1 10 100] M=3 thresholds on max detections per image
"useSegm"
: [1] if true evaluate against ground-truth segments
"useCats"
: [1] if true use category labels for evaluation
}

运行评估代码通过调用 evaluate() 和 accumulate() 产生两个数据结构来衡量检测质量。这两个结构分别是 evalImgs 和 eval，它们分别衡量每个图像的质量并聚合到整个数据集中。evalImgs 结构体具有 K x A 条目，每个评估设置一个，而 eval 结构体将这些信息组合成 precision 和 recall 数组。这两个结构的细节如下 (另请参阅 CocoEval.m 或cocoeval.py)：

evalImgs[{"dtIds"
: [1xD] id for each of the D detections (dt)
"gtIds"
: [1xG] id for each of the G ground truths (gt)
"dtImgIds"
: [1xD] image id for each dt
"gtImgIds"
: [1xG] image id for each gt
"dtMatches"
: [TxD] matching gt id at each IoU or 0
"gtMatches"
: [TxG] matching dt id at each IoU or 0
"dtScores"
: [1xD] confidence of each dt
"dtIgnore"
: [TxD] ignore flag for each dt at each IoU
"gtIgnore"
: [1xG] ignore flag for each gt
}]
eval{"params"
: parameters used for evaluation
"date"
: date evaluation was performed
"counts"
: [T,R,K,A,M] parameter dimensions (see above)
"precision"
: [TxRxKxAxM] precision for every evaluation setting
"recall"
: [TxKxAxM] max recall for every evaluation setting
}

最后，summary() 根据 eval 结构计算前面定义的 12 个检测指标。

coco.py代码

__author__ = 'tylin'
__version__ = '2.0'
# Interface for accessing the Microsoft COCO dataset.# Microsoft COCO is a large image dataset designed for object detection,
# segmentation, and caption generation. pycocotools is a Python API that
# assists in loading, parsing and visualizing the annotations in COCO.
# Please visit http://mscoco.org/ for more information on COCO, including
# for the data, paper, and tutorials. The exact format of the annotations
# is also described on the COCO website. For example usage of the pycocotools
# please see pycocotools_demo.ipynb. In addition to this API, please download both
# the COCO images and annotations in order to run the demo.# An alternative to using the API is to load the annotations directly
# into Python dictionary
# Using the API provides additional utility functions. Note that this API
# supports both *instance* and *caption* annotations. In the case of
# captions not all functions are defined (e.g. categories are undefined).# The following API functions are defined:
#  COCO       - COCO api class that loads COCO annotation file and prepare data structures.（COCO api类，加载COCO注释文件并准备数据结构）
#  decodeMask - Decode binary mask M encoded via run-length encoding.
#  encodeMask - Encode binary mask M using run-length encoding.
#  getAnnIds  - Get ann ids that satisfy given filter conditions.获取满足给定过滤条件的ann id
#  getCatIds  - Get cat ids that satisfy given filter conditions.获取满足给定过滤条件的Cat id
#  getImgIds  - Get img ids that satisfy given filter conditions.获取满足给定过滤条件的img id
#  loadAnns   - Load anns with the specified ids.加载具有指定id的ann
#  loadCats   - Load cats with the specified ids.
#  loadImgs   - Load imgs with the specified ids.
#  annToMask  - Convert segmentation in an annotation to binary mask.将注释中的分割转换为二进制掩码
#  showAnns   - Display the specified annotations.显示指定的注释
#  loadRes    - Load algorithm results and create API for accessing them.加载算法结果并创建用于访问它们的API
#  download   - Download COCO images from mscoco.org server.
# Throughout the API "ann"=annotation, "cat"=category, and "img"=image.
# Help on each functions can be accessed by: "help COCO>function".# See also COCO>decodeMask,
# COCO>encodeMask, COCO>getAnnIds, COCO>getCatIds,
# COCO>getImgIds, COCO>loadAnns, COCO>loadCats,
# COCO>loadImgs, COCO>annToMask, COCO>showAnns# Microsoft COCO Toolbox.      version 2.0
# Data, paper, and tutorials available at:  http://mscoco.org/
# Code written by Piotr Dollar and Tsung-Yi Lin, 2014.
# Licensed under the Simplified BSD License [see bsd.txt]import json
import time
import matplotlib.pyplot as plt
from matplotlib.collections import PatchCollection
from matplotlib.patches import Polygon
import numpy as np
import copy
import itertools
from . import mask as maskUtils
import os
from collections import defaultdict
import sys
PYTHON_VERSION = sys.version_info[0]
if PYTHON_VERSION == 2:from urllib import urlretrieve
elif PYTHON_VERSION == 3:from urllib.request import urlretrievedef _isArrayLike(obj):return hasattr(obj, '__iter__') and hasattr(obj, '__len__')class COCO:def __init__(self, annotation_file=None):"""Constructor of Microsoft COCO helper class for reading and visualizing annotations.:param annotation_file (str): location of annotation file:param image_folder (str): location to the folder that hosts images.:return:"""# load datasetself.dataset,self.anns,self.cats,self.imgs = dict(),dict(),dict(),dict()self.imgToAnns, self.catToImgs = defaultdict(list), defaultdict(list)if not annotation_file == None:print('loading annotations into memory...')tic = time.time()dataset = json.load(open(annotation_file, 'r'))assert type(dataset)==dict, 'annotation file format {} not supported'.format(type(dataset))print('Done (t={:0.2f}s)'.format(time.time()- tic))self.dataset = datasetself.createIndex()def createIndex(self):# create indexprint('creating index...')anns, cats, imgs = {}, {}, {}imgToAnns,catToImgs = defaultdict(list),defaultdict(list)if 'annotations' in self.dataset:for ann in self.dataset['annotations']:imgToAnns[ann['image_id']].append(ann)anns[ann['id']] = annif 'images' in self.dataset:for img in self.dataset['images']:imgs[img['id']] = imgif 'categories' in self.dataset:for cat in self.dataset['categories']:cats[cat['id']] = catif 'annotations' in self.dataset and 'categories' in self.dataset:for ann in self.dataset['annotations']:catToImgs[ann['category_id']].append(ann['image_id'])print('index created!')# create class membersself.anns = annsself.imgToAnns = imgToAnnsself.catToImgs = catToImgsself.imgs = imgsself.cats = catsdef info(self):"""Print information about the annotation file.:return:"""for key, value in self.dataset['info'].items():print('{}: {}'.format(key, value))def getAnnIds(self, imgIds=[], catIds=[], areaRng=[], iscrowd=None):"""Get ann ids that satisfy given filter conditions. default skips that filter:param imgIds  (int array)     : get anns for given imgscatIds  (int array)     : get anns for given catsareaRng (float array)   : get anns for given area range (e.g. [0 inf])iscrowd (boolean)       : get anns for given crowd label (False or True):return: ids (int array)       : integer array of ann ids"""imgIds = imgIds if _isArrayLike(imgIds) else [imgIds]catIds = catIds if _isArrayLike(catIds) else [catIds]if len(imgIds) == len(catIds) == len(areaRng) == 0:anns = self.dataset['annotations']else:if not len(imgIds) == 0:lists = [self.imgToAnns[imgId] for imgId in imgIds if imgId in self.imgToAnns]anns = list(itertools.chain.from_iterable(lists))else:anns = self.dataset['annotations']anns = anns if len(catIds)  == 0 else [ann for ann in anns if ann['category_id'] in catIds]anns = anns if len(areaRng) == 0 else [ann for ann in anns if ann['area'] > areaRng[0] and ann['area'] < areaRng[1]]if not iscrowd == None:ids = [ann['id'] for ann in anns if ann['iscrowd'] == iscrowd]else:ids = [ann['id'] for ann in anns]return idsdef getCatIds(self, catNms=[], supNms=[], catIds=[]):"""#返回类别idfiltering parameters. default skips that filter.:param catNms (str array)  : get cats for given cat names:param supNms (str array)  : get cats for given supercategory names:param catIds (int array)  : get cats for given cat ids:return: ids (int array)   : integer array of cat ids"""catNms = catNms if _isArrayLike(catNms) else [catNms]supNms = supNms if _isArrayLike(supNms) else [supNms]catIds = catIds if _isArrayLike(catIds) else [catIds]if len(catNms) == len(supNms) == len(catIds) == 0:cats = self.dataset['categories'] #类别的总类else:cats = self.dataset['categories']cats = cats if len(catNms) == 0 else [cat for cat in cats if cat['name']          in catNms]cats = cats if len(supNms) == 0 else [cat for cat in cats if cat['supercategory'] in supNms]cats = cats if len(catIds) == 0 else [cat for cat in cats if cat['id']            in catIds]ids = [cat['id'] for cat in cats]return idsdef getImgIds(self, imgIds=[], catIds=[]):'''Get img ids that satisfy given filter conditions.获取满足给定过滤条件的img id。获取满足给定过滤条件的img id。:param imgIds (int array) : get imgs for given ids:param catIds (int array) : get imgs with all given cats:return: ids (int array)  : integer array of img ids'''imgIds = imgIds if _isArrayLike(imgIds) else [imgIds]catIds = catIds if _isArrayLike(catIds) else [catIds]if len(imgIds) == len(catIds) == 0:ids = self.imgs.keys()else:ids = set(imgIds)for i, catId in enumerate(catIds):if i == 0 and len(ids) == 0:ids = set(self.catToImgs[catId])else:ids &= set(self.catToImgs[catId])return list(ids)def loadAnns(self, ids=[]):"""Load anns with the specified ids.:param ids (int array)       : integer ids specifying anns:return: anns (object array) : loaded ann objects"""if _isArrayLike(ids):return [self.anns[id] for id in ids]elif type(ids) == int:return [self.anns[ids]]def loadCats(self, ids=[]):"""Load cats with the specified ids.:param ids (int array)       : integer ids specifying cats:return: cats (object array) : loaded cat objects"""if _isArrayLike(ids):return [self.cats[id] for id in ids]elif type(ids) == int:return [self.cats[ids]]def loadImgs(self, ids=[]):"""Load anns with the specified ids.使用指定的id加载ann。:param ids (int array)       : integer ids specifying img:return: imgs (object array) : loaded img objects"""if _isArrayLike(ids):return [self.imgs[id] for id in ids]elif type(ids) == int:return [self.imgs[ids]]def showAnns(self, anns, draw_bbox=False):"""Display the specified annotations.显示指定的注释:param anns (array of object): annotations to display:return: None"""if len(anns) == 0:return 0if 'segmentation' in anns[0] or 'keypoints' in anns[0]:datasetType = 'instances'elif 'caption' in anns[0]:datasetType = 'captions'else:raise Exception('datasetType not supported')if datasetType == 'instances':ax = plt.gca()ax.set_autoscale_on(False)polygons = []color = []for ann in anns:c = (np.random.random((1, 3))*0.6+0.4).tolist()[0]if 'segmentation' in ann:if type(ann['segmentation']) == list:# polygonfor seg in ann['segmentation']:poly = np.array(seg).reshape((int(len(seg)/2), 2))polygons.append(Polygon(poly))color.append(c)else:# maskt = self.imgs[ann['image_id']]if type(ann['segmentation']['counts']) == list:rle = maskUtils.frPyObjects([ann['segmentation']], t['height'], t['width'])else:rle = [ann['segmentation']]m = maskUtils.decode(rle)img = np.ones( (m.shape[0], m.shape[1], 3) )if ann['iscrowd'] == 1:color_mask = np.array([2.0,166.0,101.0])/255if ann['iscrowd'] == 0:color_mask = np.random.random((1, 3)).tolist()[0]for i in range(3):img[:,:,i] = color_mask[i]ax.imshow(np.dstack( (img, m*0.5) ))if 'keypoints' in ann and type(ann['keypoints']) == list:# turn skeleton into zero-based indexsks = np.array(self.loadCats(ann['category_id'])[0]['skeleton'])-1kp = np.array(ann['keypoints'])x = kp[0::3]y = kp[1::3]v = kp[2::3]for sk in sks:if np.all(v[sk]>0):plt.plot(x[sk],y[sk], linewidth=3, color=c)plt.plot(x[v>0], y[v>0],'o',markersize=8, markerfacecolor=c, markeredgecolor='k',markeredgewidth=2)plt.plot(x[v>1], y[v>1],'o',markersize=8, markerfacecolor=c, markeredgecolor=c, markeredgewidth=2)if draw_bbox:[bbox_x, bbox_y, bbox_w, bbox_h] = ann['bbox']poly = [[bbox_x, bbox_y], [bbox_x, bbox_y+bbox_h], [bbox_x+bbox_w, bbox_y+bbox_h], [bbox_x+bbox_w, bbox_y]]np_poly = np.array(poly).reshape((4,2))polygons.append(Polygon(np_poly))color.append(c)p = PatchCollection(polygons, facecolor=color, linewidths=0, alpha=0.4)ax.add_collection(p)p = PatchCollection(polygons, facecolor='none', edgecolors=color, linewidths=2)ax.add_collection(p)elif datasetType == 'captions':for ann in anns:print(ann['caption'])def loadRes(self, resFile):"""Load result file and return a result api object.加载结果文件并返回一个结果api对象:param   resFile (str)     : file name of result file:return: res (obj)         : result api object"""res = COCO()res.dataset['images'] = [img for img in self.dataset['images']]print('Loading and preparing results...')tic = time.time()if type(resFile) == str or (PYTHON_VERSION == 2 and type(resFile) == unicode):anns = json.load(open(resFile))elif type(resFile) == np.ndarray:anns = self.loadNumpyAnnotations(resFile)else:anns = resFileassert type(anns) == list, 'results in not an array of objects'annsImgIds = [ann['image_id'] for ann in anns]assert set(annsImgIds) == (set(annsImgIds) & set(self.getImgIds())), \'Results do not correspond to current coco set'if 'caption' in anns[0]:imgIds = set([img['id'] for img in res.dataset['images']]) & set([ann['image_id'] for ann in anns])res.dataset['images'] = [img for img in res.dataset['images'] if img['id'] in imgIds]for id, ann in enumerate(anns):ann['id'] = id+1elif 'bbox' in anns[0] and not anns[0]['bbox'] == []:res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])for id, ann in enumerate(anns):bb = ann['bbox']x1, x2, y1, y2 = [bb[0], bb[0]+bb[2], bb[1], bb[1]+bb[3]]if not 'segmentation' in ann:ann['segmentation'] = [[x1, y1, x1, y2, x2, y2, x2, y1]]ann['area'] = bb[2]*bb[3]ann['id'] = id+1ann['iscrowd'] = 0elif 'segmentation' in anns[0]:res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])for id, ann in enumerate(anns):# now only support compressed RLE format as segmentation resultsann['area'] = maskUtils.area(ann['segmentation'])if not 'bbox' in ann:ann['bbox'] = maskUtils.toBbox(ann['segmentation'])ann['id'] = id+1ann['iscrowd'] = 0elif 'keypoints' in anns[0]:res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])for id, ann in enumerate(anns):s = ann['keypoints']x = s[0::3]y = s[1::3]x0,x1,y0,y1 = np.min(x), np.max(x), np.min(y), np.max(y)ann['area'] = (x1-x0)*(y1-y0)ann['id'] = id + 1ann['bbox'] = [x0,y0,x1-x0,y1-y0]print('DONE (t={:0.2f}s)'.format(time.time()- tic))res.dataset['annotations'] = annsres.createIndex()return resdef download(self, tarDir = None, imgIds = [] ):'''Download COCO images from mscoco.org server.:param tarDir (str): COCO results directory nameimgIds (list): images to be downloaded:return:'''if tarDir is None:print('Please specify target directory')return -1if len(imgIds) == 0:imgs = self.imgs.values()else:imgs = self.loadImgs(imgIds)N = len(imgs)if not os.path.exists(tarDir):os.makedirs(tarDir)for i, img in enumerate(imgs):tic = time.time()fname = os.path.join(tarDir, img['file_name'])if not os.path.exists(fname):urlretrieve(img['coco_url'], fname)print('downloaded {}/{} images (t={:0.1f}s)'.format(i, N, time.time()- tic))def loadNumpyAnnotations(self, data):"""Convert result data from a numpy array [Nx7] where each row contains {imageID,x1,y1,w,h,score,class}:param  data (numpy.ndarray):return: annotations (python nested list)"""print('Converting ndarray to lists...')assert(type(data) == np.ndarray)print(data.shape)assert(data.shape[1] == 7)N = data.shape[0]ann = []for i in range(N):if i % 1000000 == 0:print('{}/{}'.format(i,N))ann += [{'image_id'  : int(data[i, 0]),'bbox'  : [ data[i, 1], data[i, 2], data[i, 3], data[i, 4] ],'score' : data[i, 5],'category_id': int(data[i, 6]),}]return anndef annToRLE(self, ann):"""Convert annotation which can be polygons, uncompressed RLE to RLE.:return: binary mask (numpy 2D array)"""t = self.imgs[ann['image_id']]h, w = t['height'], t['width']segm = ann['segmentation']if type(segm) == list:# polygon -- a single object might consist of multiple parts# we merge all parts into one mask rle coderles = maskUtils.frPyObjects(segm, h, w)rle = maskUtils.merge(rles)elif type(segm['counts']) == list:# uncompressed RLErle = maskUtils.frPyObjects(segm, h, w)else:# rlerle = ann['segmentation']return rledef annToMask(self, ann):"""Convert annotation which can be polygons, uncompressed RLE, or RLE to binary mask.:return: binary mask (numpy 2D array)"""rle = self.annToRLE(ann)m = maskUtils.decode(rle)return m

coco_demo.py代码

'''
Author: 刘鸿燕 13752614153@163.com
Date: 2022-05-16 10:31:46
LastEditors: 刘鸿燕 13752614153@163.com
LastEditTime: 2022-05-16 11:06:23
FilePath: \cocoapi-master\PythonAPI\coco_demo.py
Description: 这是默认设置,请设置`customMade`, 打开koroFileHeader查看配置 进行设置: https://github.com/OBKoro1/koro1FileHeader/wiki/%E9%85%8D%E7%BD%AE
'''
from pycocotools.coco import COCO
import numpy as np
import skimage.io as io
import matplotlib.pyplot as plt
import pylab
pylab.rcParams['figure.figsize'] = (8.0, 10.0)dataDir='D:/ObjectDetection/Transform_Object/DETR Transformer/detr-main/data/coco/Detection'
dataType='val2017'
annFile='{}/annotations/instances_{}.json'.format(dataDir,dataType)# initialize COCO api for instance annotations
coco=COCO(annFile)# display COCO categories and supercategories
cats = coco.loadCats(coco.getCatIds())
nms=[cat['name'] for cat in cats]
print('COCO categories: \n{}\n'.format(' '.join(nms)))nms = set([cat['supercategory'] for cat in cats])
print('COCO supercategories: \n{}'.format(' '.join(nms)))# get all images containing given categories, select one at random
# 获取所有包含给定类别的图像，随机选择一个
catIds = coco.getCatIds(catNms=['person','dog','skateboard']);
imgIds = coco.getImgIds(catIds=catIds );
imgIds = coco.getImgIds(imgIds = [324158])
img = coco.loadImgs(imgIds[np.random.randint(0,len(imgIds))])[0]# load and display image
# I = io.imread('%s/images/%s/%s'%(dataDir,dataType,img['file_name']))
# use url to load image
I = io.imread(img['coco_url'])
plt.axis('off')
plt.imshow(I)
plt.show()# load and display instance annotations
plt.imshow(I); plt.axis('off')
annIds = coco.getAnnIds(imgIds=img['id'], catIds=catIds, iscrowd=None)
anns = coco.loadAnns(annIds)
coco.showAnns(anns)
plt.show()

cocoeval.py

__author__ = 'tsungyi'import numpy as np
import datetime
import time
from collections import defaultdict
from . import mask as maskUtils
import copyclass COCOeval:# Interface for evaluating detection on the Microsoft COCO dataset.## The usage for CocoEval is as follows:#  cocoGt=..., cocoDt=...       # load dataset and results，加载数据集和结果#  E = CocoEval(cocoGt,cocoDt); # initialize CocoEval object，初始化CocoEval对象#  E.params.recThrs = ...;      # set parameters as desired，根据需要设置参数#  E.evaluate();                # run per image evaluation，每次图像评估时运行#  E.accumulate();              # accumulate per image results，累计每个图像结果#  E.summarize();               # display summary metrics of results，显示结果的汇总指标# For example usage see evalDemo.m and http://mscoco.org/.## The evaluation parameters are as follows (defaults in brackets):#  imgIds     - [all] N img ids to use for evaluation，N个img id用于评估#  catIds     - [all] K cat ids to use for evaluation#  iouThrs    - [.5:.05:.95] T=10 IoU thresholds for evaluation#  recThrs    - [0:.01:1] R=101 recall thresholds for evaluation#  areaRng    - [...] A=4 object area ranges for evaluation#  maxDets    - [1 10 100] M=3 thresholds on max detections per image#  iouType    - ['segm'] set iouType to 'segm', 'bbox' or 'keypoints'#  iouType replaced the now DEPRECATED useSegm parameter.#  useCats    - [1] if true use category labels for evaluation# Note: if useCats=0 category labels are ignored as in proposal scoring.# Note: multiple areaRngs [Ax2] and maxDets [Mx1] can be specified.## evaluate(): evaluates detections on every image and every category and# concats the results into the "evalImgs" with fields:#  dtIds      - [1xD] id for each of the D detections (dt)#  gtIds      - [1xG] id for each of the G ground truths (gt)#  dtMatches  - [TxD] matching gt id at each IoU or 0#  gtMatches  - [TxG] matching dt id at each IoU or 0#  dtScores   - [1xD] confidence of each dt#  gtIgnore   - [1xG] ignore flag for each gt#  dtIgnore   - [TxD] ignore flag for each dt at each IoU## accumulate(): accumulates the per-image, per-category evaluation# results in "evalImgs" into the dictionary "eval" with fields:#  params     - parameters used for evaluation#  date       - date evaluation was performed#  counts     - [T,R,K,A,M] parameter dimensions (see above)#  precision  - [TxRxKxAxM] precision for every evaluation setting#  recall     - [TxKxAxM] max recall for every evaluation setting# Note: precision and recall==-1 for settings with no gt objects.## See also coco, mask, pycocoDemo, pycocoEvalDemo## Microsoft COCO Toolbox.      version 2.0# Data, paper, and tutorials available at:  http://mscoco.org/# Code written by Piotr Dollar and Tsung-Yi Lin, 2015.# Licensed under the Simplified BSD License [see coco/license.txt]def __init__(self, cocoGt=None, cocoDt=None, iouType='segm'):'''Initialize CocoEval using coco APIs for gt and dt:param cocoGt: coco object with ground truth annotations:param cocoDt: coco object with detection results:return: None'''if not iouType:print('iouType not specified. use default iouType segm')self.cocoGt   = cocoGt              # ground truth COCO APIself.cocoDt   = cocoDt              # detections COCO APIself.evalImgs = defaultdict(list)   # per-image per-category evaluation results [KxAxI] elementsself.eval     = {}                  # accumulated evaluation resultsself._gts = defaultdict(list)       # gt for evaluationself._dts = defaultdict(list)       # dt for evaluationself.params = Params(iouType=iouType) # parametersself._paramsEval = {}               # parameters for evaluationself.stats = []                     # result summarizationself.ious = {}                      # ious between all gts and dtsif not cocoGt is None:self.params.imgIds = sorted(cocoGt.getImgIds())self.params.catIds = sorted(cocoGt.getCatIds())def _prepare(self):'''Prepare ._gts and ._dts for evaluation based on params:return: None'''def _toMask(anns, coco):# modify ann['segmentation'] by referencefor ann in anns:rle = coco.annToRLE(ann)ann['segmentation'] = rlep = self.paramsif p.useCats:gts=self.cocoGt.loadAnns(self.cocoGt.getAnnIds(imgIds=p.imgIds, catIds=p.catIds))dts=self.cocoDt.loadAnns(self.cocoDt.getAnnIds(imgIds=p.imgIds, catIds=p.catIds))else:gts=self.cocoGt.loadAnns(self.cocoGt.getAnnIds(imgIds=p.imgIds))dts=self.cocoDt.loadAnns(self.cocoDt.getAnnIds(imgIds=p.imgIds))# convert ground truth to mask if iouType == 'segm'if p.iouType == 'segm':_toMask(gts, self.cocoGt)_toMask(dts, self.cocoDt)# set ignore flagfor gt in gts:gt['ignore'] = gt['ignore'] if 'ignore' in gt else 0gt['ignore'] = 'iscrowd' in gt and gt['iscrowd']if p.iouType == 'keypoints':gt['ignore'] = (gt['num_keypoints'] == 0) or gt['ignore']self._gts = defaultdict(list)       # gt for evaluationself._dts = defaultdict(list)       # dt for evaluationfor gt in gts:self._gts[gt['image_id'], gt['category_id']].append(gt)for dt in dts:self._dts[dt['image_id'], dt['category_id']].append(dt)self.evalImgs = defaultdict(list)   # per-image per-category evaluation resultsself.eval     = {}                  # accumulated evaluation resultsdef evaluate(self):'''Run per image evaluation on given images and store results (a list of dict) in self.evalImgs:return: None'''tic = time.time()print('Running per image evaluation...')p = self.params# add backward compatibility if useSegm is specified in paramsif not p.useSegm is None:p.iouType = 'segm' if p.useSegm == 1 else 'bbox'print('useSegm (deprecated) is not None. Running {} evaluation'.format(p.iouType))print('Evaluate annotation type *{}*'.format(p.iouType))p.imgIds = list(np.unique(p.imgIds))if p.useCats:p.catIds = list(np.unique(p.catIds))p.maxDets = sorted(p.maxDets)self.params=pself._prepare()# loop through images, area range, max detection numbercatIds = p.catIds if p.useCats else [-1]if p.iouType == 'segm' or p.iouType == 'bbox':computeIoU = self.computeIoUelif p.iouType == 'keypoints':computeIoU = self.computeOksself.ious = {(imgId, catId): computeIoU(imgId, catId) \for imgId in p.imgIdsfor catId in catIds}evaluateImg = self.evaluateImgmaxDet = p.maxDets[-1]self.evalImgs = [evaluateImg(imgId, catId, areaRng, maxDet)for catId in catIdsfor areaRng in p.areaRngfor imgId in p.imgIds]self._paramsEval = copy.deepcopy(self.params)toc = time.time()print('DONE (t={:0.2f}s).'.format(toc-tic))def computeIoU(self, imgId, catId):p = self.paramsif p.useCats:gt = self._gts[imgId,catId]dt = self._dts[imgId,catId]else:gt = [_ for cId in p.catIds for _ in self._gts[imgId,cId]]dt = [_ for cId in p.catIds for _ in self._dts[imgId,cId]]if len(gt) == 0 and len(dt) ==0:return []inds = np.argsort([-d['score'] for d in dt], kind='mergesort')dt = [dt[i] for i in inds]if len(dt) > p.maxDets[-1]:dt=dt[0:p.maxDets[-1]]if p.iouType == 'segm':g = [g['segmentation'] for g in gt]d = [d['segmentation'] for d in dt]elif p.iouType == 'bbox':g = [g['bbox'] for g in gt]d = [d['bbox'] for d in dt]else:raise Exception('unknown iouType for iou computation')# compute iou between each dt and gt regioniscrowd = [int(o['iscrowd']) for o in gt]ious = maskUtils.iou(d,g,iscrowd)return iousdef computeOks(self, imgId, catId):p = self.params# dimention here should be Nxmgts = self._gts[imgId, catId]dts = self._dts[imgId, catId]inds = np.argsort([-d['score'] for d in dts], kind='mergesort')dts = [dts[i] for i in inds]if len(dts) > p.maxDets[-1]:dts = dts[0:p.maxDets[-1]]# if len(gts) == 0 and len(dts) == 0:if len(gts) == 0 or len(dts) == 0:return []ious = np.zeros((len(dts), len(gts)))sigmas = p.kpt_oks_sigmasvars = (sigmas * 2)**2k = len(sigmas)# compute oks between each detection and ground truth objectfor j, gt in enumerate(gts):# create bounds for ignore regions(double the gt bbox)g = np.array(gt['keypoints'])xg = g[0::3]; yg = g[1::3]; vg = g[2::3]k1 = np.count_nonzero(vg > 0)bb = gt['bbox']x0 = bb[0] - bb[2]; x1 = bb[0] + bb[2] * 2y0 = bb[1] - bb[3]; y1 = bb[1] + bb[3] * 2for i, dt in enumerate(dts):d = np.array(dt['keypoints'])xd = d[0::3]; yd = d[1::3]if k1>0:# measure the per-keypoint distance if keypoints visibledx = xd - xgdy = yd - ygelse:# measure minimum distance to keypoints in (x0,y0) & (x1,y1)z = np.zeros((k))dx = np.max((z, x0-xd),axis=0)+np.max((z, xd-x1),axis=0)dy = np.max((z, y0-yd),axis=0)+np.max((z, yd-y1),axis=0)e = (dx**2 + dy**2) / vars / (gt['area']+np.spacing(1)) / 2if k1 > 0:e=e[vg > 0]ious[i, j] = np.sum(np.exp(-e)) / e.shape[0]return iousdef evaluateImg(self, imgId, catId, aRng, maxDet):'''perform evaluation for single category and image:return: dict (single image results)'''p = self.paramsif p.useCats:gt = self._gts[imgId,catId]dt = self._dts[imgId,catId]else:gt = [_ for cId in p.catIds for _ in self._gts[imgId,cId]]dt = [_ for cId in p.catIds for _ in self._dts[imgId,cId]]if len(gt) == 0 and len(dt) ==0:return Nonefor g in gt:if g['ignore'] or (g['area']<aRng[0] or g['area']>aRng[1]):g['_ignore'] = 1else:g['_ignore'] = 0# sort dt highest score first, sort gt ignore lastgtind = np.argsort([g['_ignore'] for g in gt], kind='mergesort')gt = [gt[i] for i in gtind]dtind = np.argsort([-d['score'] for d in dt], kind='mergesort')dt = [dt[i] for i in dtind[0:maxDet]]iscrowd = [int(o['iscrowd']) for o in gt]# load computed iousious = self.ious[imgId, catId][:, gtind] if len(self.ious[imgId, catId]) > 0 else self.ious[imgId, catId]T = len(p.iouThrs)G = len(gt)D = len(dt)gtm  = np.zeros((T,G))dtm  = np.zeros((T,D))gtIg = np.array([g['_ignore'] for g in gt])dtIg = np.zeros((T,D))if not len(ious)==0:for tind, t in enumerate(p.iouThrs):for dind, d in enumerate(dt):# information about best match so far (m=-1 -> unmatched)iou = min([t,1-1e-10])m   = -1for gind, g in enumerate(gt):# if this gt already matched, and not a crowd, continueif gtm[tind,gind]>0 and not iscrowd[gind]:continue# if dt matched to reg gt, and on ignore gt, stopif m>-1 and gtIg[m]==0 and gtIg[gind]==1:break# continue to next gt unless better match madeif ious[dind,gind] < iou:continue# if match successful and best so far, store appropriatelyiou=ious[dind,gind]m=gind# if match made store id of match for both dt and gtif m ==-1:continuedtIg[tind,dind] = gtIg[m]dtm[tind,dind]  = gt[m]['id']gtm[tind,m]     = d['id']# set unmatched detections outside of area range to ignorea = np.array([d['area']<aRng[0] or d['area']>aRng[1] for d in dt]).reshape((1, len(dt)))dtIg = np.logical_or(dtIg, np.logical_and(dtm==0, np.repeat(a,T,0)))# store results for given image and categoryreturn {'image_id':     imgId,'category_id':  catId,'aRng':         aRng,'maxDet':       maxDet,'dtIds':        [d['id'] for d in dt],'gtIds':        [g['id'] for g in gt],'dtMatches':    dtm,'gtMatches':    gtm,'dtScores':     [d['score'] for d in dt],'gtIgnore':     gtIg,'dtIgnore':     dtIg,}def accumulate(self, p = None):'''Accumulate per image evaluation results and store the result in self.eval对每个图像的评价结果进行累积，并将结果存储在self.eval中:param p: input params for evaluation:return: None'''print('Accumulating evaluation results...')tic = time.time()if not self.evalImgs:print('Please run evaluate() first')# allows input customized parametersif p is None:p = self.paramsp.catIds = p.catIds if p.useCats == 1 else [-1]T           = len(p.iouThrs)R           = len(p.recThrs)K           = len(p.catIds) if p.useCats else 1A           = len(p.areaRng)M           = len(p.maxDets)precision   = -np.ones((T,R,K,A,M)) # -1 for the precision of absent categoriesrecall      = -np.ones((T,K,A,M))scores      = -np.ones((T,R,K,A,M))# create dictionary for future indexing_pe = self._paramsEvalcatIds = _pe.catIds if _pe.useCats else [-1]setK = set(catIds)setA = set(map(tuple, _pe.areaRng))setM = set(_pe.maxDets)setI = set(_pe.imgIds)# get inds to evaluatek_list = [n for n, k in enumerate(p.catIds)  if k in setK]m_list = [m for n, m in enumerate(p.maxDets) if m in setM]a_list = [n for n, a in enumerate(map(lambda x: tuple(x), p.areaRng)) if a in setA]i_list = [n for n, i in enumerate(p.imgIds)  if i in setI]I0 = len(_pe.imgIds)A0 = len(_pe.areaRng)# retrieve E at each category, area range, and max number of detectionsfor k, k0 in enumerate(k_list):Nk = k0*A0*I0for a, a0 in enumerate(a_list):Na = a0*I0for m, maxDet in enumerate(m_list):E = [self.evalImgs[Nk + Na + i] for i in i_list]E = [e for e in E if not e is None]if len(E) == 0:continuedtScores = np.concatenate([e['dtScores'][0:maxDet] for e in E])# different sorting method generates slightly different results.# mergesort is used to be consistent as Matlab implementation.inds = np.argsort(-dtScores, kind='mergesort')dtScoresSorted = dtScores[inds]dtm  = np.concatenate([e['dtMatches'][:,0:maxDet] for e in E], axis=1)[:,inds]dtIg = np.concatenate([e['dtIgnore'][:,0:maxDet]  for e in E], axis=1)[:,inds]gtIg = np.concatenate([e['gtIgnore'] for e in E])npig = np.count_nonzero(gtIg==0 )if npig == 0:continuetps = np.logical_and(               dtm,  np.logical_not(dtIg) )fps = np.logical_and(np.logical_not(dtm), np.logical_not(dtIg) )tp_sum = np.cumsum(tps, axis=1).astype(dtype=np.float)fp_sum = np.cumsum(fps, axis=1).astype(dtype=np.float)for t, (tp, fp) in enumerate(zip(tp_sum, fp_sum)):tp = np.array(tp)fp = np.array(fp)nd = len(tp)rc = tp / npigpr = tp / (fp+tp+np.spacing(1))q  = np.zeros((R,))ss = np.zeros((R,))if nd:recall[t,k,a,m] = rc[-1]else:recall[t,k,a,m] = 0# numpy is slow without cython optimization for accessing elements# use python array gets significant speed improvementpr = pr.tolist(); q = q.tolist()for i in range(nd-1, 0, -1):if pr[i] > pr[i-1]:pr[i-1] = pr[i]inds = np.searchsorted(rc, p.recThrs, side='left')try:for ri, pi in enumerate(inds):q[ri] = pr[pi]ss[ri] = dtScoresSorted[pi]except:passprecision[t,:,k,a,m] = np.array(q)scores[t,:,k,a,m] = np.array(ss)self.eval = {'params': p,'counts': [T, R, K, A, M],'date': datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'),'precision': precision,'recall':   recall,'scores': scores,}toc = time.time()print('DONE (t={:0.2f}s).'.format( toc-tic))def summarize(self):'''Compute and display summary metrics for evaluation results.计算和显示评估结果的汇总指标Note this functin can *only* be applied on the default parameter setting'''def _summarize( ap=1, iouThr=None, areaRng='all', maxDets=100 ):p = self.paramsiStr = ' {:<18} {} @[ IoU={:<9} | area={:>6s} | maxDets={:>3d} ] = {:0.3f}'titleStr = 'Average Precision' if ap == 1 else 'Average Recall'typeStr = '(AP)' if ap==1 else '(AR)'iouStr = '{:0.2f}:{:0.2f}'.format(p.iouThrs[0], p.iouThrs[-1]) \if iouThr is None else '{:0.2f}'.format(iouThr)aind = [i for i, aRng in enumerate(p.areaRngLbl) if aRng == areaRng]mind = [i for i, mDet in enumerate(p.maxDets) if mDet == maxDets]if ap == 1:# dimension of precision: [TxRxKxAxM]s = self.eval['precision']# IoUif iouThr is not None:t = np.where(iouThr == p.iouThrs)[0]s = s[t]s = s[:,:,:,aind,mind]else:# dimension of recall: [TxKxAxM]s = self.eval['recall']if iouThr is not None:t = np.where(iouThr == p.iouThrs)[0]s = s[t]s = s[:,:,aind,mind]if len(s[s>-1])==0:mean_s = -1else:mean_s = np.mean(s[s>-1])print(iStr.format(titleStr, typeStr, iouStr, areaRng, maxDets, mean_s))return mean_sdef _summarizeDets():stats = np.zeros((12,))stats[0] = _summarize(1)stats[1] = _summarize(1, iouThr=.5, maxDets=self.params.maxDets[2])stats[2] = _summarize(1, iouThr=.75, maxDets=self.params.maxDets[2])stats[3] = _summarize(1, areaRng='small', maxDets=self.params.maxDets[2])stats[4] = _summarize(1, areaRng='medium', maxDets=self.params.maxDets[2])stats[5] = _summarize(1, areaRng='large', maxDets=self.params.maxDets[2])stats[6] = _summarize(0, maxDets=self.params.maxDets[0])stats[7] = _summarize(0, maxDets=self.params.maxDets[1])stats[8] = _summarize(0, maxDets=self.params.maxDets[2])stats[9] = _summarize(0, areaRng='small', maxDets=self.params.maxDets[2])stats[10] = _summarize(0, areaRng='medium', maxDets=self.params.maxDets[2])stats[11] = _summarize(0, areaRng='large', maxDets=self.params.maxDets[2])return statsdef _summarizeKps():stats = np.zeros((10,))stats[0] = _summarize(1, maxDets=20)stats[1] = _summarize(1, maxDets=20, iouThr=.5)stats[2] = _summarize(1, maxDets=20, iouThr=.75)stats[3] = _summarize(1, maxDets=20, areaRng='medium')stats[4] = _summarize(1, maxDets=20, areaRng='large')stats[5] = _summarize(0, maxDets=20)stats[6] = _summarize(0, maxDets=20, iouThr=.5)stats[7] = _summarize(0, maxDets=20, iouThr=.75)stats[8] = _summarize(0, maxDets=20, areaRng='medium')stats[9] = _summarize(0, maxDets=20, areaRng='large')return statsif not self.eval:raise Exception('Please run accumulate() first')iouType = self.params.iouTypeif iouType == 'segm' or iouType == 'bbox':summarize = _summarizeDetselif iouType == 'keypoints':summarize = _summarizeKpsself.stats = summarize()def __str__(self):self.summarize()class Params:'''Params for coco evaluation api'''def setDetParams(self):self.imgIds = []self.catIds = []# np.arange causes trouble.  the data point on arange is slightly larger than the true valueself.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)self.recThrs = np.linspace(.0, 1.00, int(np.round((1.00 - .0) / .01)) + 1, endpoint=True)self.maxDets = [1, 10, 100]self.areaRng = [[0 ** 2, 1e5 ** 2], [0 ** 2, 32 ** 2], [32 ** 2, 96 ** 2], [96 ** 2, 1e5 ** 2]]self.areaRngLbl = ['all', 'small', 'medium', 'large']self.useCats = 1def setKpParams(self):self.imgIds = []self.catIds = []# np.arange causes trouble.  the data point on arange is slightly larger than the true valueself.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)self.recThrs = np.linspace(.0, 1.00, int(np.round((1.00 - .0) / .01)) + 1, endpoint=True)self.maxDets = [20]self.areaRng = [[0 ** 2, 1e5 ** 2], [32 ** 2, 96 ** 2], [96 ** 2, 1e5 ** 2]]self.areaRngLbl = ['all', 'medium', 'large']self.useCats = 1self.kpt_oks_sigmas = np.array([.26, .25, .25, .35, .35, .79, .79, .72, .72, .62,.62, 1.07, 1.07, .87, .87, .89, .89])/10.0def __init__(self, iouType='segm'):if iouType == 'segm' or iouType == 'bbox':self.setDetParams()elif iouType == 'keypoints':self.setKpParams()else:raise Exception('iouType not supported')self.iouType = iouType# useSegm is deprecatedself.useSegm = None

cocoeval_demo.py

import matplotlib.pyplot as plt
from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval
import numpy as np
import skimage.io as io
import pylab
pylab.rcParams['figure.figsize'] = (10.0, 8.0)annType = ['segm','bbox','keypoints']
annType = annType[1]      #specify type here,bbox
prefix = 'person_keypoints' if annType=='keypoints' else 'instances'
print ('Running demo for *%s* results.'%(annType))#initialize COCO ground truth api
dataDir='D:/ObjectDetection/Transform_Object/DETR Transformer/detr-main/data/coco/Detection'
dataType='val2017'
annFile = '%s/annotations/%s_%s.json'%(dataDir,prefix,dataType)
cocoGt=COCO(annFile)#initialize COCO detections api
resFile='%s/results/%s_%s_fake%s100_results.json'
resFile = resFile%(dataDir, prefix, dataType, annType)
cocoDt=cocoGt.loadRes(resFile)imgIds=sorted(cocoGt.getImgIds())
imgIds=imgIds[0:100]
imgId = imgIds[np.random.randint(100)]# running evaluation
cocoEval = COCOeval(cocoGt,cocoDt,annType)
cocoEval.params.imgIds  = imgIds
cocoEval.evaluate()
cocoEval.accumulate()
cocoEval.summarize()

Analysis Code

除了评估代码外，我们还提供一个函数 analyze() 来执行误报的详细分类。这受到了 Derek Hoiem 等人在诊断物体检测器中的错误 (Diagnosing Error in Object Detectors) 的启发，但在实现和细节方面却有很大不同。代码生成这样的图像：

这两幅图显示了来自 2015 年检测挑战赛获胜者 Kaiming He 等人的 ResNet (bbox) 检测器的分析结果。左图显示了 ResNet 的人员类别错误；右图是 ResNet 对所有类别平均值的整体分析。
每个绘图是一系列准确率召回率 (precision recall) 曲线，其中每个 PR 曲线被保证严格地高于之前的评估设置变得更宽容。曲线如下：

C75：在 IoU = 0.75 (严格的 IoU 的 AP) 的 PR (precision recall)，对应于 APIoU=.75 度量曲线下的面积 (area under curve)。
C50：在 IoU = 0.50 (PASCAL IoU 的 AP) 的 PR (precision recall)，对应于 APIoU=.50 度量曲线下的面积。
Loc：在 IoU = 0 .10 (定位误差 (localization errors ignored) 忽略，但不重复检测) 的 PR (precision recall)。所有其余的设置使用 IoU = 0.1。
Sim：超类别误报 (fps，false positives) 删除后的 PR。具体而言，与具有不同类标签但属于同一个超类别的对象的任何匹配都不会被视为 fp (或tp)。通过设置同一超类别中的所有对象与所讨论的类具有相同的类标签并将它们的忽略标志设置为1来计算 Sim。注意，人是单例超类别，因此其 Sim 结果与 Loc 完全相同。
Oth：所有类型混乱被删除后的PR。与 Sim 类似，除了现在如果检测与任何其他对象匹配，则不再是 fp (或tp)。Oth 通过如下方式进行计算，设置所有其他对象具有与所讨论的类相同的类标签，并将它们的忽略标志设置为 1。
BG：所有的背景(和类混乱 (class confusion)) fps 被删除后的 PR。对于单个类别，BG 是一个阶跃函数，直到达到最大召回后才降为0 (跨类别平均后曲线更平滑)。
FN：在所有剩余错误被删除后 (AP = 1) 的 PR。

每条曲线下面的区域显示在图例的括号中。在 ResNet 检测器的情况下，IoU = 0.75 的整体 AP 为 0.399，完美定位将使 AP 增加到 0.682。有趣的是，消除所有类别混乱 (超范畴内和超范畴内) 只会将 AP 略微提升至 0.713。除去背景 fp 会将性能提高到 0.870 AP，而其余的错误则缺少检测 (尽管假设更多的检测被添加，这也会增加大量的 fps)。总之，ResNet 的错误来自不完美的定位和背景混淆。
对于一个给定的检测器 (detector)，该代码总共产生了 372 个图 (plots)！共有 80 个类别 (category)，12 个超类别 (supercategory)，1 个总体结果，总共 93 个不同的设置，分析是在 4 个尺度 (scales) (全部，小，中，大，所以 93 * 4 = 372 个块) 进行。文件命名为 [supercategory] - [category] - [size].pdf (对于 80 * 4 每个分类结果)，overall-[supercategory]-[size].pdf (对于 12 * 4 每个超类别结果)，overall-all-[size].pdf 为 1 * 4 的整体结果。在所有plots 中，通常总体和超类别的结果是最感兴趣的。

注意：analyze() 可能需要很长时间才能运行，请耐心等待。因此，我们通常不会在评估服务器上运行此代码，您必须使用验证集在本地运行代码。最后，目前 analyze() 只是 Matlab API 的一部分，Python 代码即将推出。
wordbook
true positive，TP：真正例
true negative，TN：真反例
false positive，FP：假正例
false negative，FN：假反例
receiver operating characteristic curve，ROC curve：受试者工作特征曲线，ROC曲线，感受性曲线
Intersection over Union，IoU：交并比
recall：召回率，查全率
precision：准确率，查准率
average precision，AP
average recall，AR
mean Average Precision，mAP：平均精度均值
precision recall，PR

Visualization Code

FiftyOne

是一个开源工具，促进可视化和访问COCO数据资源，并作为COCO模型分析的评估工具。
FiftyOne提供coco风格的对象检测评估COCO-style object detection evaluation，不仅可以计算AP，还可以轻松地可视化单个样本和对象级别的结果visualize individual sample- and object-level results，查看精度-召回曲线view precision-recall curves，绘制交互混淆矩阵plot interactive confusion matrices.。
请参阅如何运行该指标的指南this guide。