实验 70:A64 Per-view Residual Decomposition

per-view residual 改善 global mean,但仍漏掉区域/方向错误

实验定位 per-view 正向但不足

A64 把 A61 的 global MV depth error 拆成 per-view residual:每个候选不再只有 mean MVD,而是记录每个输入视角相对 carrier 的改善/恶化、最大恶化视角、view spread 和 all-view-nonworse 状态。

结果是混合但很有价值:最好的 no-GT per-view selector 是 a64_min_view_spread,mean ΔChamfer=-0.0001264、ΔF@5=0.004870,比 A61 的 global mean selector 更好;但它距离 GT Chamfer oracle 仍差 0.0004317,且 gso008 有 all-view-improve 但 Chamfer 变差的反例。

实验设计(Image-2 风格)

实验设计图:A64 per-view residual decomposition
实验设计图:A64 per-view residual decomposition
模块设计图:per-view to per-region signed residual
模块设计图:per-view to per-region signed residual

模块设计(Image-2 风格)

Per-view Signals

  • mean_view_delta:每个视角 depth error 相对 carrier 的平均变化。
  • max_view_worsen:最坏视角的 residual 恶化量。
  • view_delta_spread:不同视角 residual 变化的分歧程度。
  • all_view_nonworse:所有输入视角都不比 carrier 差。

Missing Signal

per-view 只能说明某个相机的整体 depth 更像 VGGT,但不能判断是可见区域该外扩、该内缩,还是背景/遮挡区域被错误解释。因此 A65 需要 per-region signed residual。

实验结果(表格)

selector 汇总

selectormean ΔChamfermean ΔF@5mean max worsenmean spreadall-view nonworseselected iters
a64_min_mean_view_delta0.00005540.0034580.00011060.00071951{'2': 1, '3': 1, '4': 1}
a64_min_max_view_worsen0.00014750.0026860.00005350.00062532{'1': 2, '2': 1}
a64_min_view_spread-0.00012640.0048700.00005350.00028942{'1': 2, '8': 1}
a64_per_view_risk0.00029610.0005940.00009610.00061141{'1': 3}
a64_all_views_nonworse-0.00005570.0049510.00000000.00023493{'0': 1, '1': 1, '8': 1}
a64_majority_improve_low_worst0.00036690.0006750.00004260.00055692{'0': 1, '1': 2}
gt_chamfer_oracle-0.00055810.0114630.00244440.00107501{'3': 1, '8': 2}
gt_fscore_oracle-0.00051670.0116280.00249600.00108301{'4': 1, '8': 2}

case-level 失败模式

caseGT Chamfer iterGT F@5 iterper-view risk iterall-view itermin worst iterkey failure
gso_000_input434101best_chamfer_has_some_view_worsen
gso_002_input488182late_gt_improvement_with_small_view_worsen
gso_008_input488111all_views_improve_but_chamfer_worse

关键选择明细

caseselectoriterΔChamferΔF@5mean view Δmax worsenspreadworse viewsall nonworse
gso_000_input4a64_min_view_spread1-0.0002122-0.0002400.00002390.00016060.00016333False
gso_000_input4a64_all_views_nonworse00.00000000.0000000.00000000.00000000.00000000True
gso_000_input4gt_chamfer_oracle3-0.00041590.000512-0.00005640.00020230.00025202False
gso_002_input4a64_min_view_spread8-0.00160530.015600-0.00080490.00000000.00039510True
gso_002_input4a64_all_views_nonworse8-0.00160530.015600-0.00080490.00000000.00039510True
gso_002_input4gt_chamfer_oracle8-0.00160530.015600-0.00080490.00000000.00039510True
gso_008_input4a64_min_view_spread10.0014382-0.000748-0.00041710.00000000.00030970True
gso_008_input4a64_all_views_nonworse10.0014382-0.000748-0.00041710.00000000.00030970True
gso_008_input4gt_chamfer_oracle80.00034680.0182770.00282360.00713100.00257794False

per-view 信号与 GT Chamfer 的相关性

casesignalSpearman rhop-value
gso_000_input4mean_view_delta0.46670.2054
gso_000_input4max_view_worsen0.20000.6059
gso_000_input4view_delta_spread0.18330.6368
gso_000_input4per_view_risk0.21670.5755
gso_002_input4mean_view_delta-0.11670.7650
gso_002_input4max_view_worsen0.03650.9257
gso_002_input4view_delta_spread0.10000.7980
gso_002_input4per_view_risk-0.95000.0001
gso_008_input4mean_view_delta-0.03330.9322
gso_008_input4max_view_worsen0.41000.2730
gso_008_input4view_delta_spread0.40000.2861
gso_008_input4per_view_risk0.30000.4328

可视化结果

A64 Chamfer comparison
A64 Chamfer comparison
A64 F@5 comparison
A64 F@5 comparison
A64 remaining failure modes
A64 remaining failure modes

实验结论

下一步想法