实验 35:A29 Pixel Depth Residual Lock Map

从 VGGT/candidate depth 生成 per-pixel residual map,把局部锁定变成连续几何证据

实验定位 像素级诊断正结果

A29 把 A28 的离散 lock map 升级成 per-pixel VGGT depth residual map。它复用 A23 的 VGGT depth cache 和 baseline/repair mesh,重新渲染候选 depth 到输入相机,并在每个视角做 robust scale align,得到 repair-base |depth residual|

结果非常清楚:gso_002 的 repair 让 mean depth residual 改善 -0.02092,improve pixels=0.3660;gso_008 的 repair 让 mean depth residual 恶化 0.02270,worse pixels=0.3347。这把 A27/A28 的解释变成了像素级连续证据。

实验设计(Image-2 风格)

实验设计图:A29 pixel depth residual lock map
实验设计图:A29 pixel depth residual lock map
模块设计图:从 lock map 到 depth residual
模块设计图:从 lock map 到 depth residual

模块设计(Image-2 风格)

Legend

  • VGGT depth:blue-red relative depth visualization after downsampling
  • baseline/repair residual:absolute candidate depth residual against VGGT depth; colder means lower residual
  • repair-base |res|:red means repair worsens residual, blue means repair improves residual

Why It Matters

相比 A28 的 mask-level 区域标签,A29 给出了连续 depth residual:红色像素是 repair 让可见几何远离 VGGT,蓝色像素是 repair 让可见几何更接近 VGGT。这可以直接变成训练 loss、采样 guidance 或 surface/token lock 权重。

实验结果(表格)

casebaseline depth absrepair depth absrepair-baseline Δimprove pixel ratioworse pixel ratio
gso_002_input40.088380.06746-0.020920.36600.2277
gso_008_input40.023850.046560.022700.10240.3347

逐视角 residual

caseviewvalid pixelsbaseline depth absrepair depth absrepair-baseline Δimprove pixelsworse pixels
gso_002_input4023550.114080.05401-0.060070.77280.1792
gso_002_input4122680.076340.04071-0.035630.18610.0670
gso_002_input4219610.100330.118210.017890.36510.5783
gso_002_input4322960.062790.05691-0.005870.13980.0862
gso_008_input402090.029970.109000.079030.11480.8325
gso_008_input418950.011150.016120.004970.03240.1799
gso_008_input424060.046140.052700.006570.22170.2783
gso_008_input439120.008160.008410.000250.04060.0482

可视化结果

A29 pixel depth residual metrics
A29 pixel depth residual metrics
gso_002 四视角 depth residual panel
gso_002 四视角 depth residual panel
gso_008 四视角 depth residual panel
gso_008 四视角 depth residual panel
gso_008 view0:repair depth residual 明显变差
gso_008 view0:repair depth residual 明显变差
gso_002 view0:repair depth residual 明显改善
gso_002 view0:repair depth residual 明显改善
gso_002 view2:局部 depth conflict,但非全局拒绝
gso_002 view2:局部 depth conflict,但非全局拒绝

实验结论

下一步想法