实验定位 可行动诊断 / 仍需深度约束
A21 把 A20 的 2D visible-safe 思路推进到 3D:对 baseline 和 repair mesh 采样表面点,投影回原始 4 个输入相机,并用候选自身的近似 z-buffer 过滤背面点。一个点只有在“自可见”且落入输入前景 mask 时,才被认为是 visible-supported surface。
结果说明:3D support field 可以作为区域锁定的基础,但单独使用 silhouette-support 仍不够。repair 的平均 visible support 只比 baseline 高 0.0004,但 unsupported self-visible 也增加 0.0021;gso_008 仍主要依赖 A20 的逐像素可见损失才能被拒绝。
实验设计(Image-2 风格)
模块设计(Image-2 风格)
Support Field Definition
- sample points on candidate mesh surface
- project each point into the 4 original input cameras
- a point is input-supported if it lands inside at least one input foreground mask
- unsupported-front means the point is in front of at least one camera but never supported by an input mask
Repair Gate
- reject repair if support_rate drops by more than 1.5%
- reject repair if unsupported_front_rate rises by more than 3.5%
- reject repair if A20 2D visible-safe gate rejected it
实验结果(表格)
| 方法 | case | visible support | multi-view visible support | unsupported self-visible | mean visible support count | surface area |
|---|---|---|---|---|---|---|
| baseline | 10 | 0.9641 | 0.8745 | 0.0303 | 2.8502 | 1.7842 |
| repair_hybrid | 10 | 0.9646 | 0.8746 | 0.0324 | 2.8512 | 2.2947 |
逐 case repair gate
| case | A21 gate | A19选择 | baseline support | repair support | support Δ | baseline unsupported | repair unsupported | unsupported Δ | GT Chamfer Δ | GT F@5 Δ | 原因 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| gso_000_input4 | pass | ranker_selected | 0.7472 | 0.8430 | 0.0958 | 0.2331 | 0.1523 | -0.0808 | -0.00160 | 0.0202 | [] |
| gso_001_input4 | reject | ranker_selected | 0.9796 | 0.9779 | -0.0017 | 0.0029 | 0.0037 | 0.0007 | -0.00105 | 0.0082 | ['2D visible-safe gate rejects repair'] |
| gso_002_input4 | pass | ranker_selected | 0.9970 | 0.9976 | 0.0006 | 0.0030 | 0.0024 | -0.0006 | -0.00377 | 0.0502 | [] |
| gso_003_input4 | reject | ranker_selected | 0.9979 | 0.9976 | -0.0003 | 0.0020 | 0.0024 | 0.0004 | -0.00005 | -0.0041 | ['2D visible-safe gate rejects repair'] |
| gso_004_input4 | reject | ranker_selected | 0.9821 | 0.9838 | 0.0017 | 0.0179 | 0.0163 | -0.0017 | -0.00041 | 0.0032 | ['2D visible-safe gate rejects repair'] |
| gso_005_input4 | reject | ranker_selected | 0.9991 | 0.9978 | -0.0012 | 0.0000 | 0.0000 | 0.0000 | 0.00119 | -0.0079 | ['2D visible-safe gate rejects repair'] |
| gso_006_input4 | reject | repair_hybrid | 0.9778 | 0.8830 | -0.0948 | 0.0107 | 0.1138 | 0.1031 | 0.00144 | 0.0204 | ['repair reduces self-visible 3D input-supported surface', 'repair increases self-visible unsupported surface', '2D visible-safe gate rejects repair'] |
| gso_007_input4 | reject | ranker_selected | 0.9615 | 0.9653 | 0.0038 | 0.0328 | 0.0328 | 0.0000 | 0.00281 | -0.0188 | ['2D visible-safe gate rejects repair'] |
| gso_008_input4 | reject | baseline | 1.0000 | 0.9998 | -0.0002 | 0.0000 | 0.0000 | 0.0000 | 0.00474 | -0.0168 | ['2D visible-safe gate rejects repair'] |
| gso_009_input4 | pass | ranker_selected | 0.9992 | 0.9999 | 0.0007 | 0.0005 | 0.0001 | -0.0004 | -0.00006 | -0.0029 | [] |
可视化结果
实验结论
- A21 证明可以在无 GT 情况下构造一个 3D visible support field:表面点 -> 输入相机投影 -> 输入前景 mask -> 自遮挡可见性过滤。
- 10-case 上 repair_hybrid 相比 baseline:visible support Δ=0.0004,unsupported self-visible Δ=0.0021,support helped cases=5。
- gso_006 被 A21 拒绝:support Δ=-0.0948,unsupported Δ=0.1031;这解释了为什么它虽然 F@5 有收益,但形状保真风险很高。
- gso_008 的 3D silhouette-support 对 baseline/repair 几乎无区分,说明只看 silhouette 的 3D support 仍会漏掉厚度、轮廓边界和局部形状错误。
- 因此 A21 的价值不是直接提升 mesh,而是把下一步算法目标收窄:需要 support field + depth/normal reprojection + topology closure 的联合约束。
下一步想法
- A22:渲染候选 depth/normal 到输入相机,构造 depth-aware visible support;把 A21 中 silhouette 无法区分的 gso_008 变成可区分。
- 把 support field 迁移到 TRELLIS.2 的 sparse token 或 SLat 空间:visible-supported 区域锁定,unobserved 区域允许水密闭合。
- 设计 differentiable 或 sampler-time gate:不是生成后筛选 mesh,而是在 diffusion step 中约束支持场 drift。
- 扩展到 100-case 统计:比较 A20 2D loss、A21 3D support、未来 A22 depth/normal support 谁最能预测 Chamfer/F@5 变化。