I’m trying to generate a depth pass from the gaussian splatting render to do some depth of field and other effects and don’t see any of the splats in the depth top, if I add additional geometry or particles to the scene, they are visible in the depth TOP but I still can’t see anything in the depth top.
I am facing similar issues using the stable release and the TOPs version of the component provided by Tim.
I was able to generate a pseudo depth map by outputting the distance from the camera as an additional buffer from the material shader, but this didn’t give me the desired result i was hoping for. Any advice on how I can get an accurate depth pass from the gaussian splat render would be appreciated.
I’m afraid that’s not really possible with the current version of the gaussian splat due to the technique of rendering. It’s a small point cloud but instead of rendering particles, you render blurry splats (gaussian curves). These splats are alpha blended and can be quite huge spanning over your whole scene. 1 splat can be used for multiple pixels and so world space positions in your image depending of the direction you look at it. So there is not really depth data inside this other than the particle positions. The reason it’s not visible in your depthTOP is because of the ‘Write depth values’ is toggled off of the glslSplat material. If you switch it on you’ll see what I mean with how those splats look like.
I think I saw something about a new gaussian splat model that can be used for relighting, so perhaps that one could give depth information as well though I haven’t tried this yet.
Other than that, only solution I can think of is using something like DepthAnything after your rendering, but this is quite heavy having an AI model predict the depth. (Also not sure about the temporal stability)
Sorry for such bad news
Maybe I reacted too fast to your depth question Seems like @watershed just released a GS tool that is able to render depth. Haven’t checked myself yet, but the instagram post looked epic
some context for those curious: i used a separate image output on the render TOP. The vertex shader calculates the camera distance to splat, which is then passed to the fragment shader of the material to render out to the render image output
known limitations:
cant blend on the other image outputs AFAIK so there are some artifacts, especially when the splats are large relative to the view frustum
pretty expensive as requires another 32 bit float pass on any splats being rendered
AFAIK the alpha/experimental POPs builds do not support render image output yet
In theory you can implement a blended version using an override material (i think??) but I have not yet implemented this
