I’m not entirely sure how you’re combining the player index and point cloud images, but yeah any traditional 2d blur on the point cloud will likely pull points out of the background into the player.
You may need to add additional filters to ignore points that are outside of the range where you expect your players i.e. if your players are around 2 meters from the camera, you could maybe exclude all values greater than 3 or 3.5.
You could also consider some 3D edge detection and look for large jumps in the distance values of your point cloud as the edge around the person and exclude those from the blur.