Mediapipe and background removal TD

Does anybody of you tested the Mediapipe background removal?
I tested the standard code in python (outside TD) and it works, even if with non perfect results.
When I adapt the code to TD (TOP Script) it works very bad, the segmentation is poor.
Here’s my code, suggestions? Where I am wrong?:

Anyway, for TD developers, it’s time to embed all the Mediapipe features in TD standard OPs!!!

import cv2
import mediapipe as mp
import numpy as np
mp_drawing = mp.solutions.drawing_utils
mp_selfie_segmentation = mp.solutions.selfie_segmentation
bg_image = None

def onCook(scriptOp):
global bg_image
with mp_selfie_segmentation.SelfieSegmentation( model_selection=1) as selfie_segmentation:
out_mask = True #enable to out the mask instead of masked image( better for filtering etc.)

	image = scriptOp.inputs[0].numpyArray(delayed=True, writable=True)
	image *= 255
	image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)		
	image = image.astype('uint8')
	image.flags.writeable = False
	results = selfie_segmentation.process(image)
	image.flags.writeable = True
	image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
	condition = np.stack((results.segmentation_mask,)*3 , axis=-1) > 0.1
	if out_mask:
		mask = condition*255
		mask = mask.astype('uint8')
		scriptOp.copyNumpyArray(mask)
	else:
		if bg_image is None:
			bg_image = np.zeros(image.shape, dtype=np.uint8)
			bg_image[:] = (0, 192, 0) #back color
		output_image = np.where(condition, image, bg_image)
		scriptOp.copyNumpyArray(output_image)
		scriptOp.copyNumpyArray(output_image)
return
1 Like

i suggest to propose your request about the mediapipe integration in the wishlist channel in forum
+1

@lucesintetica I’m not sure to understand what is the hype around MediaPipe? There is still no GPU support for Windows. You can add the python library following our Anaconda tutorial easily, but performances are poor, it all runs on the CPU and it all needs to be sent to an Engine COMP.

1 Like

my point of view was only in direction of performance,( and for sure an approach via nodes give more power to “non coder” area of users). Cheers :slight_smile: