FIXED: ImportError: `load_weights` requires h5py when loading weights from HDF5

Bugs
1

I tried many solutions, and still get the same error.

Windows 10
h5py is well installed by pip : 3.1.0
python : 3.7.2
numpy : 1.20.0
tensorflow & tensorflow-gpu version : 2.5.1

FULL error :

Traceback (most recent call last):
File “/project1/script1_callbacks”, line 22
File “C:\Users\ACTLD-17\anaconda3\envs\GPUTD78\Lib\site-packages\pixellib\semantic_init_.py”, line 49, in load_pascalvoc_model
self.model.load_weights(model_path)
File “C:\Users\ACTLD-17\anaconda3\envs\GPUTD78\Lib\site-packages\tensorflow\python\keras\engine\training.py”, line 2336, in load_weights
‘load_weights requires h5py when loading weights from HDF5.’)
ImportError: load_weights requires h5py when loading weights from HDF5.

the code blocks at this line :
segment_image.load_pascalvoc_model(‘C:\Users\…\deeplabv3_xception_tf_dim_ordering_tf_kernels.h5’)

ps : the same code & TD project runs very well on my macbook pro, no issues with same packages & python version

2

Hey @marco.AI

What TouchDesigner version are you trying to load the library into ?

Is the Python AI/ML project you are trying to run available in Github or somewhere for me to give it a try? Or is it some code you wrote yourself, relying on Tensorflow / Keras as far as I can tell ?

Thanks,
Michel

3

hey @JetXS Thanks for interest.

=> TouchDesigner 64-Bit Build 2021.16410

yes this is a project wrote from scratch

to make a summary :

Touch Designer:
TOP: VideoDeviceIn1 — res1 — flip1 — null1 / top-script — Out

Into the top-script callbacks, i’ve this (The error is on the ‘load_pascalvoc_model’ line) :

import sys 
mypath = 'C:\\Users\\user1\\anaconda3\\envs\\GPUTD\\Lib\\site-packages' 
if mypath not in sys.path:
    sys.path.append(mypath)

import numpy
import cv2
import pixellib
from pixellib.semantic import semantic_segmentation 
from skimage.io import imread, imsave 

pathIn = 'C:\\Users\\user1\\Desktop\\TouchDesigner\\segment_shape_1\\img_rec\\imgIn.png'
pathOut = 'C:\\Users\\user1\\Desktop\\TouchDesigner\\segment_shape_1\\img_rec\\imgOut.png'

segment_image = semantic_segmentation()
segment_image.load_pascalvoc_model('C:\\Users\\user1\\Desktop\\TouchDesigner\\segment_shape_1\\Segmentation\\models\\deeplabv3_xception_tf_dim_ordering_tf_kernels.h5')


def onSetupParameters(scriptOp):
    return
def onPulse(par):
    return
def onCook(scriptOp):
    img = op('null1').numpyArray(delayed=False)
    imsave(pathIn, img)
	segment_image.segmentAsPascalvoc(pathIn, output_image_name = pathOut)
    imgOut = cv2.imread(pathOut)
    scriptOp.copyNumpyArray(imgOut)
    return

Just tested with differents python and tensorflow versions, i still have the same error.

On macOS it works well but i need to run it actually on windows 10 for an installation :weary: .

When i run the following code on Visual studio Code on the same conda env, there is no problem with h5py :

import pixellib
from pixellib.semantic import semantic_segmentation
import cv2
cam = cv2.VideoCapture(0)
cam.set(3,1280)
cam.set(4,720)

semgment_video = semantic_segmentation()
semgment_video.load_pascalvoc_model('models/deeplabv3_xception_tf_dim_ordering_tf_kernels.h5')

seg , result = semgment_video.process_camera_pascalvoc(cam ,overlay=True, frames_per_second=30 , show_frames=True , frame_name="frame" )

thank you, :pray:

Marco

4

@marco.AI

Trying to reproduce, I’ll get back to you.

Best,
Michel

5

@JetXS Thank you Michel :pray:

6

I’m having a similar issue using a training script with tensorflow. I tried installing h5py and including an ‘import h5py’ in the script but now I get:

“ImportError: DLL load failed while importing defs: The specified procedure could not be found.”

Also using the Anaconda virtual environment (python 3.9.7) with the latest TD (experimental build) in order to make sure the DLL load script (from the Anaconda+TD tutorial) works.

7

still not working for me, same error with h5py :weary:

@JetXS any try on your side ?

*edit:

I tried with pixellib alter_bg and it works with no h5py problem,

the problem is when i use ‘from pixellib.semantic import semantic_segmentation’ (and this is the one i need)

8

Hey @marco.AI @pdavila

Sorry for the delay.

@marco.AI Your original issue

ImportError: load_weights requires h5py when loading weights from HDF5.

Seem to be a version / conflict issue somewhere. It’s documented in google such as here.

However, not much luck getting it working either at the moment, I’ve been encountering the same issue “ImportError: DLL load failed while importing defs: The specified procedure could not be found.”.

@pdavila Note that your python version should preferably be 3.9.5

If this is an emergency, I’d suggest using those scripts outside of TouchDesigner for now and sending the data from / to a standalone Python process using OSC and / or Spout.

1 Like
9

hey Michel,

yes i did this for now, running the script outsite TD and get data by OSC and path.

I will looks deeper about the right packages versions with python 3.9.5 to run everything inside TD.

Thank you for your times and tips :pray:

I keep you posted here about the following.

10

@marco.AI

If you are working in Build 2021.16410 then the python version would be 3.7.2

Best,
Michel

11

Thanks! I’ll check the Python version but the same DLL issue occurs with 3.7.0/3.7.2…

12

@JetXS

Good morning Michel,

I just tried with the following config and still the same issue :

Python 3.7.2
TF/TF-gpu 2.5.1
H5py 3.1.0
Pixel lib 0.7.1
numpy 1.21.5

I’m actualy doing the detection job outside TD, i save each frame as png and open it into TD script by cv2.imread() it’s working but it create some glitch and latency

13

Hey @JetXS any progress with this bug?
I’m having a very similar error with a ML setup I’m working on.
Everything works fine as external application, but when inside of TD it gives this error:

Traceback (most recent call last):
  File "/project1/script4_callbacks", line 17
  File "C:\Users/dsant/AppData/Local/Programs/Python/Python39/Lib/site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "C:\Users/dsant/AppData/Local/Programs/Python/Python39/Lib/site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
    return fn(*args, **kwargs)
  File "C:\Users/dsant/AppData/Local/Programs/Python/Python39/Lib/site-packages\keras\saving\save.py", line 212, in load_model
    raise ImportError(
ImportError: Filepath looks like a hdf5 file but h5py is not available. filepath=C:/Users/dsant/Desktop/emotion_detection_model_100epochs.h5

It looks like TD can’t load the h5 file.

14

Hey @Deltacut

We are having another look at the h5py-related issues.

We will let you know if we have more details regarding what is going on and how to get things to work as expected.

Best,
Michel

15

Hi Michel @JetXS is there any updates about the h5py issue? I am having this issue with the newest TD. Thank you very much!

16

Quick update: I’ve been looking into the h5py issue and I think it’s a compatibility issue between the hdf5 libraries that are already included with TouchDesigner and the ones that the h5py module is looking for.

I’m not sure yet on a solution that will work for everything, but we’ll let you know what we find.

1 Like
17

In the next build we release I’ve removed our dependency on HDF5 and we no longer ship with it, so the one you are trying to use should be picked up now. Thanks for the report.

18

This was tested internally by importing h5py and loading the deeplabv3_xception_tf_dim_ordering_tf_kernels.h5 model, that the original poster “pixellib” is relying on, in a tensorflow-gpu environment.

Marking this thread as solved.

Best,
Michel