FIXED: Web Render TOP does not handle Unicode when loaded via DAT

I’ve got some HTML, that if I serve it over a port via the Web Server DAT and then load it into a Web Render TOP, works fine. I can see unicode such as emoji’s and special characters.

However, if I want to load that exact same content directly from the DAT that I serve it from (setting the Source to DAT and selecting the html content). Special characters and emoji’s don’t work.

Here you can see an example in the word Neukölln for instance:


My HTML source already contains the <meta charset="utf-8"> tag. Both of these screenshots look at the same HTML code, just one via the web server, and the other via a DAT.

<html>
<head>
<meta charset="utf-8">
<style>body {
	font-family: "arial";
	font-size: 30px;
	overflow-x: hidden;
	padding: 0;
	margin: 0;
}

body a {
	color: blue;
}

p {
	margin:0;
	padding:0;
}

#caption {
	background-color: white;
	padding: 20px;
	margin: 0;
}</style>
</head>
<body>
<div id="caption">
<p>
Hi everyone, <a>@studiobruell.de</a> and I are offering an exciting internship position at our space the NODE Institute in Berlin Neukölln. We're looking for motivated individuals interested in working on interactive installations using a variety of tools.

<a>https://bit.ly/3p35MAv</a>
link in bio

<a>#internship</a> <a>#newmedia</a> <a>#art</a> <a>#artinstallation</a> <a>#vvvv</a> <a>#touchdesigner</a> <a>#glsl</a> <a>#blender</a> <a>#creativecode</a>
</p>
</div>
</body>
</html>

Hello @mickeyvanolst and thanks for the report!

I just gave it a try and removing <meta charset="utf-8"> from your DAT solves the issue.

The ö of Neukölln appears after that.

I will notify a developer to double-check what could be going on and if it’s a DAT issue or CEF issue.

Thanks,
Michel

Hi Michel,

Sadly removing the meta tag stops emoji’s from working though. But good find!

Best,
Mickey

Correction: Hmm, I have to rephrase this, removing the metatag enabled special characters. But did not enable emoji’s. Try placing an emoji into the body and you’ll see it works with the Web Render TOP if you look at it via localhost.

Thanks for the details,

I added an issue to our bug database for a developer to look into this :slight_smile:

Thanks,
Michel

1 Like

@JetXS one other thing you might want to look at, is that the Web Render TOP does not render a texture if you’re not directly looking at it. Its easy to test, just create a Web Render TOP, turn off the viewer, make a copy, wait a bit to be sure and then turn on the viewer on the copy. Only at the moment you enable the viewer the Web Render TOP gets to work and renders the page. Even though it was already set to Active, and Cook Always is turned on.

This is quite limiting, for instance I’m trying to use the Web Render TOP to cache a bunch of HTML content into textures to use for instancing (I only need the initial page render, after that I set them to inactive to not take any memory). But because I need to “look” at each of the TOPs to have them cook/render it won’t work.


Ok this seems to be partly on me, using force cook via python does do something

@JetXS just curious, did this end up getting resolved in the latest build? I’ve got a project that’s waiting to be delivered until either this works or I have to figure out some kind of alternative.

Best,
Mickey

Hey @mickeyvanolst

Apologies for the delay. We looked into it and here are a few points/questions for you.

  • Need of removing the meta tag for Neukölln
    It appears like a CEF issue, it could possibly be solved after a CEF update, but we don’t have any ETA on that.

  • Emojis
    We’ll need an example here… How are you using emojis? By copy-pasting them in the DAT? If you use the proper HTML codes, they will render properly - with or without meta tag. HTML Emoji Reference


  • WebRender TOP not cooking
    So if you are referring to texture instancing here, it looks like it could be an issue related to this and not an issue with the WebRender TOP… We are looking into it. An example would help, however, to make sure we are talking about the same thing.

Hi @JetXS,

Thank you for looking into this!

Regarding the emoji’s (and any other text). They are coming directly from instagram/twitter via their API’s.

I’m then using a script DAT to place that content into HTML code which then gets rendered directly by a webbrowserTOP.

Here you can find a copy of the incoming HTML data:

<html>
<head>
<style>body {
	font-family: "arial";
	font-size: 30px;
	overflow-x: hidden;
	padding: 0;
	margin: 0;
}

body a {
	color: blue;
}

p {
	margin:0;
	padding:0;
}

#caption {
	background-color: white;
	padding: 20px;
	margin: 0;
	font-weight: normal;
}

.user {
	font-weight: bold;
	margin-bottom:20px;
}

.user p {
}

svg {
	transform: scale(2.0) translate(50%, 50%);
	margin: 20px 0 40px 0;
}</style>
<meta charset="utf-8">
</head>
<body>
<div id="caption">
<div class="user">
<p>mickeyvanolst</p>
<div><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-instagram"><rect x="2" y="2" width="20" height="20" rx="5" ry="5"></rect><path d="M16 11.37A4 4 0 1 1 12.63 8 4 4 0 0 1 16 11.37z"></path><line x1="17.5" y1="6.5" x2="17.51" y2="6.5"></line></svg></div>
</div>
<div class="user">
<p>Ghost town rave 👻🤷‍♂️🌵</p>
</div>
</div>
</body>
</html>

The issues with cooking for the WebRender TOP I managed to solve mostly by using force cook methods in python.

Hey @mickeyvanolst

Thanks for the extra details.

So it seems there is some ongoing issue with those emotes and one of our developers will look into it.

A workaround I found is that writing the content to file and loading the file within the WebRender TOP succeed in displaying the emotes. This means, for the time being, you could write to file and load the file within the webrender TOP. It should be fairly easy to automatize it all with python.

I know this is not the best solution but debugging CEF can get quite tricky. I hope this helps as a temporary fix.

Thanks,
Michel

@JetXS the current workaround we have is serving the page via a Server DAT and then loading it in the TOP as a localhost. This works too.

1 Like

This will be fixed in the next release. Thanks for the bug report.

Happy to hear, great news! :slight_smile: