How does the Attribute Create SOP calculate tangents?

I’ve been exploring some other pbr effects lately, one specific one being anisotropy. Through trying to debug some visual inconsistencies from other software, like substance painter for example, I notice that I get some weird faceting on faces when anisotropy is present.

Anyways, lots of googling keeps bringing me to an industry standard for tangent calculation called Mikktspace. http://www.mikktspace.com/

I haven’t quite had success yet implementing it, c++ is still not a strong suite for me, but it got me wondering, how DOES Touch calculate tangents?

I do know that the TBN matrix is calculated per vertex, via the function TDCreateTBNMatrix() but that’s still taking in CPU calculated tangents, and calculating the bitangent in vtx land as well.

I wanted to find out how TD does things behind the scenes in this regard before dumping too much time into trying to implement it.

Thanks!

To expand a bit, here’s my anisotropic material in TD:

here’s the mesh:
tangentDebug.zip (367 Bytes)
image

In this mesh there are 10 vertices, and 6 points. so some overlapping vertices.

This looks totally normal to me, but when I visualize the TBN vectors , that’s where I start to get confused as to whether or not this is correct or a bug somewhere. the overlapping vertices are more noticeable now, as the TBN’s for the overlapping vertices do not align perfectly:

To be clear, there’s no faceting visible ever when using the standard, non anisotropic pbr shading model, just this one… so it’s possible there’s a bug in my shader implentation too.

Also interesting test, I imported this same geometry into substance painter, no issues:

So, trying to understand ultimately if my shader code has a bug or if they process the mesh differently upon import.

We use a pretty standard method, you can see it here:

Not sure offhand if the difference between the vertices is expected, offhand I’d say it’s not.

1 Like

Sorry @malcolm just now getting a chance to come back to this - So I did some more research, and came across assimp (open asset import library) and also came across a subsequent python library/bindings for it called impasse.

With some work, I was able to get the dll compiled and the python library working in TD.
The script SOP loads the obj with the below code:

import impasse
import numpy


def onCook(scriptOp):
	scriptOp.clear()

	scriptOp.vertexAttribs.create('N')
	scriptOp.vertexAttribs.create('T')
	scriptOp.vertexAttribs.create('B',(0.0,0.0,0.0))
	scriptOp.vertexAttribs.create('uv')
	scriptOp.vertexAttribs.create('Cd')

	file_path = 'C:/ASSIMP/bin/Release/mesh.obj'
	scene = impasse.load(file_path)

	# https://github.com/SaladDais/Impasse/blob/master/impasse/constants.py
	import_actions = (
		0
		| impasse.constants.ProcessingStep.CalcTangentSpace
		| impasse.constants.ProcessingStep.JoinIdenticalVertices
		# | impasse.constants.ProcessingStep.MakeLeftHanded
		| impasse.constants.ProcessingStep.Triangulate
		# | impasse.constants.ProcessingStep.RemoveComponent
		# | impasse.constants.ProcessingStep.GenNormals
		# | impasse.constants.ProcessingStep.GenSmoothNormals
		# | impasse.constants.ProcessingStep.SplitLargeMeshes
		# | impasse.constants.ProcessingStep.PreTransformVertices
		# | impasse.constants.ProcessingStep.LimitBoneWeights
		# | impasse.constants.ProcessingStep.ValidateDataStructure
		| impasse.constants.ProcessingStep.ImproveCacheLocality
		# | impasse.constants.ProcessingStep.RemoveRedundantMaterials
		# | impasse.constants.ProcessingStep.FixInfacingNormals
		# | impasse.constants.ProcessingStep.SortByPType
		# | impasse.constants.ProcessingStep.FindDegenerates
		# | impasse.constants.ProcessingStep.FindInvalidData
		| impasse.constants.ProcessingStep.GenUVCoords
		# | impasse.constants.ProcessingStep.TransformUVCoords
		# | impasse.constants.ProcessingStep.FindInstances
		| impasse.constants.ProcessingStep.OptimizeMeshes
		| impasse.constants.ProcessingStep.OptimizeGraph
		# | impasse.constants.ProcessingStep.FlipUVs
		# | impasse.constants.ProcessingStep.FlipWindingOrder
		# | impasse.constants.ProcessingStep.SplitByBoneCount
		# | impasse.constants.ProcessingStep.Debone
		# | impasse.constants.ProcessingStep.GenEntityMeshes
		# | impasse.constants.ProcessingStep.OptimizeAnimations
		# | impasse.constants.ProcessingStep.FixTexturePaths
		# | impasse.constants.ProcessingStep.EmbedTextures
	)
	scene = impasse.load(file_path, processing=import_actions )


	### get some data from mesh while converting numpy lists to regular lists.
	name = scene.meshes[0].name
	primitive_types = scene.meshes[0].primitive_types
	num_uv_components = scene.meshes[0].num_uv_components

	anim_meshes = [ list(item) for item in scene.meshes[0].anim_meshes ]
	faces = [ list(item) for item in scene.meshes[0].faces ]
	vertices = [ list(item) for item in scene.meshes[0].vertices ]
	texture_coords = [ list(item) for item in scene.meshes[0].texture_coords[0] ]
	normals = [ list(item) for item in scene.meshes[0].normals ]
	tangents = [ list(item) for item in scene.meshes[0].tangents ]
	bitangents = [ list(item) for item in scene.meshes[0].bitangents ]
	colors = [ list(item) for item in scene.meshes[0].colors ]

	bones = [ list(item) for item in scene.meshes[0].bones ]

	
	for i,vtx in enumerate(vertices):
		p = scriptOp.appendPoint()
		p.x , p.y , p.z = vtx
		
	for face in faces:
		f = scriptOp.appendPoly(len(face), closed=True, addPoints=False)
		for vtx in f:
			i = face[vtx.index]
			vtx.point = scriptOp.points[i]
			if len(normals):
				vtx.N = normals[i]
			if len(tangents):
				vtx.T = tangents[i]+[1] ### NOTE this is technically wrong, we should not be setting this to 1 hardcoded, if winding order is reversed, it'd be -1..
				vtx.B = bitangents[i] ### calculating this in shader.. but we could eventually pass it in via sop if it's faster..
			if len(texture_coords):
				vtx.uv = texture_coords[i]
			if len(colors):
				vtx.Cd = colors[i]



	return

The result of this is a 3d mesh with tangent/bitangent vectors averaged for vertices that share the same point.

This produces smoothly interpolated tangents and bitangents similar to how normals are smoothly interpolated when vertices share the same point.

normals_compare
tangents_compare
bitangents_compare

You can see for the tangents and bitangents, you get these noticeable seams across faces when the vectors are not aligned/averaged for shared points, which is the root cause of the artifacts I posted about earlier.

I know this is totally nit picking for the current version of TD’s pbr shading model :smiley: as the artifact does not manifest it’s self in any meaningful way, but it will certainly be a problem for anisotropic material models of the future, and also it just feels like something that should behave similar to normals, in how they are interpreted for smooth vs faceted surfaces.

Also, I do believe Substance Painter/Designer etc use this in their backend for importing 3d models, so since we are also trying to have parity with substance sbars, and all that, it might make sense to interpret/import/export the meshes the same way they do as well, to ensure sbar’s and materials look as closely matched between ecosystems. I haven’t seen anywhere substance say they use it, I’ve only seen it mentioned sparsely in logs and errors occasionally, fairly sure it’s in use though.

Happy to package up this assimp module/python stuffs for further investigation if needed.
Also, has Derivative considered using assimp? They support a huge family of 3d file formats, and have some other cool features as well (see flags in above code)

COMMON INTERCHANGE FORMATS
Autodesk ( .fbx )
Collada ( .dae )
glTF ( .gltf, .glb )
Blender 3D ( .blend )
3ds Max 3DS ( .3ds )
3ds Max ASE ( .ase )
Wavefront Object ( .obj )
Industry Foundation Classes (IFC/Step) ( .ifc )
XGL ( .xgl,.zgl )
Stanford Polygon Library ( .ply )
*AutoCAD DXF ( .dxf )
LightWave ( .lwo )
LightWave Scene ( .lws )
Modo ( .lxo )
Stereolithography ( .stl )
DirectX X ( .x )
AC3D ( .ac )
Milkshape 3D ( .ms3d )
* TrueSpace ( .cob,.scn )
MOTION CAPTURE FORMATS
Biovision BVH ( .bvh )
* CharacterStudio Motion ( .csm )
GRAPHICS ENGINE FORMATS
Ogre XML ( .xml )
Irrlicht Mesh ( .irrmesh )
* Irrlicht Scene ( .irr )
GAME FILE FORMATS
Quake I ( .mdl )
Quake II ( .md2 )
Quake III Mesh ( .md3 )
Quake III Map/BSP ( .pk3 )
* Return to Castle Wolfenstein ( .mdc )
Doom 3 ( .md5* )
*Valve Model ( .smd,.vta )
*Open Game Engine Exchange ( .ogex )
*Unreal ( .3d )
OTHER FILE FORMATS
BlitzBasic 3D ( .b3d )
Quick3D ( .q3d,.q3s )
Neutral File Format ( .nff )
Sense8 WorldToolKit ( .nff )
Object File Format ( .off )
PovRAY Raw ( .raw )
Terragen Terrain ( .ter )
3D GameStudio (3DGS) ( .mdl )
3D GameStudio (3DGS) Terrain ( .hmp )
Izware Nendo ( .ndo )

Oh, and here’s a great dive into the flags/features of the assimp importer


I specifically wanted to point out this part, which sheds some light on how the assimp importer handles tangent smoothing:

class ProcessingStep(enum.IntFlag):
    # <hr>Calculates the tangents and bitangents for the imported meshes.
    #
    # Does nothing if a mesh does not have normals. You might want this post
    # processing step to be executed if you plan to use tangent space calculations
    # such as normal mapping  applied to the meshes. There's a config setting,
    # <tt>#AI_CONFIG_PP_CT_MAX_SMOOTHING_ANGLE<tt>, which allows you to specify
    # a maximum smoothing angle for the algorithm. However, usually you'll
    # want to leave it at the default value.
    #
    CalcTangentSpace = 0x1

    # <hr>Identifies and joins identical vertex data sets within all
    #  imported meshes.
    #
    # After this step is run, each mesh contains unique vertices,
    # so a vertex may be used by multiple faces. You usually want
    # to use this post processing step. If your application deals with
    # indexed geometry, this step is compulsory or you'll just waste rendering
    # time. <b>If this flag is not specified<b>, no vertices are referenced by
    # more than one face and <b>no index buffer is required<b> for rendering.
    #
    JoinIdenticalVertices = 0x2

the variable called “AI_CONFIG_PP_CT_MAX_SMOOTHING_ANGLE” seems relevant. maybe edges who’s angle is greater than that threshold, do not have their tangent vectors averaged, which would produce a hopefully desired hard edge there, similar to how normals are sometimes smoothed by angle threshold in 3d softwares / facet SOP.

Hey Lucas, thanks for the detailed report!
So an easy fix for this particular mesh would be to promote both normals and uvs to point attributes, this way TD will create a point tangent attribute (instead of a vertex one) which is smooth across shared points.
Of course that will give you glitchy uvs along seams if you have discontinuous uvs (like on a closed shape).

Otherwise indeed when the tangent attribute is calculated per vertex there is no averaging happening between neighboring faces.

We will look into implementing the mikkT standard in the future, which seems widely adopted. By the way funny that Assimp doesn’t support it for now, though it seems on their list

good find! didn’t catch that conversation. Seems like they are simply implementing their own tangent/bitangent smoothing algo using that angle variable for now.

I actually implemented something similar to what you’re talking about where I simply average the tangents for any vertices that reference the same point via script SOP, but this is dreadfully slow as you can imagine, and there are some edge cases where it doesn’t quite work well, especially where uv islands are split.

Substance Painter uses assimp for importing I believe… but I have not been able to find any public documentation on the flags or configuration they use for importing an obj mesh, thus there could still be discrepancies.


Anywayws, mikkT would be a good move! glad to hear it may find it’s way onto the roadmap :slight_smile:

pointTangents.toe (3.9 KB)

Oh I just meant promote normals and uvs with the point SOP, which will be faster than script SOP, see the toe.

Otherwise this pretty detailed blog post mentions substance calculates tangents according to mikktspace if they’re not on the model, but haven’t tested myself.

Substance Painter will default to using whatever tangent data the imported mesh uses. Unless you’re exporting from Blender, you do not want this! But there are no options to disable this feature. Make sure you’re exporting your meshes with out tangents before importing into Substance Painter. Substance’s documentation previously erroneously listed Blender as matching Unity’s bitangent calculations, but that’s been fixed!

wow what a deep dive write up on, thanks for sharing going to comb through that.

Also I had no idea the point SOP could promote attributes from uv attrs… Super cool, and very valuable for models that come from the internet with vertex attrs already.

1 Like

I don’t think it made it to the wiki but there’s a parameter to use mikkTSpace for tangents calculation on the attribute create SOP in 2022.25370, this should fix your initial issue with discontinuous tangents.

1 Like

Amazing!! Will definitely be plugging that back into the mix. Thank you!

1 Like