Converts a mesh of vertex colors to a UV-mapped textured mesh.
introduction
Vertex colors are an easy way to add color information directly to the vertices of a mesh. This is often how generative 3D models like InstantMesh generate meshes. However, for most applications, a UV-mapped textured mesh is preferred.
This tutorial provides a simple solution to convert a vertex color mesh to a UV mapped textured mesh. This includes a short version for quick results and a long version for a detailed walkthrough.
short version
For easy conversion, install the InstantTexture library. This is a small library we created that implements the steps described in the “long version” below.
pip install git+https://github.com/dylanebert/InstantTexture
Usage
The code below converts a vertex-colored .obj mesh into a UV-mapped textured .glb mesh and saves it to output.glb.
from instant texture import Converter input mesh path = “https://raw.githubusercontent.com/dylanebert/InstantTexture/refs/heads/main/examples/chair.obj”
Converter = Converter() converter.convert(input_mesh_path)
Let’s visualize the output mesh.
import trimesh mesh = trimesh.load(“output.glb”) mesh.show()
that’s it!
Keep reading for a detailed walkthrough.
long version
Install the following dependencies:
numpy mesh for numerical operations trimesh for loading and saving data xatlas for generating UV maps Pillow for image processing opencv-python for image processing httpx for image processing pip install numpy trimesh xatlas opencv- pythonpillow httpx
Import dependencies.
import CV2
import numb as NP
import trimesh
import Zatras
from pill import images, image filters
Loads the input mesh with vertex colors. This must be an .obj file located in input_mesh_path.
For local files, use trimesh.load() instead of trimesh.load_remote().
mesh = trimesh.load_remote(input_mesh_path) mesh.show()
Access the vertex colors of the mesh.
If this fails, make sure the mesh is a valid .obj file with vertex colors.
vertex color = mesh.visual.vertex color
Generate a UV map using xatlas.
This is the most time-consuming part of the process.
vmapping, index, uvs = xatlas.parametrize(mesh.vertices, Mesh.faces)
Remaps vertices and vertex colors to UV maps.
vertex = mesh.vertex(vmapping) vertex_color = vertex_color(vmapping) mesh.vertex = vertex mesh.face = index
Define the desired texture size.
Builds a texture buffer upscaled by upscale_factor to create higher quality textures.
texture size = 1024
Upscale factor = 2
buffer size = texture size * upscale factor texture buffer = np.zeros((buffer size, buffer size, 4), dtype=np.uint8)
Fills the texture of a UV-mapped mesh using centroid interpolation.
Centroid interpolation: Computes the interpolated color of point p within the triangle defined by vertices v0, v1, v2 and corresponding colors c0, c1, c2. Point-in-Triangle test: Determine whether point p lies within the triangle defined by vertices v0, v1, and v2. Texture Filling Loop: Iterates over each face of the mesh. Gets the UV coordinates (uv0, uv1, uv2) and color (c0, c1, c2) of the current face. Convert UV coordinates to buffer coordinates. Determines the bounding box of a triangle on the texture buffer. For each pixel in the bounding box, use the triangle-inside-point test to check whether the pixel is inside the triangle. If inside, use centroid interpolation to calculate the interpolated color. Assigns a color to the corresponding pixel in the texture buffer.
surely barycentric_interpolate(v0, v1, v2, c0, c1, c2, p): v0v1 = v1 – v0 v0v2 = v2 – v0 v0p = p – v0 d00 = np.dot(v0v1, v0v1) d01 = np.dot(v0v1, v0v2) d11 = np.dot(v0v2, v0v2) d20 = np .dot(v0p, v0v1) d21 = np.dot(v0p, v0v2) denom = d00 * d11 – d01 * d01
if abs(denom) 1e-8:
return (c0 + c1 + c2) / 3
v = (d11 * d20 – d01 * d21) / denom w = (d00 * d21 – d01 * d20) / denom u = 1.0 – v – wu = np.clip(u, 0, 1) v = np.clip(v, 0, 1) w = np.clip(w, 0, 1) interpolate_color = u * c0 + v * c1 + w * c2
return np.clip(interpolation color, 0, 255)
surely is_point_in_triangle(p, v0, v1, v2):
surely sign(p1, p2, p3):
return (p1(0) – p3(0)) * (p2(1) – p3(1)) – (p2(0) – p3(0)) * (p1(1) – p3(1)) d1 = sign(p, v0, v1) d2 = sign(p, v1, v2) d3 = sign(p, v2, v0) has_neg = (d1 0) or (d2 0) or (d3 0) has_pos = (d1 > 0) or (d2 > 0) or (d3 > 0)
return do not have (I’m a negative person. and has_pos)
for face in Mesh.faces: uv0, uv1, uv2 = uvs(face) c0, c1, c2 = vertex_colors(face) uv0 = (uv0 * (buffer_size – 1)).astype(integer) uv1 = (uv1 * (buffer size – 1)).astype(integer) uv2 = (uv2 * (buffer size – 1)).astype(integer) min_x = maximum(integer(np.floor(minutes(uv0(0), uv1(0), uv2(0)))), 0) max_x = minutes(integer(np.ceil(maximum(uv0(0), uv1(0), uv2(0)))), buffer_size – 1) min_y = maximum(integer(np.floor(minutes(uv0(1), uv1(1), uv2(1)))), 0) max_y = minutes(integer(np.ceil(maximum(uv0(1), uv1(1), uv2(1)))), buffer_size – 1)
for y in range(min_y, max_y + 1):
for × in range(min_x, max_x + 1): p = np.array((x + 0.5,y + 0.5))
if is_point_in_triangle(p, uv0, uv1, uv2): color = barycentric_interpolate(uv0, uv1, uv2, c0, c1, c2, p) texture_buffer(y, x) = np.clip(color, 0, 255).astype( np.uint8 )
Let’s visualize what the texture looks like so far.
from IPython.display import display image_texture = Image.fromarray(texture_buffer) display(image_texture)
As you can see, the texture has a lot of holes.
To fix this, combine the following four techniques:
Inpainting: Fills holes using the average color of surrounding pixels. Median filter: Removes noise by replacing each pixel with the median color of surrounding pixels. Gaussian Blur: Smoothes the texture and removes any remaining noise. Downsample: Use LANCZOS resampling to resize up to texture_size. image_bgra = texture_buffer.copy() mask = (image_bgra(:, :, 3) == 0).astype(np.uint8) * 255
image_bgr = cv2.cvtColor(image_bgra, cv2.COLOR_BGRA2BGR) inPaint_bgr = cv2.inpaint( image_bgr, mask, inpaintRadius=3flags=cv2.INPAINT_TELEA ) inPaint_bgra = cv2.cvtColor(inPaint_bgr, cv2.COLOR_BGR2BGRA) texture_buffer = inPaint_bgra(::-1) image_texture = Image.fromarray(texture_buffer) image_texture = image_texture.filter(ImageFilter.MedianFilter(size=3)) ImageTexture = ImageTexture.filter(ImageFilter.GaussianBlur(radius=1)) image_texture = image_texture.resize((texture_size, texture_size), Image.LANCZOS) display(image_texture)
As you can see, the texture is now much smoother and has no holes.
This can be further improved with more advanced techniques and manual texture editing.
Finally, you can construct a new mesh using the generated UV coordinates and texture.
Material = trimesh.visual.material.PBRmaterial(baseColorFactor=(1.0, 1.0, 1.0, 1.0), baseColorTexture=image_texture, metallicFactor=0.0roughness coefficient =1.0) Visuals = trimesh.visual.TextureVisuals(uv=uvs,material=material) Mesh.visual = Visuals Mesh.show()
here we go! The mesh is UV mapped and textured.
To export when running locally, call mesh.export(“output.glb”).
Restrictions
As you can see, the mesh still has many small artifacts.
The quality of the UV maps and textures is also well below the standard for producible meshes.
However, if you’re looking for a quick solution to mapping from a vertex-colored mesh to a UV-mapped mesh, this approach might be useful.
conclusion
This tutorial showed you how to convert a vertex color mesh to a UV-mapped textured mesh.
If you have any questions or feedback, feel free to open an issue on GitHub or Space.
Thank you for reading!
