Engineering an Infinite Voxel Engine: Data Structures, Hybrid Pipelines, and Multi-Scale Simulation

29 YouTube videos analyzed · 6h 0m of video

Engineering an Infinite Voxel Engine: Data Structures, Hybrid Pipelines, and Multi-Scale Simulation

Developing a real-time, destructible voxel engine capable of handling billions of voxels alongside interactive physics requires overcoming severe GPU memory bandwidth constraints, cache stalls, and algorithmic scaling bottlenecks. Over several years of iterative development, the architecture transitioned through three core rendering paradigms, two full programming language rewrites, and increasingly sophisticated custom acceleration structures.


🔑 Key Takeaways


Spatial Acceleration: SVOs, Contrees, and Empty Space Skipping

Storing voxels as uncompressed flat grids requires unmanageable memory ($256^3$ voxels require 16 MB at 1 byte per voxel; planetary scales exceed exabytes) Implementing sparse voxel octrees and the ray caster [Voxel Devlog #1] @ 04:04, How I tripled the render distance in my game engine [Voxel Devlog #16] @ 01:03. The foundation of efficient storage and empty-space traversal lies in hierarchical tree structures.

Hierarchical Contree Structure (4x4x4 branching factor):
Root Volume
└── 64 Child Nodes (Encoded via 64-bit Bitmask)
    ├── Empty Node (Bit = 0: Skipped in 1 cycle)
    └── Populated Node (Bit = 1: Traverse or step with DDA)
  1. Sparse Voxel Octrees (SVO): Initial implementations compressed uniform cubic regions into single nodes, reducing memory footprints from 16 MB down to under 0.5 MB Implementing sparse voxel octrees and the ray caster [Voxel Devlog #1] @ 04:04. However, traditional octrees introduce deep pointer graphs. To resolve branch material offsets without dynamic lookup stalls, aligning full-octant material data and position offsets introduced a 30% to 50% data size penalty but doubled traversal throughput by eliminating buffer reads and branching loops Textures, lighting, and MUCH faster rendering [Voxel Devlog #2] @ 05:01.
  2. 64-Trees (Contrees) and Bitmask Caching: Deep traversal hierarchies thrash GPU cache lines. By widening the branching factor from 8 (octree) to 64 (subdividing each axis by 4 into $4^3$ voxel bricks), tree depth drops significantly Adding ray tracing (back) to my game engine [Voxel Devlog #17] @ 07:20. Storing each brick as a 64-bit mask alongside child payload pointers allows compute shaders to perform bitwise AND masks against precomputed ray paths, allowing rays to advance up to 10 steps within GPU registers without touching external memory Adding ray tracing (back) to my game engine [Voxel Devlog #17] @ 08:21, Doubling the speed of my game's graphics [Voxel Devlog #18] @ 07:11.
  3. Inner-Brick DDA: While inter-brick traversals across tree levels rely on bounding-box boundary calculations, movement inside dense $4^3$ leaves uses digital differential analyzers (DDA). Storing initial plane distance offsets and stepping only the nearest axis component using additions reduces register strain and cycle counts compared to 3-component vector division Doubling the speed of my game's graphics [Voxel Devlog #18] @ 03:05.

Evolution of the Rendering Pipeline

Achieving real-time frame rates across diverse hardware forced multiple architectural shifts across three core rendering paradigms:

Technique Primary Strengths Primary Bottlenecks / Weaknesses Optimal Hardware Target
Compute Shader Ray Marching Implementing sparse voxel octrees and the ray caster [Voxel Devlog #1] @ 05:05 Natural dynamic lighting, shadows, and reflections; direct voxel lookups. GPU memory bandwidth saturation on low-end cards; register pressure Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 01:00. Discrete high-bandwidth GPUs (e.g., GTX 1660 Ti, RTX series).
Parallax Ray Marching Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 05:04 Leverages GPU raster hardware; bypasses quad generation per face; low triangle count. Requires manual fragment depth writes, disabling hardware Early-Z optimizations Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 08:08. Integrated GPUs (e.g., Intel UHD) and WebAssembly.
Greedy Meshed Rasterization Codebase OVERHAUL, new EVENT SYSTEM, LODs, and MORE [Voxel Devlog #9] @ 07:10 Fully hardware-accelerated pipeline; optimal use of early depth rejection. Triangles explode on non-coplanar rough surfaces (slopes, natural terrain) Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 04:04. Standard desktop polygon pipelines.
Hybrid Path Tracing / DDGI Adding global illumination to my game engine w/ DDGI [Voxel Devlog #23] @ 01:01 Physically plausible multi-bounce global illumination without stochastic noise. Substantial bookkeeping for probe memory allocation and sampling interpolation Adding global illumination to my game engine w/ DDGI [Voxel Devlog #23] @ 04:06. Modern mid-to-high tier GPUs.

To make parallax ray marching viable on integrated GPUs, early depth testing must be reclaimed by rendering bounding box geometry and deferring precise depth generation to a separate custom depth pass Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 09:09. Furthermore, CPU-side culling pipelines reduce pipeline driver overhead via glMultiDrawArrays combined with three spatial tests: expanded frustum culling, camera-relative octant culling (which prunes 50% of box faces located entirely to one side of the viewer), and directional face culling The PERFECT voxel rendering pipeline (and online demo) [Voxel Devlog #7] @ 08:15.

When full-fidelity ray tracing was reintroduced, primary rays were accelerated via a conservative beam optimization: a coarse-resolution pass establishes minimal ray distances across a neighborhood of pixels, allowing full-resolution rays to safely skip empty foreground air without missing thin features Doubling the speed of my game's graphics [Voxel Devlog #18] @ 09:13.


Illumination, Denoising, and Volumetric Light Transport

Direct ray tracing supports instantaneous hard shadow rays Textures, lighting, and MUCH faster rendering [Voxel Devlog #2] @ 02:00, but simulating soft ambient occlusion and multi-bounce indirect lighting requires specialized caching models:


Physics, Collision Resolution, and Destruction

Simulating arbitrary destructible voxel rigid bodies introduces severe stability challenges that off-the-shelf mesh engines do not address efficiently.

Physics Iterative Simulation Loop (TGS):
1. Detect Contacts (Narrow-phase: Corners & Edges with rounded shells)
2. Predict Velocities (External Forces + Warm Starting from Contact Map)
3. Solve Velocity Constraints Iteratively
4. Integrate Displacements (Zero drift for resting bodies)
5. Generate Fracture Events (Evaluate Impulse > Threshold)

Collision Detection Geometry

Treating voxels as pure cubes generates massive, redundant contact manifolds; an object resting on a flat floor creates contact points across every face voxel, over-constraining the solver Coding rigid body physics for voxels [Voxel Devlog #20] @ 03:46. Conversely, approximating voxels as pure spheres causes sinking artifacts and artificial surface friction My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 08:51.

The optimal configuration isolates collision checks to corner (empty along 2+ axes) and edge (empty along 1 axis) voxels using rounded geometrical offsets, running sphere-sphere, plane-sphere, and cylinder-cylinder queries against adjacent volumes Coding rigid body physics for voxels [Voxel Devlog #20] @ 04:47, My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 09:54.

Solver Dynamics and Warm Starting

Standard split-impulse solvers update positions prior to constraint solving, generating numerical jitter for resting bodies. Adopting Temporal Gauss-Seidel (TGS) executes an inner loop where velocity constraints are iteratively solved before updating spatial transforms My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 11:57. Persistent contact manifolds track reaction forces between frames using the coordinate pairs of the two colliding voxels as hash keys, enabling warm starting that stabilizes stacks of up to six continuous bodies My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 02:03, 12:58.

Structural Disconnection and Dynamic Fracture

When voxels are destroyed, structural integrity is evaluated via connected component labeling. Rather than running flood fills over millions of raw voxels, a hierarchical Depth-First Search (DFS) traverses the contree, treating uniform occupied octants as unified meta-nodes and caching visited spaces inside a bitmask Chopping trees DOWN [Voxel Devlog #12] @ 06:04, 07:06.

If a body suffers an impact where the contact impulse (independent of simulation tick length $\Delta t$) exceeds material-specific rupture thresholds, a fracture event triggers How I made voxels SHATTER on impact [Voxel Devlog #28] @ 02:02. Precomputed directional boolean voxel volumes carve empty cracks into the object, allowing the existing disconnector to naturally split the volume into distinct rigid bodies without custom geometry fragmenting algorithms How I made voxels SHATTER on impact [Voxel Devlog #28] @ 05:09, 07:12.


World Streaming and Algorithmic Acceleration

Generating and modifying massive worlds in real time requires avoiding brute-force sampling over volumetric data:

  1. Interval Arithmetic Terrain Generation: Terrain algorithms compiled to low-level bytecodes execute two compute shader stages: an interval arithmetic stage and a fine-sampling stage GORGEOUS, speedy terrain generation [Voxel Devlog #14] @ 03:05, 07:24. By tracking minimum and maximum range outputs across monotonic functions (e.g., $f(x) = 2x$), the pipeline determines whether an entire $16^3$ voxel chunk is guaranteed to evaluate purely to air ($>0$) or purely to stone ($\le 0$), skipping fine evaluation for 50% to 80% of all generated volumes GORGEOUS, speedy terrain generation [Voxel Devlog #14] @ 07:24, 08:26.
  2. Server-Generated LOD Caching with Dirty Sub-Octants: Generating full-resolution chunks on the server to downsample them for distant clients wastes memory and CPU time. Instead, the server uses scale factors directly within procedural noise invocations to evaluate LOD chunks natively How I tripled the render distance in my game engine [Voxel Devlog #16] @ 04:06. When a player edits terrain, sub-octants in the spatial LOD octree are flagged as dirty; higher-level cached LODs are loaded from disk, merged with selectively downsampled dirty sub-octants, and saved back to storage How I tripled the render distance in my game engine [Voxel Devlog #16] @ 07:49.
  3. Implicit Voxel Normals and Memory Alignment: Explicitly packing per-voxel normals creates massive data authoring friction during runtime CSG operations (such as sphere filling or copying between trees) Grass, textures, and a new codebase [Voxel Devlog #22] @ 04:05. Instead, calculating surface normals implicitly at upload time allows voxels to remain minimal material indices, enabling procedural triplanar mapping and GPU-driven foliage decoration buffers to be generated cleanly on the fly Grass, textures, and a new codebase [Voxel Devlog #22] @ 08:34, 12:40, 13:42.

Software Architecture & Language Transitions

Building an extensible, sandboxed engine across web and desktop platforms involved several architectural rewrites:

Runtime Engine Stack:
┌──────────────────────────────────────────────────────────┐
│ C# High-Level Front-End (Entities, User Logic, egui.NET) │
└────────────────────────────┬─────────────────────────────┘
                             │ Interop (C FFI / Serde)
┌────────────────────────────┴─────────────────────────────┐
│ Rust Core Engine Backend (SIMD Contrees, Physics, TGS)   │
└────────────────────────────┬─────────────────────────────┘
                             │ Compute / Graphics API
┌────────────────────────────┴─────────────────────────────┐
│ WebGPU / Vulkan Shaders (DDA Traversal, DDGI Probes)     │
└──────────────────────────────────────────────────────────┘
  1. Language Ecosystem Progression: Early iterations relied on C# with Vulkan Implementing sparse voxel octrees and the ray caster [Voxel Devlog #1] @ 02:00, but memory management limits and CLR safety overhead showed lower algorithmic execution speeds than native equivalents Sparse voxel octree modification and benchmarking [Voxel Devlog #3] @ 05:07. The engine pivoted to Rust for native WebAssembly compilation Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 02:02. However, building a plugin API using pure WebAssembly runtimes proved awkward due to sandboxed memory address isolation and complex interface plumbing Grass, textures, and a new codebase [Voxel Devlog #22] @ 02:02.
  2. The Hybrid Architecture: The final architecture splits cleanly: a performance-critical Rust backend handles tree storage, SIMD transformations, and GPU command dispatch, while exposing a safe C# scripting layer via C-FFI for gameplay, entities, and tooling Grass, textures, and a new codebase [Voxel Devlog #22] @ 06:09.
  3. Automated Interop Generation: To integrate modern UI without exposing unsafe pointers or relying on fragile manual bindings, an automated tool ingests rustdoc JSON definitions and serializes types using serde_generate, automatically compiling C# wrapper methods for 80% of the immediate-mode egui library Adding UI to my hybrid C# game engine was surprisingly tricky [Voxel Devlog #24] @ 11:14, 12:16.

⚡ Action Items

  1. Replace Deep Octrees with Wide Contrees: Re-architect spatial indices to use $4^3$ branching layouts backed by a 64-bit occupancy mask. Evaluate ray occupancy using bitwise instructions (AND, shifts) to allow rays to traverse up to 10 empty regions per step within registers Adding ray tracing (back) to my game engine [Voxel Devlog #17] @ 08:21.
  2. Implement Interval Arithmetic for Procedural Generation: Wrap world-generation math routines inside range bounds $([\min, \max])$. Query bounding volumes prior to voxel generation and write uniform solid or air tags to skip evaluating 50%+ of all blocks GORGEOUS, speedy terrain generation [Voxel Devlog #14] @ 07:24.
  3. Switch to Temporal Gauss-Seidel for Voxel Physics: Move velocity constraint resolution before position updates in the rigid body loop. Cache contact reaction forces between ticks keyed on voxel coordinate pairs to enable warm starting and eliminate resting jitter My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 11:57, 12:58.
  4. Prune Voxel Collision Tests to Corner and Edge Offsets: Classify voxel geometry locally as corners, edges, faces, or interiors. Restrict narrow-phase contact testing strictly to corner and edge primitives modeled with rounded radii to prevent over-constraining the solver while maintaining rotation invariance Coding rigid body physics for voxels [Voxel Devlog #20] @ 04:47, My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 09:54.

Source Overview

Video Channel Duration Quality Only here
Implementing sparse voxel octrees and the ray caster [Voxel Devlog #1] Douglas 8:41 Must Watch Sparse voxel octrees compress 256³ grids from 16 MB to under 0.5 MB by storing homogeneous regions as single nodes.
Textures, lighting, and MUCH faster rendering [Voxel Devlog #2] Douglas 7:39 Worth It Aligning position offsets and material data per octree level instead of keeping them separate eliminates buffer reads and doubles traversal speed despite 30–50% data size penalty.
Sparse voxel octree modification and benchmarking [Voxel Devlog #3] Douglas 7:36 Worth It —
Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] Douglas 12:10 Must Watch Parallax ray marching technique draws 8³ voxel bounding boxes via rasterization with bounded ray casts delegated to fragment shaders, reducing triangle counts by an order of magnitude.
Adding MULTIPLAYER NETWORKING with WebRTC to my game engine [Voxel Devlog #5] Douglas 11:41 Worth It —
Designing a FLEXIBLE game engine with Rust [Voxel Devlog #6] Douglas 12:31 Worth It —
The PERFECT voxel rendering pipeline (and online demo) [Voxel Devlog #7] Douglas 14:40 Worth It —
GPU-generated DISTANCE FIELDS [Voxel Devlog #8] Douglas 12:53 Skip —
Codebase OVERHAUL, new EVENT SYSTEM, LODs, and MORE [Voxel Devlog #9] Douglas 8:49 Worth It —
Adding SNOW and MODEL IMPORTS [Voxel Devlog #10] Douglas 13:32 Skip —
Making voxels MOVE with the separating axis test [Voxel Devlog #11] Douglas 14:08 Worth It —
Chopping trees DOWN [Voxel Devlog #12] Douglas 16:49 Worth It —
OPTIMIZING my physics engine [Voxel Devlog #13] Douglas 9:30 Worth It —
GORGEOUS, speedy terrain generation [Voxel Devlog #14] Douglas 12:20 Must Watch Interval arithmetic on compute shaders evaluates whether entire voxel regions output uniform air or solid material, skipping fine-grained sampling for 50–80% of terrain chunks.
Adding ambient occlusion to my game engine [Voxel Devlog #15] Douglas 13:35 Skip —
How I tripled the render distance in my game engine [Voxel Devlog #16] Douglas 13:50 Worth It —
Adding ray tracing (back) to my game engine [Voxel Devlog #17] Douglas 12:29 Must Watch 64-tree (Tetra-Hexacontatree / contree) with 64-bit occupancy masks allows rays to traverse up to 10 empty steps per lookup by evaluating spatial occupancy entirely within GPU registers via bitwise operations.
Doubling the speed of my game's graphics [Voxel Devlog #18] Douglas 13:01 Must Watch Beam optimization conservative coarse-resolution pass establishes minimum ray skip distances, enabling full-resolution rays to safely traverse empty space without missing thin geometry.
Emissive voxels and fancy lighting [Voxel Devlog #19] Douglas 13:08 Worth It —
Coding rigid body physics for voxels [Voxel Devlog #20] Douglas 13:15 Worth It —
Creating a modding system with Rust and WebAssembly [Voxel Devlog #21] Douglas 10:35 Skip —
Grass, textures, and a new codebase [Voxel Devlog #22] Douglas 18:32 Worth It —
Adding global illumination to my game engine w/ DDGI [Voxel Devlog #23] Douglas 13:39 Must Watch Dynamic Diffuse Global Illumination (DDGI) encodes per-probe incoming radiance into octahedral mappings with per-texel depth maps to prevent light leaking through walls during trilinear probe blending.
Adding UI to my hybrid C# game engine was surprisingly tricky [Voxel Devlog #24] Douglas 14:34 Skip —
Prototyping physics and procedural generation [Voxel Devlog #25] Douglas 9:26 Skip —
My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] Douglas 15:37 Must Watch Temporal Gauss-Seidel (TGS) solver integrates position only after velocity constraints are solved, eliminating resting-body jitter; warm starting reuses contact reaction forces keyed on voxel coordinate pairs between frames.
How I made voxels SHATTER on impact [Voxel Devlog #28] Douglas 9:34 Skip —
Coding a character controller [Voxel Devlog #29] Douglas 12:01 Skip —
Adding joints to my physics engine [Voxel Devlog #30] Douglas 14:44 Skip —