Engineering an Infinite Voxel Engine: Data Structures, Hybrid Pipelines, and Multi-Scale Simulation
29 YouTube videos analyzed · 6h 0m of video
Engineering an Infinite Voxel Engine: Data Structures, Hybrid Pipelines, and Multi-Scale Simulation
Developing a real-time, destructible voxel engine capable of handling billions of voxels alongside interactive physics requires overcoming severe GPU memory bandwidth constraints, cache stalls, and algorithmic scaling bottlenecks. Over several years of iterative development, the architecture transitioned through three core rendering paradigms, two full programming language rewrites, and increasingly sophisticated custom acceleration structures.
🔑 Key Takeaways
- Memory layout dictates ray tracing performance over raw algorithmic complexity. Transitioning from pointer-chasing octrees to shallow, high-branching 64-trees (Tetra-Hexacontatrees / contrees) combined with 64-bit integer masks eliminates up to 10 sequential memory lookups per ray step by evaluating spatial occupancy entirely within GPU registers Adding ray tracing (back) to my game engine [Voxel Devlog #17] @ 08:21. (stated in 1 of 29 sources)
- Pure ray marching frequently bottlenecks on memory bandwidth across commodity and integrated GPUs. A hybrid parallax ray marching pipeline—which rasterizes tight $8^3$ bounding boxes and delegates bounded ray casts to fragment shaders—reduces triangle counts by an order of magnitude compared to greedy meshing while running at 60 FPS on low-end hardware Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 05:04. (stated in 1 of 29 sources)
- Voxel physics requires dedicated geometrical approximations rather than general mesh colliders. Accurate and stable rigid-body simulation demands reducing voxel collision primitives to rounded corners and edges, using temporal Gauss-Seidel (TGS) impulse solvers with warm starting, and resolving topological breaks through hierarchical graph traversals My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 09:54. (stated in 1 of 29 sources)
- Decoupled multi-rate architectures stabilize large dynamic worlds. High-performance terrain generation and structural fracturing depend on skipping uniform space through interval arithmetic on compute shaders and offloading structural graph re-evaluations to modular event loops GORGEOUS, speedy terrain generation [Voxel Devlog #14] @ 05:21. (stated in 1 of 29 sources)
Spatial Acceleration: SVOs, Contrees, and Empty Space Skipping
Storing voxels as uncompressed flat grids requires unmanageable memory ($256^3$ voxels require 16 MB at 1 byte per voxel; planetary scales exceed exabytes) Implementing sparse voxel octrees and the ray caster [Voxel Devlog #1] @ 04:04, How I tripled the render distance in my game engine [Voxel Devlog #16] @ 01:03. The foundation of efficient storage and empty-space traversal lies in hierarchical tree structures.
Hierarchical Contree Structure (4x4x4 branching factor):
Root Volume
└── 64 Child Nodes (Encoded via 64-bit Bitmask)
├── Empty Node (Bit = 0: Skipped in 1 cycle)
└── Populated Node (Bit = 1: Traverse or step with DDA)
- Sparse Voxel Octrees (SVO): Initial implementations compressed uniform cubic regions into single nodes, reducing memory footprints from 16 MB down to under 0.5 MB Implementing sparse voxel octrees and the ray caster [Voxel Devlog #1] @ 04:04. However, traditional octrees introduce deep pointer graphs. To resolve branch material offsets without dynamic lookup stalls, aligning full-octant material data and position offsets introduced a 30% to 50% data size penalty but doubled traversal throughput by eliminating buffer reads and branching loops Textures, lighting, and MUCH faster rendering [Voxel Devlog #2] @ 05:01.
- 64-Trees (Contrees) and Bitmask Caching: Deep traversal hierarchies thrash GPU cache lines. By widening the branching factor from 8 (octree) to 64 (subdividing each axis by 4 into $4^3$ voxel bricks), tree depth drops significantly Adding ray tracing (back) to my game engine [Voxel Devlog #17] @ 07:20. Storing each brick as a 64-bit mask alongside child payload pointers allows compute shaders to perform bitwise
ANDmasks against precomputed ray paths, allowing rays to advance up to 10 steps within GPU registers without touching external memory Adding ray tracing (back) to my game engine [Voxel Devlog #17] @ 08:21, Doubling the speed of my game's graphics [Voxel Devlog #18] @ 07:11. - Inner-Brick DDA: While inter-brick traversals across tree levels rely on bounding-box boundary calculations, movement inside dense $4^3$ leaves uses digital differential analyzers (DDA). Storing initial plane distance offsets and stepping only the nearest axis component using additions reduces register strain and cycle counts compared to 3-component vector division Doubling the speed of my game's graphics [Voxel Devlog #18] @ 03:05.
Evolution of the Rendering Pipeline
Achieving real-time frame rates across diverse hardware forced multiple architectural shifts across three core rendering paradigms:
| Technique | Primary Strengths | Primary Bottlenecks / Weaknesses | Optimal Hardware Target |
|---|---|---|---|
| Compute Shader Ray Marching Implementing sparse voxel octrees and the ray caster [Voxel Devlog #1] @ 05:05 | Natural dynamic lighting, shadows, and reflections; direct voxel lookups. | GPU memory bandwidth saturation on low-end cards; register pressure Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 01:00. | Discrete high-bandwidth GPUs (e.g., GTX 1660 Ti, RTX series). |
| Parallax Ray Marching Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 05:04 | Leverages GPU raster hardware; bypasses quad generation per face; low triangle count. | Requires manual fragment depth writes, disabling hardware Early-Z optimizations Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 08:08. | Integrated GPUs (e.g., Intel UHD) and WebAssembly. |
| Greedy Meshed Rasterization Codebase OVERHAUL, new EVENT SYSTEM, LODs, and MORE [Voxel Devlog #9] @ 07:10 | Fully hardware-accelerated pipeline; optimal use of early depth rejection. | Triangles explode on non-coplanar rough surfaces (slopes, natural terrain) Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 04:04. | Standard desktop polygon pipelines. |
| Hybrid Path Tracing / DDGI Adding global illumination to my game engine w/ DDGI [Voxel Devlog #23] @ 01:01 | Physically plausible multi-bounce global illumination without stochastic noise. | Substantial bookkeeping for probe memory allocation and sampling interpolation Adding global illumination to my game engine w/ DDGI [Voxel Devlog #23] @ 04:06. | Modern mid-to-high tier GPUs. |
To make parallax ray marching viable on integrated GPUs, early depth testing must be reclaimed by rendering bounding box geometry and deferring precise depth generation to a separate custom depth pass Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 09:09. Furthermore, CPU-side culling pipelines reduce pipeline driver overhead via glMultiDrawArrays combined with three spatial tests: expanded frustum culling, camera-relative octant culling (which prunes 50% of box faces located entirely to one side of the viewer), and directional face culling The PERFECT voxel rendering pipeline (and online demo) [Voxel Devlog #7] @ 08:15.
When full-fidelity ray tracing was reintroduced, primary rays were accelerated via a conservative beam optimization: a coarse-resolution pass establishes minimal ray distances across a neighborhood of pixels, allowing full-resolution rays to safely skip empty foreground air without missing thin features Doubling the speed of my game's graphics [Voxel Devlog #18] @ 09:13.
Illumination, Denoising, and Volumetric Light Transport
Direct ray tracing supports instantaneous hard shadow rays Textures, lighting, and MUCH faster rendering [Voxel Devlog #2] @ 02:00, but simulating soft ambient occlusion and multi-bounce indirect lighting requires specialized caching models:
- Per-Voxel Hashmap Accumulation: Evaluating Monte Carlo path-traced lighting on a per-pixel basis causes high-frequency noise. Storing visible surface voxels inside a GPU hashmap (using atomic compare-and-swap operations) allows the renderer to evaluate direct sun shadow rays once per voxel rather than once per pixel, saving 1–2 ms on discrete GPUs Emissive voxels and fancy lighting [Voxel Devlog #19] @ 08:45.
- Temporal and Edge-Preserving Denoising: Path-traced indirect samples undergo four iterations of À-Trous wavelet filtering to blur irradiance without bleeding across geometry discontinuities, followed by exponential temporal accumulation within the per-voxel hashmap Emissive voxels and fancy lighting [Voxel Devlog #19] @ 09:45.
- Dynamic Diffuse Global Illumination (DDGI): To eliminate path-traced stochastic boiling altogether, irradiance probes are arranged on a uniform $16^3$ voxel cell grid. Breadth-first traversal down local contrees pushes probes into empty spaces closest to surrounding geometry surfaces Adding global illumination to my game engine w/ DDGI [Voxel Devlog #23] @ 05:08. During runtime, an active probe compute shader evaluates whether surfaces exist within adjacent cells, culling inactive probes from the worklist Adding global illumination to my game engine w/ DDGI [Voxel Devlog #23] @ 07:11. Surviving probes sample incoming radiance via Fibonacci spheres, encoding irradiance and radial depth into octahedral mappings that are sampled via trilinear interpolation with visibility test weights to eliminate wall light leaking Adding global illumination to my game engine w/ DDGI [Voxel Devlog #23] @ 02:03, 10:15.
- Volumetric Fullness Ambient Occlusion: In rasterized pipelines lacking compute probes, analytical ambient occlusion is approximated by calculating the voxel density inside $16^3$ neighborhood volumes during the meshing phase. Storing this density allows the GPU to interpolate shadowing between cells using native hardware texture filtering in a single texture read Adding ambient occlusion to my game engine [Voxel Devlog #15] @ 08:34, 11:37.
Physics, Collision Resolution, and Destruction
Simulating arbitrary destructible voxel rigid bodies introduces severe stability challenges that off-the-shelf mesh engines do not address efficiently.
Physics Iterative Simulation Loop (TGS):
1. Detect Contacts (Narrow-phase: Corners & Edges with rounded shells)
2. Predict Velocities (External Forces + Warm Starting from Contact Map)
3. Solve Velocity Constraints Iteratively
4. Integrate Displacements (Zero drift for resting bodies)
5. Generate Fracture Events (Evaluate Impulse > Threshold)
Collision Detection Geometry
Treating voxels as pure cubes generates massive, redundant contact manifolds; an object resting on a flat floor creates contact points across every face voxel, over-constraining the solver Coding rigid body physics for voxels [Voxel Devlog #20] @ 03:46. Conversely, approximating voxels as pure spheres causes sinking artifacts and artificial surface friction My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 08:51.
The optimal configuration isolates collision checks to corner (empty along 2+ axes) and edge (empty along 1 axis) voxels using rounded geometrical offsets, running sphere-sphere, plane-sphere, and cylinder-cylinder queries against adjacent volumes Coding rigid body physics for voxels [Voxel Devlog #20] @ 04:47, My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 09:54.
Solver Dynamics and Warm Starting
Standard split-impulse solvers update positions prior to constraint solving, generating numerical jitter for resting bodies. Adopting Temporal Gauss-Seidel (TGS) executes an inner loop where velocity constraints are iteratively solved before updating spatial transforms My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 11:57. Persistent contact manifolds track reaction forces between frames using the coordinate pairs of the two colliding voxels as hash keys, enabling warm starting that stabilizes stacks of up to six continuous bodies My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 02:03, 12:58.
Structural Disconnection and Dynamic Fracture
When voxels are destroyed, structural integrity is evaluated via connected component labeling. Rather than running flood fills over millions of raw voxels, a hierarchical Depth-First Search (DFS) traverses the contree, treating uniform occupied octants as unified meta-nodes and caching visited spaces inside a bitmask Chopping trees DOWN [Voxel Devlog #12] @ 06:04, 07:06.
If a body suffers an impact where the contact impulse (independent of simulation tick length $\Delta t$) exceeds material-specific rupture thresholds, a fracture event triggers How I made voxels SHATTER on impact [Voxel Devlog #28] @ 02:02. Precomputed directional boolean voxel volumes carve empty cracks into the object, allowing the existing disconnector to naturally split the volume into distinct rigid bodies without custom geometry fragmenting algorithms How I made voxels SHATTER on impact [Voxel Devlog #28] @ 05:09, 07:12.
World Streaming and Algorithmic Acceleration
Generating and modifying massive worlds in real time requires avoiding brute-force sampling over volumetric data:
- Interval Arithmetic Terrain Generation: Terrain algorithms compiled to low-level bytecodes execute two compute shader stages: an interval arithmetic stage and a fine-sampling stage GORGEOUS, speedy terrain generation [Voxel Devlog #14] @ 03:05, 07:24. By tracking minimum and maximum range outputs across monotonic functions (e.g., $f(x) = 2x$), the pipeline determines whether an entire $16^3$ voxel chunk is guaranteed to evaluate purely to air ($>0$) or purely to stone ($\le 0$), skipping fine evaluation for 50% to 80% of all generated volumes GORGEOUS, speedy terrain generation [Voxel Devlog #14] @ 07:24, 08:26.
- Server-Generated LOD Caching with Dirty Sub-Octants: Generating full-resolution chunks on the server to downsample them for distant clients wastes memory and CPU time. Instead, the server uses scale factors directly within procedural noise invocations to evaluate LOD chunks natively How I tripled the render distance in my game engine [Voxel Devlog #16] @ 04:06. When a player edits terrain, sub-octants in the spatial LOD octree are flagged as dirty; higher-level cached LODs are loaded from disk, merged with selectively downsampled dirty sub-octants, and saved back to storage How I tripled the render distance in my game engine [Voxel Devlog #16] @ 07:49.
- Implicit Voxel Normals and Memory Alignment: Explicitly packing per-voxel normals creates massive data authoring friction during runtime CSG operations (such as sphere filling or copying between trees) Grass, textures, and a new codebase [Voxel Devlog #22] @ 04:05. Instead, calculating surface normals implicitly at upload time allows voxels to remain minimal material indices, enabling procedural triplanar mapping and GPU-driven foliage decoration buffers to be generated cleanly on the fly Grass, textures, and a new codebase [Voxel Devlog #22] @ 08:34, 12:40, 13:42.
Software Architecture & Language Transitions
Building an extensible, sandboxed engine across web and desktop platforms involved several architectural rewrites:
Runtime Engine Stack:
┌──────────────────────────────────────────────────────────┐
│ C# High-Level Front-End (Entities, User Logic, egui.NET) │
└────────────────────────────┬─────────────────────────────┘
│ Interop (C FFI / Serde)
┌────────────────────────────┴─────────────────────────────┐
│ Rust Core Engine Backend (SIMD Contrees, Physics, TGS) │
└────────────────────────────┬─────────────────────────────┘
│ Compute / Graphics API
┌────────────────────────────┴─────────────────────────────┐
│ WebGPU / Vulkan Shaders (DDA Traversal, DDGI Probes) │
└──────────────────────────────────────────────────────────┘
- Language Ecosystem Progression: Early iterations relied on C# with Vulkan Implementing sparse voxel octrees and the ray caster [Voxel Devlog #1] @ 02:00, but memory management limits and CLR safety overhead showed lower algorithmic execution speeds than native equivalents Sparse voxel octree modification and benchmarking [Voxel Devlog #3] @ 05:07. The engine pivoted to Rust for native WebAssembly compilation Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] @ 02:02. However, building a plugin API using pure WebAssembly runtimes proved awkward due to sandboxed memory address isolation and complex interface plumbing Grass, textures, and a new codebase [Voxel Devlog #22] @ 02:02.
- The Hybrid Architecture: The final architecture splits cleanly: a performance-critical Rust backend handles tree storage, SIMD transformations, and GPU command dispatch, while exposing a safe C# scripting layer via C-FFI for gameplay, entities, and tooling Grass, textures, and a new codebase [Voxel Devlog #22] @ 06:09.
- Automated Interop Generation: To integrate modern UI without exposing unsafe pointers or relying on fragile manual bindings, an automated tool ingests
rustdocJSON definitions and serializes types usingserde_generate, automatically compiling C# wrapper methods for 80% of the immediate-modeeguilibrary Adding UI to my hybrid C# game engine was surprisingly tricky [Voxel Devlog #24] @ 11:14, 12:16.
⚡ Action Items
- Replace Deep Octrees with Wide Contrees: Re-architect spatial indices to use $4^3$ branching layouts backed by a 64-bit occupancy mask. Evaluate ray occupancy using bitwise instructions (
AND, shifts) to allow rays to traverse up to 10 empty regions per step within registers Adding ray tracing (back) to my game engine [Voxel Devlog #17] @ 08:21. - Implement Interval Arithmetic for Procedural Generation: Wrap world-generation math routines inside range bounds $([\min, \max])$. Query bounding volumes prior to voxel generation and write uniform solid or air tags to skip evaluating 50%+ of all blocks GORGEOUS, speedy terrain generation [Voxel Devlog #14] @ 07:24.
- Switch to Temporal Gauss-Seidel for Voxel Physics: Move velocity constraint resolution before position updates in the rigid body loop. Cache contact reaction forces between ticks keyed on voxel coordinate pairs to enable warm starting and eliminate resting jitter My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 11:57, 12:58.
- Prune Voxel Collision Tests to Corner and Edge Offsets: Classify voxel geometry locally as corners, edges, faces, or interiors. Restrict narrow-phase contact testing strictly to corner and edge primitives modeled with rounded radii to prevent over-constraining the solver while maintaining rotation invariance Coding rigid body physics for voxels [Voxel Devlog #20] @ 04:47, My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] @ 09:54.
Source Overview
| Video | Channel | Duration | Quality | Only here |
|---|---|---|---|---|
| Implementing sparse voxel octrees and the ray caster [Voxel Devlog #1] | Douglas | 8:41 | Must Watch | Sparse voxel octrees compress 256³ grids from 16 MB to under 0.5 MB by storing homogeneous regions as single nodes. |
| Textures, lighting, and MUCH faster rendering [Voxel Devlog #2] | Douglas | 7:39 | Worth It | Aligning position offsets and material data per octree level instead of keeping them separate eliminates buffer reads and doubles traversal speed despite 30–50% data size penalty. |
| Sparse voxel octree modification and benchmarking [Voxel Devlog #3] | Douglas | 7:36 | Worth It | — |
| Drawing MILLIONS of voxels on an integrated GPU with parallax ray marching [Voxel Devlog #4] | Douglas | 12:10 | Must Watch | Parallax ray marching technique draws 8³ voxel bounding boxes via rasterization with bounded ray casts delegated to fragment shaders, reducing triangle counts by an order of magnitude. |
| Adding MULTIPLAYER NETWORKING with WebRTC to my game engine [Voxel Devlog #5] | Douglas | 11:41 | Worth It | — |
| Designing a FLEXIBLE game engine with Rust [Voxel Devlog #6] | Douglas | 12:31 | Worth It | — |
| The PERFECT voxel rendering pipeline (and online demo) [Voxel Devlog #7] | Douglas | 14:40 | Worth It | — |
| GPU-generated DISTANCE FIELDS [Voxel Devlog #8] | Douglas | 12:53 | Skip | — |
| Codebase OVERHAUL, new EVENT SYSTEM, LODs, and MORE [Voxel Devlog #9] | Douglas | 8:49 | Worth It | — |
| Adding SNOW and MODEL IMPORTS [Voxel Devlog #10] | Douglas | 13:32 | Skip | — |
| Making voxels MOVE with the separating axis test [Voxel Devlog #11] | Douglas | 14:08 | Worth It | — |
| Chopping trees DOWN [Voxel Devlog #12] | Douglas | 16:49 | Worth It | — |
| OPTIMIZING my physics engine [Voxel Devlog #13] | Douglas | 9:30 | Worth It | — |
| GORGEOUS, speedy terrain generation [Voxel Devlog #14] | Douglas | 12:20 | Must Watch | Interval arithmetic on compute shaders evaluates whether entire voxel regions output uniform air or solid material, skipping fine-grained sampling for 50–80% of terrain chunks. |
| Adding ambient occlusion to my game engine [Voxel Devlog #15] | Douglas | 13:35 | Skip | — |
| How I tripled the render distance in my game engine [Voxel Devlog #16] | Douglas | 13:50 | Worth It | — |
| Adding ray tracing (back) to my game engine [Voxel Devlog #17] | Douglas | 12:29 | Must Watch | 64-tree (Tetra-Hexacontatree / contree) with 64-bit occupancy masks allows rays to traverse up to 10 empty steps per lookup by evaluating spatial occupancy entirely within GPU registers via bitwise operations. |
| Doubling the speed of my game's graphics [Voxel Devlog #18] | Douglas | 13:01 | Must Watch | Beam optimization conservative coarse-resolution pass establishes minimum ray skip distances, enabling full-resolution rays to safely traverse empty space without missing thin geometry. |
| Emissive voxels and fancy lighting [Voxel Devlog #19] | Douglas | 13:08 | Worth It | — |
| Coding rigid body physics for voxels [Voxel Devlog #20] | Douglas | 13:15 | Worth It | — |
| Creating a modding system with Rust and WebAssembly [Voxel Devlog #21] | Douglas | 10:35 | Skip | — |
| Grass, textures, and a new codebase [Voxel Devlog #22] | Douglas | 18:32 | Worth It | — |
| Adding global illumination to my game engine w/ DDGI [Voxel Devlog #23] | Douglas | 13:39 | Must Watch | Dynamic Diffuse Global Illumination (DDGI) encodes per-probe incoming radiance into octahedral mappings with per-texel depth maps to prevent light leaking through walls during trilinear probe blending. |
| Adding UI to my hybrid C# game engine was surprisingly tricky [Voxel Devlog #24] | Douglas | 14:34 | Skip | — |
| Prototyping physics and procedural generation [Voxel Devlog #25] | Douglas | 9:26 | Skip | — |
| My voxel physics engine was BROKEN - here's every fix [Voxel Devlog #26] | Douglas | 15:37 | Must Watch | Temporal Gauss-Seidel (TGS) solver integrates position only after velocity constraints are solved, eliminating resting-body jitter; warm starting reuses contact reaction forces keyed on voxel coordinate pairs between frames. |
| How I made voxels SHATTER on impact [Voxel Devlog #28] | Douglas | 9:34 | Skip | — |
| Coding a character controller [Voxel Devlog #29] | Douglas | 12:01 | Skip | — |
| Adding joints to my physics engine [Voxel Devlog #30] | Douglas | 14:44 | Skip | — |