Mastering VL.Fuse in vvvv: Real-Time GPU Programming and Advanced Rendering

About the Instructor

www.riekoff.com

Christian Riekoff (aka @texone) is a media artist based in Schwerin. He studied Experimental Media Design at the Berlin University of the Arts and Media Programming at the University of Applied Science Berlin. Christhian’s work mainly focuses on interactive installations, generative systems and kinetic sculptures, merging a love for code with design skills. He is the main developer of VL.Fuse

01 Introduction to THE CORE PRINCIPLES

In this introductory workshop, we delve into the core principles of FUSE. We’ll explore how FUSE seamlessly integrates with vvvv gamma, empowering developers and artists with the ability to harness the full power of the GPU for visual programming. This lesson covers:

• Gain insight into the integration of FUSE with vvvv gamma and the Stride engine.
• Learn to port ShaderToy creations to FUSE.
• How to use tooltips and the viewer in FUSE for intuitive development.
• Exploring the shader drawing techniques, from simple to advanced versions.
• Understanding semantics in FUSE and how they influence programming.
• Experiment with texture sampling and its applications in GPU programming.
• Dive into the geometry stage and learn how to patch custom geometry shaders.
• Discover control structures like if regions, for regions, and custom functions.
• Harness the power of delegates for flexible and efficient programming.

02 All about noise functions

Noise functions are fundamental tools for generating textures, patterns, and simulations. In this lesson, we’ll explore the diverse range of noise functions available in FUSE and demonstrate how they can be utilized to enhance complexity and detail in your projects. From understanding the basics to mastering advanced techniques, this workshop will equip you with the skills to leverage noise functions creatively in your visual programming endeavours. This lesson covers:

• Explore domain warping techniques to distort noise patterns for unique visual effects.
• Explore the concept of randomness and its application in generating procedural textures.
• Understand lattice noises and learn techniques to manipulate them for desired effects.  
• Learn about value noise, gradient noise, and simplex noise.
• Dive into cellular noises, also known as Worley noise, and understand their applications.
• Gain insights into derivatives and their significance in noise functions.
• Implement Fractal Brownian Motion (FBM) for generating complex textures.
• Understand basic implementation, octaves, gain, and lacunarity.
• Create custom FBM variations using derivatives for added flexibility.
• Experiment with custom noises like wave noise, shard noise, and Gabor noise.

03 ALL ABOUT signed distance fields

Signed Distance Fields (SDFs) are powerful tools for rendering complex shapes and effects in 2D and 3D spaces. In this workshop, tailored for FUSE users, you’ll dive into the theory behind SDFs and learn how to implement them in FUSE to create intricate 3D models and animations. From understanding the basics to applying advanced techniques, this session will equip you with the knowledge to leverage SDFs effectively in your visual programming projects with FUSE. This lesson covers:

• Apply the same concepts and techniques to 3D SDFs with basic raymarching.
• Explore the concept of Signed Distance Fields and their applications in 2D space.
• Gain an overview of various 2D SDFs and learn how to combine them for complex shapes.
• Apply noise functions and use it as an SDF and explore its modulation and cracking effects.
• Learn domain operations to modulate the input of SDFs.
• Experiment with domain repeat and domain warping techniques.
• Utilize attributes to apply up to three values to an SDF and explore their usage based on color.

04 All about Raymarching in FUSE

Raymarching, a sophisticated rendering technique employed in FUSE, offers a unique approach to creating captivating visuals with intricate light interactions. Unlike traditional rasterization methods, raymarching traces rays through a scene to determine the intersection with objects, resulting in highly detailed and realistic renderings. This lesson will serve as your gateway to mastering the art of raymarching in FUSE, guiding you from fundamental concepts to advanced applications. You’ll delve into the core principles of raymarching, gaining a deeper understanding of its inner workings and how to harness its full potential for your creative projects. This lesson covers:

What is Raymarching Overview

• Explore the foundational principles and essential raymarching concepts and its advantages 
• Patch basic raymarching techniques to render simple shapes and scenes.
• Discover the versatility of material raymarching and how it enables the creation of complex surface materials and effects.
• Build a custom ray setup to tailor it to your specific project
• Dive into the realm of volumetric raymarching, where you’ll learn to simulate atmospheric effects, fog, and other volumetric phenomena with stunning realism.

Extension on 3D Signed Distance Fields (SDFs)

• Extend your raymarching expertise to 3D Signed Distance Fields (SDFs)
• Explore the world of fractals and terrains, leveraging raymarching techniques to generate intricate landscapes and structures.

Examples

• Put your newfound knowledge into practice with hands-on examples:
• Build a maze scene using raymarching techniques. 
• Create a morphing fractal, where you’ll witness the mesmerizing transformation of shapes and patterns through advanced raymarching methods.

05 Rendering techniques IN FUSE

Explore the rendering capabilities of FUSE, powered by the Stride 3D Engine. This workshop will provide insights on how to utilize the necessary components to produce high-quality graphics, featuring physically-based rendering (PBR) materials, advanced lighting techniques, and post-effects. Whether you’re a seasoned developer or an aspiring artist, this session will equip you with the knowledge to get the best out of your rendering pipeline. This lesson covers:

Color

• Discover various blend modes for achieving desired color effects.
• Understand color spaces and conversion techniques for accurate color representation.
• Explore utilities for manipulating colors and enhancing visual fidelity.

Feedback Texture

• Learn how to use feedback textures for dynamic visual effects and interactions.

• Implement Mat Cap materials for simulating reflective surfaces.
• Dive into normal mapping to add intricate surface details.
• Master triplanar texture mapping techniques for texture application on complex geometries.
• Understand the principles of Fresnel and its role in material rendering.

Techniques

• Explore indirect rendering for enhanced realism and lighting effects.
• Patch a skybox light to create realistic outdoor environments.
• Utilize cubemaps for realistic reflections and environment mapping.
• Render 360-degree scenes for immersive experiences and virtual tours.

06 All about Compute shader in Fuse

Explore the power of compute shaders, vital for performing complex calculations on the GPU. This session will guide you through the utilization of compute shaders in FUSE, enabling you to optimize performance and achieve breathtaking computational graphics. Whether you’re a graphics enthusiast or a performance-oriented developer, this workshop will equip you with the knowledge to leverage compute shaders effectively for your visual projects. Lesson covers:

Compute Shader

• Understand the role of compute shaders and their capabilities.
• Learn how to utilize buffers and textures within compute shaders for efficient data processing.
• Explore interlocked operations for synchronization and data manipulation.
• Bring together various compute shader techniques to solve complex computational tasks.
• Append Buffers for dynamic data storage and processing.

Compute System

• Explore texture-based compute systems, with a focus on a basic example of simulating fluid dynamics.
• Set up a basic compute system, including resource management and compute stages.
• Implement buffer-based compute systems for efficient data manipulation.
• Select elements using indices.
• Render elements for visual feedback.
• Utilize debug data for performance analysis.
• Create animations without a particle system using compute shaders.

07 Working with Particles

Learn how to create and control particle systems in FUSE, empowering you to simulate realistic environments and effects, from smoke and fire to complex animated scenes. This session will provide you with the knowledge and skills to harness the power of particle systems, enabling you to bring dynamic and immersive visual experiences to life.

Use Emission:

• Understand the principles of particle emission, including amount and probability.
• Learn how to utilize the recycler for efficient particle management.
• Explore different emission strategies for achieving desired effects.

How to Integrate:

• Utilize the simulation node for integrating particle systems into your projects.
• Create custom integrations for tailored control over particle behavior.
• Harness the power of vector fields and steering behaviors to guide particle motion.

Render Particles:

• Render particles using draw shaders for custom visual effects.
• Utilize the material pipeline for advanced rendering of particle systems.

Use SDF with Particles:

• Integrate Signed Distance Fields (SDF) with particle systems for emission, collision, and forces.

Using the Trails Node:

• Learn how to emit lead particles and use lead particle data in emitted trails for dynamic effects.

Using PLY:

• Load PLY data into FUSE and render it within particle systems.
• Animate PLY data to create dynamic particle effects.

Use Spatial Hash:

• Create networked particle systems using spatial hash techniques.
• Implement flocking behavior for lifelike particle motion.

08 Fluid Simulations 

Dive into the world of fluid dynamics within FUSE and learn how to simulate liquids and gases. This session will cover the principles of fluid simulations and teach you how to implement them for creating realistic and interactive environments. Whether you’re interested in creating dynamic water effects or immersive atmospheric simulations, this workshop will provide you with the knowledge and skills to bring your visions to life. Lesson covers:

Introduction to Fluid:

• Gain an understanding of the fundamental principles of fluid dynamics.

Compose Fluid Interactions with SDFs:

• Explore how fluids interact with Signed Distance Fields (SDFs) to create realistic simulations. 
• Learn how to compose fluid interactions using SDFs for precise control over fluid behavior.

Render Fluids with Volumetric Raymarching:

• Discover how to render fluids using volumetric raymarching for stunning visual effects.

Particle-Fluid Interaction:

• Explore techniques for simulating particle-fluid interaction, adding realism to fluid simulations.