Case Study – What You Don’t See

I have spent the past 15 years circling around the entertainment industry, facsinated by all aspects and due to my nature, I could not settle on anything specific. As a generalist, chipping away at each facet of a production to become holistically capable, this project brought together all the parts I care about and more.

I am a trained Architect, however I never focused on the conventional. Since school, I have been a host for events, eventually formalizing productions with a focus in experiences. These experiences, which began in a boundless creative zone because of my education, started in the built environment and have slowly grown into a multimedia collage. I started with adding animations to space and from there, my first offical productions gigs, led me to higher tech in the industry. Lights, fog, lasers, pyro, automations, all able to be controlled…. Since the start I was hooked into realtime interaction, and being able to control the environment became the focus. Once I entered FOH, I understood that this is where I can make things happen.

In recent years, building up a brand has allowed me to fomalize this process. Each time I get another opportunity, the refinement continues. The Banyan Tree marks my biggest solo exhbition to date. I willl promise you one thing, you never stop learning, and I will never do it that way again!

We are all here because the path we have chosen led us here. The choices you have made, the knowledge you have gain and the curiosity for a greater understanding have brought to this moment. If it happened a different way, you would not read this. I think its important to remember that. Keep going and you will end up where you want to go.

The Process

One thing that I believe that is left unnoticed often is the amount of work that goes into the project before show day. Alot of people here might be involved with a single aspect to the creation of an installation, but rarely people within the Touch Designer realm engage in all aspects of the project. It is important to make sure you are not overloaded, especially in a production with a finish line. Many factors come into play and you must but flexible to succeed. You also must have a relaxed perspective on what is “complete”.

Here is my list of steps to accomplish a project at a professional level:

  • RFP – The Space, Theme, Time, Type of Event

https://www.thesixthsensefestival.com

  • Design Phase
  • What It Takes to Win a Bid
    • Concept + Renders
    • Narrative
    • Initial Budget Range
    • Flexibility
    • Negotiation / Scope of Work
  • Pitching the Design + Initial Budget
  • Project Management
    • What Roles are Needed
    • What Do I Want to Do In the Project
    • Production Timeline / Phases
  • Meetings
  • Changes Along the Way
  • Working In Teams
  • Technical Design
    • Power
    • Data
    • Audio
    • Lighting
  • Procurement + Actual Budget
    • Local Vendors Quotes
  • Software / Hardware
    • Preproduction
    • Integration
    • Calibration
  • Build – Fabrication / Install
  • Tech Install
  • Other On Site Cooperations
  • TESTING / Programming
  • Show Days
    • Run of Show procedures
  • Strike
  • Post Mortem

These steps are crutial for a production process. They exist in any project but may differ in amount of time spent depending on the scale of the work. When producing your gantt charts, a big picture overview of the entire timeline, it is important to give space for all of these elements. I normally break my schedule into two, built elements and technical elements. Each of these will have a pre production and production phase. Before the pre production phase, I include an overall design / research phase.

Project Breakdown

The Banyan Tree Concept:

Emerging amidst a time of critical mass, nurtured by the surrounding history of the glass manufacturing complex, The Banyan Tree observes the patterns of societal development. Through details of fabricated technologies, material definition and synthetic emotions, it offers a reflection of the natural intelligence that surrounds us. It is the manifestation of Humanity’s connection to Nature.

Ignoring the power that exists in our environments, We, the People of Technology, have been enchanted by the digital revolution. The Banyan Tree, formed from the elements that we have adopted into our identity: windows, mirrors, lights, smoke and lasers, is here to speak with us. To those who do not give focus for more than a moment; anything except the minor dopamine rush, the exhilaration of what’s next… 

There is some faint echo of discernable language emitting from the core. A voice, a warning, a mantra, layered with sounds from Earth, natural and unnatural, organic and mechanical. The Banyan Tree knows… It is aware of who is within its domain. It invites its visitors to stop and reflect on the power of Nature through the lens we have embraced: Technology, the built, not  the grown. It is a conversation between humanity and their circumstantial truth. 

It asks us….. When did we stop communicating? What happened to the connection within ourselves? What are we… but Nature?

The Banyan Tree presents a story of mutual relationships and symbiosis between elements. As the vine is to the tree, so is the light to the fog. As the ant to the queen, is the mirror to the laser. The architecture of the space is built around layers of abstract digital information, to be revealed by the People. Only through the collective intelligence of the community will The Banyan Tree be shown in its true form. The bouncing networks of light within reminds the viewer that while we may have lost our bond, Nature continues to manifest around us. It is the beauty of interconnectedness – that One does not compare to the brilliance of All. Move through the space and you will become part of the story.

The Main Conceptual Elements:

  • Mutual Relationships = Symmetry, + Asymmetry in architecture. A Natural Symbiosis
  • Interconnectivity – Lasers around the perimeter connecting to the core made of Mirrors, prisms, glass (reflecting on the physical presence of the location).
  • Collective intelligence – interactivity using lidars. As people interacted with the space, more things happened. moving of mirror panels and lights, LED screens activating. Lasers bouncing around. Smoke creating the atmosphere.

What is the Heartbeat?

The project takes lidar input covering the exhibition space and will trigger different sounds according to conditions prescribed by the concept story.

These triggers are also adjusting parameter connections to the audio levels across the content. The main elements of data I am using for my trigger condtions are:

  • Position
  • Number of Groups
  • Average Distance to Center
  • Fastest Group Speed
  • Time in Zones

With these data points, I can trigger a collection of presets that are timed with a cuelist. The trigger configuration table is easily reconfigured in real time, not ever messing with the actual trigger logic, but simply the conditions for why they trigger. The cuelist is also working with the Touch Designer lister, making it extremely easy to add presets with fade times and delays. These presets are controlling a number of outputs that I personally chose for the composition of the piece. They include:

  • 8 Sollinger Phaneon 10 Accuate Lasers
  • 8 35 KG dual cabled DMX winches (Custom LightNLight Product)
  • 42 Galaxy Transparent LED Panels
  • 36 Clay Paky X Frames
  • 21 Pixis Par Cans
  • 18 Clay Paky Tambora Flashes
  • 8 Clay Paky Tambora Battens
  • 8 Atomic 3000 Strobes
  • 5 Clay Paky K15 BEyes
  • 6 Clay Paky Panify 2
  • 2 Look Solutions Vipe NT Fog Machines
  • 2 Antari HZ1000 Hazers

These fixtures were broken up into 4 groups:

  • Audio Control
  • DMX fixtures to include and atompshere FX control
  • Laser Output
  • Video Control

PC1 controls all of the timeline, lidar system, trigger and presets system, along with the manipulation and output of DMX fixtures and lasers.

PC2’s Touch application was designed to maximize the resource needs for the installation. I sent over the commands through a LAN connection to control both audio coming from Ableton and video, which successfully dealt with the latency across the project. It was carefully considered to not use the audio data being sent to PC1 for controlling the video paraemters, so we chose to create a simpler sync locally which used scripts to align the audio data to certain parameters.

CAD to DATA

CAD is a necessary tool when relying on math in spatial art. Not only the accuracy, but for the speed of iteration and adaptation of the project as it gets closer to the result. This becomes the source of truth for you rentals and fabricated parts. In a perfect world, the virtual to physical should be a 1:1 copy. You should foresee any situation that may arise prior to the build to make it come to life as you imagine it.

This doesnt mean that you use all of the CAD files in your software to control the installtion, but taking the most important data points required to lay the foundation for your outputs. For every fixture I used, I exported a CAD file that held the real world coordinates based off a global origin that is reflected across the entire project. These files are then used at the starting point of my output patch creating the structure I needed to speak to each piece of Tech.

When exporting, OBJ files work best for speed and reliability for instancing, however it doesnt not store the rotations. This is the next step and must be done with another method. This may include custom internal POP workflows or exporting transform data from CAD software and implementing via the lookup channel POP.

It is important to align on your scale in the beginning phases and stay with it. Syncronize across all software, so that the reimport process is simple without needing to realign.

Every element that is for fabrication must undergo the same scrutiny. All dimensions for 3D printing, metal work, wood work, decorations must be accurate otherwise there will be hold ups during the on site process.

Project Walk Through:

*Please watch included recording to understand each bullet point.

Project Folder

  • On larger projects, it is important to keep the structure clear and readable. I am normally keeping my assets separated from the externalized components and scripts.
  • Having a some sort of project settings to quickly set main project variables will speed up your project restarts

Startup

  • Every project has an order of operations to consider when initially opening each day. Some parts must be enabled only once others have begun. I keep my scripts broken up by each major element. In case I dont have it set in the my project settings to start, other scripts will still run to get the project opened correctly.
  • This is the same in the closing procedure. Having a strict process can save alot of time going through your project saving elements individually

UI

  • The entire project is operated in perform mode.
  • The front end is only needed for those not operating the installation. This is good for branding and credits, sometimes also giving the client a basic layout to allow for easy updates to the configuration.
  • The back end is carefully chosen for system operation. We rely on keeping the controls as necessary as possible. It is important to make sure you have only exposed parameters and windows that are being used. Un used elements can slow down the project.
  • Previewing your data in an way that gives you the feedback to uderstand what is happening is, in my opinion, the best way to make sure youre on target. Staring at the data in the pipeline can give you incomplete feedback on what is actually going at the end. This is a time consumer, but worth it when your not able to have the rpoject tech with you during softwar production.

Presets

  • The main project holds a preset ssytem designed by PlusPlusOne (formly Alphamoonbase) and utilizes the dashboard and cuelist to complete the system.
  • Each main element component (ie lasers, video, audio, DMX, etc) will contain some data to link to the main dashboard found at the root. Everytime i recall a new asset. The dashboard is updated to reflect the presets that pertain to that element.
  • The presets live in a global folder, also found at the root. sine the current configuration will not change, there is no need to localize the presets per element. A global storgae can hold all presets, while the dashboard only links those that i find relevant. I wrote a preset finder script to help locate what i need to edit quickly.

Base Page

  • Within the main project folder you will find all compartmentalized elements divided with a description. Each COMP holds an entire collection of custom parameters that will affect what is happening internally. These parent components will then talk to one another through various means. Another reason this becomes helpful is for finding project bugs.
  • Ideally, in each project you will split up the workload. By having a strong base to work from, with clearly divided tasks, teams can work together without overriding work or doubing up on the same task.

Input – Lidar

  • Ive always been a believer in doing what you love and finding tools to speed up your flow gets you closer to this. For most of my lidar tracking projects, where content is my focus, I implement tools from the community. Dan Molnar, Aka FunctionStore, has created a wonderful tool that does the dirty work for me. Ive left a link to his patreon below. As well as an image to the UI he has created for calibration.

https://www.patreon.com/cw/function_store

  • It doesnt go without integration. While his tool works, finding a way to use the data is the real challenge. I implemented a previz component that translates to my project and took the data a step further when analyzing in the next stage of the project.

Data Analysis

  • After getting the collection of information, the only way to make sense of it is to tone it down a notch.
  • By finding varibles such as mean, median, maximum and minimum, you can take a large data set and reduce it to a single float.
  • Smoothing out the data is also important. Jittery information can collapse a setup. Adding delays before tirggers will clean up alot of these rapid shifts that makes the project look broken
  • This COMP also contains the global colors shared across the project

Triggers

  • Developing a complex system will eventually lead you to mulitple types of specific feature. By using a replicator and a table it is possible to get a vast range of conditions for whatever your project may need. The trigger system used in the project relys on 3 different types:
  • Zone Triggers: Each region designated by the lidar system along with any custom spaces, will be used as a I/O switch
  • Group Triggers: The amount of groups in the room will be used as a major trigger for what mode the project is in
  • Special Triggers: Certain special conditionsthat are particular with certain condtions, such as the fastest group, or average distance from the center can be enabled easily with an idex modifier.

Outputs

  • To complete the cycle, the outputs hve been divided by each type so that is possible to work on them one at a time. When developing, you can use the project settings to quickly disable all others not being used, and for the run of show, all can be activated. By setting things up this way, each output can have a collection of presets and when designing the triggers, you can stack the cuelist with each elements progression at that time. The 4 outputs used in this project are:

Audio

  • This was a major driver in the project. Most triggers were accompanied by an audio element.
  • TDAbleton controlled the audio and by using Midi notes, we were able to change a random note to trigger a new sound. Each sound had its place in the Midi rack and was also connected to a form of modulation.
  • The sounds were classified into categories that were defined in the concept. There was a blend of natural and unnatural fx sounds that were triggered on top of a collection of base and top layers tracks. The timeline triggered a small group of calm tracks for a time and then another section would play a collection of excited tracks.
  • On top of the interactive audio, a timecoded audio track was also triggered at a specific moment.
  • The Audio came back in as an additional input to be used to drive alot of parameters for the other outputs. It was important that all levels were clean on the Ableton side of things. The cleaner the data coming in, the easier time we had for global calibration instead of updating the levels frequently.
  • It was important to understand the levels needed to be unique from the interactive element vs the timecoded element.

DMX

  • When dealing with many fixtures, a repeatable process is the only way to stay in a controlled circuit. Once the format was decided, each fixture repeated the same process which allowed for me to use the same control parametrs on each type. Keeping the thought process clean.
  • While a main workflow was used, each fixture was unique, making it impossible to make an exact clone. Using the immune function, I was able to control alot from the custom parameters which out having to worry about whats inside.
  • Winch control existed inside the DMX flow, which intimately needed to be synced with the lasers for proper displacement and scaling. There was alot of back and forth for real world alignment.
  • This project used a GrandMA3 Lighting console. The output worked best with sACN due to hardware compatibility with artNet versions.
  • A more indepth understanding will follow in the next topic, where we will dive into the process of using the DMX POP workflow.

Lasers

  • I knew that, while the lasers we not the only element in the project, they were clearly the defining tech in the piece. We built an entire system to facilitate the user friendly content creation. Using TOPs to create the low resolution grid of lasers to targets. All of the control for color and brightness dereived from this TOP workflow.
  • From there, each laser took their data stream and applied the proper attributes, in POPs, before outputting to Sollingers proprietary system. The Touch flow, began in POPs, moved to SOPs and CHOPs, came back into POPs to implement TOP data, Then back into SOPs for laser output. What is interesting is that we used Sollinger inputs at the end of our process, simply using the copy POP before the final conversion.
  • I will stress again, using tools created by others is part of the process. By utilizing what LaserAnimation had to offer, the work stayed in the realm of storytelling vs, building systems.

Video

  • Traditional application of video has been saturated in the past couple of decades and I wanted to create a unique display that gave character to the piece. The a-symmetrical arrangement, divided between trunk and canopy, or top and bottom, showed real time content that was spatially divided.
  • We used a camera technique to grab the actual view from that screen in space. and arranged that onto the appropriate mapping for the system.
  • Three defining tecniques used in this process included
    • Masking techniques implemented per visual.
    • a basic quick setup state machine with a calm and excited mode
    • audio syncing on two major parameters per visual
  • All content was running on PC2 and used a sync technique that allowed it to be controlled seamlessly from PC1.

System Architecture Progression

Building a Touch Designer project can be as easy or complicated as you want. Sometimes it involves creating new tools, sometimes you may find you can reuse. Most of the time there is an adaptation to get it working nicely with your new intention. I always try and learn from previous projects as i move onto the next. I dont always have the time to make the perfect version, so I constantly update tools, expand them, or just reverse engineer until I get the results im looking for. Knowing the extents of the tools, whether it be the Touch ecosystem, tools in the palette or your own, will greatly increase the capacity of your projects.

Using Touch as a Tool

Something that I see alot people doing is creating a single component that does everything. No use of a working UI, data charts, or exploration tools. The beauty of working in Touch is its ability for rapid prototyping while researching your topics. Before I decide on a final workflow and dive deep into production, I make sure I have a simple working version. This allows me to see if my intention will hold up without wasting many hours to implement. Sometimes i create a side project to run a theory through without the overhead of opening the project. Touch Designer allows the developer to help themselves solve problems as they go. Even if you may never use a tool to help you understand how space is handled, it could be the right stepping stone to save you days of work later on, when you are “in the weeds” on site.

Multi PC Setups

In a perfect world we would get proejcts with unlimited resources. We would create amazing and complex stories with many outputs that would run seemlessly with no latency. And the world would live in aweee……

I have been there. Its a beautiful thing. However, this is not always the case. Understanding the bottlenecks and what elements will cosume what resources is the beggining of setting up a technical plan for your install.

For the Banyan Tree, I found a resonable solution. Since we we able to rent our gear, I speced the setup to use two units. With so much data being processed it would be hard for all the outputs to come out clean. Especially since we went for real time audio.

Our two heaviest hitters were Ableton and video outputs. While ableton was mainly pulling from the CPU, the video complemented this with mainly operating on the GPU. This was an easy configuration for PC2.

This left the remainder of the project to run on PC1. The main control panel was also responsible for the UI, lidar input, data anaylsis and triggers, DMX and laser output and all presets for the project.

When the project is running, I normally left most of the UI turned off, only eposing the textport for feedback and the basic readout of the lidar positioning. Each moment of consideration about what is relevant helped maximize the resources available on the main machine.

Tips:

Compartmentalization

  • Global Data – You should always have a single source shared data. Do not recreate something that you will use in another place.
  • Autonomous Connections – Components should run stand alone when you drop them into your main porject. Your connections to other elements should be on the parent components as much as possible.
  • UI / Visibility – Seeing your results in realtime is importantto understand how the project functions. It can be a graphic or text, but make it easy to understand. print statements must be labeled so you can easily locate where things are happening. Be careful with cooking UIs! They are not alwyas needed.

How to Connect

  • Extensions – running scripts inside a component can get a bit messy. When using extensions, you can keep a clean collection of all your functions and call them throughout the project when needed.

https://derivative.ca/UserGuide/Extensions

  • Global Op Shortcuts – An easy way to find a component anywhere in the project.

https://derivative.ca/UserGuide/Global_OP_Shortcut

  • Internal Ops – Another way to access data inside of a component. But this is only working within, not globally

https://derivative.ca/UserGuide/Internal_Operators

  • Binding – bi directional use of data. When you touch it one place, you change it everywhere its linked. Sometimes, paired with referencing when implementing control interfaces.

https://derivative.ca/UserGuide/Binding

Repeated Data

  • Clones – From UI toolkits, to repeaded workflows, cloning with its 3 states, can keep thing partial or completely the same. Just remember when you left your unique clones!

https://derivative.ca/UserGuide/Clone

  • Replicators – A brilliant way to iterate over unique data in a static environment. Using replicators in you UI and configuations can greatly extend the options you have available to control.

https://derivative.ca/UserGuide/Replicator_COMP