Integrating AI with Touchdesigner

Recording of October 2024 in English Language
Recordings are available and accessible for one year after purchase.

Brooklyn-based interaction designer Lyell Hintz, also known as DotSimulate, will demonstrate how to integrate AI into TouchDesigner components through hands-on demonstrations and shared project files.

Building on his Summer Season 24 course, Lyell will focus on operator design and workflows. Lyell will focus on operator design and workflows. The course will place less emphasis on ComfyUI itself (for a deeper dive into that, consider checking out Luna Nanes’ NODE Toolkit course, ComfyUI – Generative AI for Professional Production). Instead, the focus will be on design decisions when creating workflows in ComfyUI for use in TouchDesigner, as well as addressing common challenges. Lyell will demonstrate how to implement a specific workflow in TouchDesigner while providing resources to generalize the approach for broader applications.

He will introduce a collection of techniques aimed at making his operators more transparent and generally applicable, enhancing TouchDesigner’s core functionality with custom operators and AI-driven components.

Additionally, Lyell will break down a few functional AI TouchDesigner components (such as LivePortrait and ChatTD) to illustrate how he employs extensions, Python passthrough tricks, and OSC to communicate with external processes. This will offer participants an optimized template for experimenting with new techniques and models. Finally, Lyell will share how he leverages tools like Claude or other large language models (LLMs) to support initial research and accelerate the coding process.

What You’ll Learn:

1. AI-Driven Workflows & Operator Design
   Learn how to integrate generative AI models into your TouchDesigner projects effectively.
2. ComfyUI & TouchDesigner Integration
   Explore a case study on implementing workflows in both ComfyUI and TouchDesigner using ComfyTD, highlighting common challenges and automation techniques.
3. Practical AI Components
   Analyze functional AI-powered tools in TouchDesigner, such as LivePortrait and ChatTD, while learning how to customize and expand them.
4. Working with External Tools
   Master the use of Python libraries, OSC, and web APIs to connect AI systems with TouchDesigner.
5. Using AI for Development
   Leverage Large Language Models (LLMs) like Claude to accelerate research, coding, and workflow automation.

What You’ll Get:

• Downloadable resources, including project files, AI-driven components, and templates.
• Access to a private community forum for networking, feedback, and support.
• A certificate of completion to showcase your expertise in integrating AI with TouchDesigner.

Why Enroll:

This course offers a unique opportunity to learn from Lyell Hintz, aka DotSimulate, an expert in interaction design and AI integration. By mastering the tools and techniques to incorporate AI into your TouchDesigner projects, you’ll be equipped to integrate new AI models and techniques into your work.

Requirements

This module is designed for participants with an intermediate understanding of TouchDesigner and some familiarity with Python. or a more basic introduction to AI integration, please refer to Lyells Summer Season course. If you just getting started, please visit our Beginners Course first.

• A recent PC or MAC Computer (discrete GPU recommended) with stable internet connection
• The latest build of TouchDesigner installed
  
• All techniques teached in this course will work on MAC, but note that the generation time for certain things will be considerably slower.
• Be prepared to download and store some of the larger model files beforehand (~20GB)
• 3-Button Mouse recommended

About the Instructor

instagram.com/dotsimulate

Lyell Hintz, aka dotsimulate, based out of Brooklyn, New York, is a creative/technical artist passionate about building memorable interactions with technology. With a background spanning theater, performance, graphic design, and music, Lyell excels at translating narratives into abstract creations of new media and technology. For the past four years, he has been focused on generative simulations and procedural animation, and more recently, experimenting with AI within creative workflows.His passion for exploring the potential of technology has led him to developing tools using TouchDesigner as his primary development tool. Through his Patreon, Lyell shares many custom tools, tutorials, and feedback with a community of artists and enthusiasts, searching for ways to empower himself and others to develop and learn new ways of creating.