Live Diffusion Visuals with TouchDesigner

This came out of a TouchDesigner class I took, and it’s one of those projects that just kept getting more interesting the further I pushed it. The setup: TouchDesigner generates live visuals using Stable Diffusion, and you control the output with hand tracking and face tracking inputs. Geometric solids serve as compositional guides, OpenAI handles the language model side, Fal runs the Stable Diffusion model, and a Replit-hosted WebSocket server ties everything together. The really fun part is changing the prompt live — you can go from “Pacific Northwest forest” to “pizza party” and watch the visuals regenerate in real time while you’re still waving your hands around controlling the composition.

The stack and why it works

What makes this technically interesting is how many systems are talking to each other simultaneously. Hand tracking data flows through WebSockets to influence the diffusion output, face tracking adjusts another dimension, and the LLM is interpreting and enriching prompts on the fly. It’s not a polished product — it’s an experiment in what happens when you wire a bunch of cutting-edge tools together and start performing with them. And honestly, “performing” is the right word here. There’s something fundamentally different about generative visuals that respond to your body versus ones that just play back.

The creative technology mindset

The second half of this video is a reflection I’ve been wanting to articulate for a while — the “how hard can it be?” philosophy. My background bounces between architecture and graphic design, and I think that cross-pollination is what lets me make connections that feel novel. I lurk in channels, run random experiments, try things that might not go anywhere, and all of it builds this mental inventory of what’s possible. So when someone on a team mentions a problem, I can say “I might have a solution for that” — not because I planned for it, but because I’d stumbled into the right corner of the technology landscape at some point. I even told the story of automating Figma-to-Google-Drive file organization for a colleague, which only happened because I’d been tinkering with those APIs for unrelated reasons. The takeaway is that creative technologists aren’t just building things — they’re collecting possibilities.