Squishy Sonics

SquishySonics is an interface designed to explore mappings between organic physical shapes made through interactions with deformable materials and sound production using machine learning. Deformable materials are proposed as an intuitive yet ambiguous medium for interfacing with AI-driven techniques, such as navigating the latent space in complex AI audio models, which are often difficult for musicians to understand and control expressively.

The system comprises three modular elements: a camera, an interface-to-sound mapping tool, and sound generation methods. Flexibility and modularity are afforded by loosely coupled components and a bottom-up design approach. Polymer clay is used for its ergonomic, low-cost, and familiar properties, enabling hands-on exploration without fear of damaging the interface. Users can represent their perceptual links between shapes and sound by mapping deformations to parameters controlling real-time sound processes. SquishySonics encourages curiousity in the unknown, minimising frustration in complex information tasks, by allowing for physical engagement with abstract concepts.

Project by Trish Khallaghi, supervised by Atau Tanaka and Pete Bennett

Squishy-Sonics Github repo: https://github.com/trishaqk/squishy-sonics

Squishy-Sonics will be presented as a demo in the Interactivity track of CHI’25 in Yokohama, Japan