Riffusion

Riffusion is an open-source AI model designed to compose music by visualizing it with spectograms. It uses the v1.5 stable diffusion model, which takes a spectrogram and text as input and generates AI music as output. To create this experience, the developers have used Next.js, React, Typescript, three.js, TailwindCSS and Vercel to build the application that communicates over an API for inference calls on a GPU server. The team utilized Truss to package the model and test it locally prior to deploying it onto Baseten – providing GPU-backed inference along with auto-scaling and observability capabilities; in production they are using NVIDIA A10Gs GPUs for this purpose. If you possess a sufficiently powerful GPU card capable of generating stable diffusion results within five seconds or less then you may opt to run the experience on your machine through our local test flask server setup.