Tu banner alternativo

Riffusion

In this article, we are going to analyze Riffusion in detail, exploring its different facets and characteristics to understand its impact in various contexts. From its origin to its relevance today, Riffusion has aroused notable interest and debate, becoming a topic of interest for experts and the general public. Along these lines, we will examine its historical evolution, its implications in contemporary society and the possible ramifications it has for the future. This article seeks to provide a comprehensive perspective on Riffusion, thus offering a solid starting point for those interested in delving into this complex and fascinating topic.

Tu banner alternativo
Riffusion
Developers
  • Seth Forsgren
  • Hayk Martiros
Initial releaseDecember 15, 2022
Repositorygithub.com/hmartiro/riffusion-inference
Written inPython
TypeText-to-image model
LicenseMIT License
Websiteriffusion.com
Generated spectrogram from the prompt "bossa nova with electric guitar" (top), and the resulting audio after conversion (bottom)

Riffusion is a neural network, designed by Seth Forsgren and Hayk Martiros, that generates music using images of sound rather than audio.[1]

The resulting music has been described as "de otro mundo" (otherworldly),[2] although unlikely to replace man-made music.[2] The model was made available on December 15, 2022, with the code also freely available on GitHub.[3]

The first version of Riffusion was created as a fine-tuning of Stable Diffusion, an existing open-source model for generating images from text prompts, on spectrograms,[1] resulting in a model which used text prompts to generate image files which could then be put through an inverse Fourier transform and converted into audio files.[3] While these files were only several seconds long, the model could also use latent space between outputs to interpolate different files together[1][4] (using the img2img capabilities of SD).[5] It was one of many models derived from Stable Diffusion.[5]

In December 2022, Mubert[6] similarly used Stable Diffusion to turn descriptive text into music loops. In January 2023, Google published a paper on their own text-to-music generator called MusicLM.[7][8]

Forsgren and Martiros formed a startup, also called Riffusion, and raised $4 million in venture capital funding in October 2023.[9][10]

References

  1. ^ a b c Coldewey, Devin (December 15, 2022). "Try 'Riffusion,' an AI model that composes music by visualizing it".
  2. ^ a b Llano, Eutropio (December 15, 2022). "El generador de imágenes AI también puede producir música (con resultados de otro mundo)".
  3. ^ a b Nasi, Michele (December 15, 2022). "Riffusion: creare tracce audio con l'intelligenza artificiale". IlSoftware.it.
  4. ^ "Essayez "Riffusion", un modèle d'IA qui compose de la musique en la visualisant". December 15, 2022.
  5. ^ a b "文章に沿った楽曲を自動生成してくれるAI「Riffusion」登場、画像生成AI「Stable Diffusion」ベースで誰でも自由に利用可能". GIGAZINE. 16 December 2022.
  6. ^ "Mubert launches Text-to-Music interface – a completely new way to generate music from a single text prompt". December 21, 2022.
  7. ^ "MusicLM: Generating Music From Text". January 26, 2023.
  8. ^ "5 Reasons Google's MusicLM AI Text-to-Music App is Different". January 27, 2023.
  9. ^ Gal, Dr. Itay (February 10, 2025). "Free A.I. music creation platform launches, competing with Suno". The Jerusalem Post. Retrieved February 16, 2025.
  10. ^ Nuñez, Michael (January 30, 2025). "Riffusion's free AI music platform could be the Spotify of the future". VentureBeat. Retrieved February 16, 2025.


Stub icon

This artificial neural network-related article is a stub. You can help Wikipedia by expanding it.

Stub icon

This scientific software article is a stub. You can help Wikipedia by expanding it.

Wikious
Boobota
Sagapedia