Tu banner alternativo

EleutherAI

In today's world, EleutherAI is a topic that has captured the attention of many people. With its complexity and relevance in today's society, EleutherAI has sparked the interest of experts and hobbyists alike. From its impact on popular culture to its influence on technology and politics, EleutherAI is a phenomenon that does not go unnoticed. In this article we will explore the different facets of EleutherAI and its importance in our daily lives.

Tu banner alternativo

EleutherAI
Type of businessResearch co-operative
Founded3 July 2020 (2020-07-03)[1]
IndustryArtificial intelligence
ProductsGPT-Neo, GPT-J, GPT-NeoX, Pythia, The Pile, VQGAN-CLIP
URLeleuther.ai
Part of a series on
Artificial intelligence (AI)
Major goals
Approaches
Applications
Philosophy
History
Controversies
Glossary

EleutherAI (/əˈluːθər/[2]) is a grass-roots non-profit artificial intelligence (AI) research group. The group, considered an open-source version of OpenAI,[3] was formed in a Discord server in 2020 to create an open-source version of GPT-3.[4] In early 2023, it formally incorporated as the EleutherAI Institute, a non-profit research institute.[5] As of 2025, the nonprofit no longer trains its own models and instead maintains widely-used training datasets, conducts research and is involved in public policy.[4]

History

EleutherAI began as a Discord server on July 7, 2020, under the tentative name "LibreAI" before rebranding to "EleutherAI" later that month,[6][better source needed] in reference to eleutheria, the Greek word for liberty.[3] Its founding members are Connor Leahy, Len Gao, and Sid Black.[3] They co-wrote the code for Eleuther to serve as a collection of open source AI research, creating a machine learning model similar to GPT-3.[5]

On December 31, 2020, EleutherAI released The Pile, a curated dataset of diverse text for training large language models. While the paper referenced the existence of the GPT-Neo models, the models themselves were not released until March 21, 2021.[7] On June 9, 2021, EleutherAI followed this up with GPT-J-6B, a six billion parameter language model that was again the largest open-source GPT-3-like model in the world.[8][better source needed] These language models were released under the Apache 2.0 free software license and are considered to have "fueled an entirely new wave of startups".[5]

While EleutherAI initially turned down funding offers, preferring to use Google's TPU Research Cloud Program to source their compute,[3] by early 2021 they had accepted funding from CoreWeave (a small cloud computing company) and SpellML (a cloud infrastructure company) in the form of access to powerful GPU clusters that are necessary for large scale machine learning research. On Feb 10, 2022, they released GPT-NeoX-20B, a model similar to their prior work but scaled up thanks to the resources CoreWeave provided.[9][10]

In early 2023, EleutherAI incorporated as a non-profit research institute run by Stella Biderman, Curtis Huebner, and Shivanshu Purohit.[5] This announcement came with the statement that EleutherAI's shift of focus away from training larger language models was part of a deliberate push towards doing work in interpretability, alignment, and scientific research.[11][non-primary source needed] While EleutherAI is still committed to promoting access to AI technologies, they feel that "there is substantially more interest in training and releasing LLMs than there once was", enabling them to focus on other projects.[12]

In July 2024, an investigation by Proof news found that EleutherAI's The Pile dataset includes subtitles from over 170,000 YouTube videos across more than 48,000 channels. The findings drew criticism and accusations of theft from YouTubers and others who had their work published on the platform.[13]

As of February 2025, Eleuther was no longer training their own open-source AI models given their budget constraints of $3 million per year and the increasing number and capabilities of other models, instead looking for other niches.[4] In 2025, Eleuther released a new dataset for training AI that does not have the controversial copyrighted material contained in its previous release of The Pile.[14]

Research

EleutherAI works with hundreds of volunteer researchers.[15]

The Pile

Main article: The Pile (dataset)

The Pile is an 886 GB dataset designed for training large language models. It was originally developed to train EleutherAI's GPT-Neo models[16] but has become widely used to train other models, including Microsoft's Megatron-Turing Natural Language Generation.[17] Compared to other datasets, the Pile's main distinguishing features are that it is a curated selection of data chosen by researchers at EleutherAI to contain information they thought language models should learn and that it is the only such dataset that is thoroughly documented by the researchers who developed it.[18] The initial Pile dataset has come under scrutiny for containing copyrighted material[14] including books[19][20][21] and subtitles from documentaries, movies, television and online videos[22] including from YouTube.[23][13]

Common Pile

Common Pile v0.1, released in partnership with a large number of collaborators in June 2025, contains only works where the licenses permit their use for training AI models.[14]

GPT models

EleutherAI's most prominent research relates to its work to train open-source large language models inspired by OpenAI's GPT-3.[7] EleutherAI's "GPT-Neo" model series has released 125 million, 1.3 billion, 2.7 billion, 6 billion, and 20 billion parameter models.

  • GPT-Neo (125M, 1.3B, 2.7B):[24] released in March 2021, it was the largest open-source GPT-3-style language model in the world at the time of release.
  • GPT-J (6B): released in March 2021, it was the largest open-source GPT-3-style language model in the world at the time of release.
  • GPT-NeoX-20B, released on Feb 10, 2022.[9][10]

VQGAN-CLIP

An artificial intelligence art created with VQGAN-CLIP, a text-to-image model created by EleutherAI
An artificial intelligence art created with CLIP-Guided Diffusion, another text-to-image model created by Katherine Crowson of EleutherAI[25][26]

Following the release of DALL-E by OpenAI in January 2021, EleutherAI started working on text-to-image synthesis models. When OpenAI did not release DALL-E publicly, EleutherAI's Katherine Crowson and digital artist Ryan Murdock developed a technique for using CLIP (another model developed by OpenAI) to convert regular image generation models into text-to-image synthesis ones.[27][28][29][30] Building on ideas dating back to Google's DeepDream,[31] they found their first major success combining CLIP with another publicly available model called VQGAN and the resulting model is called VQGAN-CLIP.[32] Crowson released the technology by tweeting notebooks demonstrating the technique that people could run for free without any special equipment.[33][34][35]

See also

References

  1. ^ Leahy, Connor; Hallahan, Eric; Gao, Leo; Biderman, Stella (7 July 2021). "What A Long, Strange Trip It's Been: EleutherAI One Year Retrospective". Archived from the original on 29 August 2023. Retrieved 1 March 2023.
  2. ^ "Talk with Stella Biderman on The Pile, GPT-Neo and MTG". The Interference Podcast. 2 April 2021. Retrieved 26 March 2023.
  3. ^ a b c d Smith, Craig (21 March 2022). "EleutherAI: When OpenAI Isn't Open Enough". IEEE Spectrum. IEEE. Archived from the original on 29 August 2023. Retrieved 8 August 2023.
  4. ^ a b c Herschander, Sara (11 February 2025). "How Philanthropy Built, Lost, and Could Reclaim the A.I. Race". Chronicle of Philanthropy (Contributor). Retrieved 10 December 2025.
  5. ^ a b c d Wiggers, Kyle (2 March 2023). "Stability AI, Hugging Face and Canva back new AI research nonprofit". TechCrunch. Archived from the original on 29 August 2023. Retrieved 8 August 2023.
  6. ^ Leahy, Connor; Hallahan, Eric; Gao, Leo; Biderman, Stella (7 July 2021). "What A Long, Strange Trip It's Been: EleutherAI One Year Retrospective". EleutherAI Blog. Archived from the original on 29 August 2023. Retrieved 14 April 2023.
  7. ^ a b "GPT-3's free alternative GPT-Neo is something to be excited about". VentureBeat. 15 May 2021. Archived from the original on 9 March 2023. Retrieved 14 April 2023.
  8. ^ "GPT-J-6B: An Introduction to the Largest Open Source GPT Model". Forefront.ai. 14 October 2021. Archived from the original on 9 March 2023. Retrieved 1 March 2023.
  9. ^ a b "AI Weekly: DeepMind's AlphaCode, automatic age verification, and a new open language model". VentureBeat. 4 February 2022. Archived from the original on 2 June 2023. Retrieved 10 December 2025.
  10. ^ a b "Ukraine uses Clearview AI to identify Russian dead". The Register. 28 March 2022. Archived from the original on 8 September 2025. Retrieved 10 December 2025.
  11. ^ "The View from 30,000 Feet: Preface to the Second EleutherAI Retrospective". Eleuther AI Blog. 2 March 2023.
  12. ^ "AI Research Lab Launches Open Source Research Nonprofit". The NonProfit Times. 7 March 2023.
  13. ^ a b Gilbertson, Annie; Reisner, Alex (16 July 2024). "Apple, Nvidia, Anthropic Used Thousands of Swiped YouTube Videos to Train AI". WIRED. Retrieved 18 July 2024.
  14. ^ a b c Wiggers, Kyle (6 June 2025). "EleutherAI releases massive AI training dataset of licensed and open domain text". TechCrunch. Retrieved 10 December 2025.
  15. ^ Herschander, Sara (11 February 2025). "How Philanthropy Built, Lost, and Could Reclaim the A.I. Race". Chronicle of Philanthropy (Contributor). Retrieved 10 December 2025.
  16. ^ Knight, Will (29 March 2021). "This AI Can Generate Convincing Text—and Anyone Can Use It". Wired. ISSN 1059-1028. Retrieved 10 December 2025.
  17. ^ Wiggers, Kyle (11 October 2021). "Microsoft and Nvidia team up to train one of the world's largest language models". VentureBeat. Archived from the original on 27 March 2023. Retrieved 8 March 2023.
  18. ^ Khan, Mehtab; Hanna, Alex (2023). "The Subjects and Stages of AI Dataset Development: A Framework for Dataset Accountability". Ohio State Technology Law Journal. 19 (2): 171–256. hdl:1811/103549. SSRN 4217148.
  19. ^ Roth, Emma (20 August 2024). "Authors sue Anthropic for training AI using pirated books". The Verge. Retrieved 10 December 2025.
  20. ^ Knibbs, Kate (4 September 2023). "The Battle Over Books3 Could Change AI Forever". WIRED. Retrieved 13 October 2023.
  21. ^ Barr, Kyle (18 August 2023). "Anti-Piracy Group Takes Massive AI Training Dataset 'Books3′ Offline". Gizmodo. Retrieved 10 December 2025.
  22. ^ Deck, Andrew (7 January 2025). "Thousands of documentaries are fueling AI models built by Apple, Meta, and Nvidia". Nieman Lab. Archived from the original on 1 July 2025. Retrieved 10 December 2025.
  23. ^ Sato, Mia (16 July 2024). "Apple, Anthropic, and other companies used YouTube videos to train AI". The Verge. Retrieved 10 December 2025.
  24. ^ Andonian, Alex; Biderman, Stella; Black, Sid; Gali, Preetham; Gao, Leo; Hallahan, Eric; Levy-Kramer, Josh; Leahy, Connor; Nestler, Lucas; Parker, Kip; Pieler, Michael; Purohit, Shivanshu; Songz, Tri; Phil, Wang; Weinbach, Samuel (10 March 2023). GPT-NeoX: Large Scale Autoregressive Language Modeling in PyTorch (Preprint). doi:10.5281/zenodo.5879544.
  25. ^ "CLIP-Guided Diffusion". EleutherAI. Archived from the original on 29 August 2023. Retrieved 20 August 2023.
  26. ^ "CLIP Guided Diffusion HQ 256x256.ipynb - Colaboratory". Google Colab. Archived from the original on 29 August 2023. Retrieved 20 August 2023.
  27. ^ MIRANDA, LJ (8 August 2021). "The Illustrated VQGAN". ljvmiranda921.github.io. Archived from the original on 20 March 2023. Retrieved 8 March 2023.
  28. ^ "Inside The World of Uncanny AI Twitter Art". Nylon. 24 March 2022. Archived from the original on 29 August 2023. Retrieved 8 March 2023.
  29. ^ "This AI Turns Movie Text Descriptions Into Abstract Posters". Yahoo Life. 20 September 2021. Archived from the original on 27 December 2022. Retrieved 8 March 2023.
  30. ^ Quach, Katyanna (22 August 2021). "A man spent a year in jail on a murder charge involving disputed AI evidence. Now the case has been dropped". www.theregister.com. Archived from the original on 8 March 2023. Retrieved 8 March 2023.
  31. ^ "Alien Dreams: An Emerging Art Scene - ML@B Blog". Alien Dreams: An Emerging Art Scene - ML@B Blog. Archived from the original on 10 March 2023. Retrieved 8 March 2023.
  32. ^ "VQGAN-CLIP". EleutherAI. Archived from the original on 20 August 2023. Retrieved 20 August 2023.
  33. ^ "We asked an AI tool to 'paint' images of Australia. Critics say they're good enough to sell". ABC News. 14 July 2021. Archived from the original on 7 March 2023. Retrieved 8 March 2023 – via www.abc.net.au.
  34. ^ Nataraj, Poornima (28 February 2022). "Online tools to create mind-blowing AI art". Analytics India Magazine. Archived from the original on 8 February 2023. Retrieved 8 March 2023.
  35. ^ "Meet the Woman Making Viral Portraits of Mental Health on TikTok". Vice.com. 30 November 2021. Archived from the original on 11 May 2023. Retrieved 8 March 2023.
Wikious
Boobota
Sagapedia