Training Transformers Together
Borzunov, Alexander, Ryabinin, Max, Dettmers, Tim, Lhoest, Quentin, Saulnier, Lucile, Diskin, Michael, Jernite, Yacine, Wolf, Thomas
Year of Publication 07.07.2022
Year of Publication 07.07.2022
Get full text
Journal Article
OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents
Laurençon, Hugo, Saulnier, Lucile, Tronchon, Léo, Bekman, Stas, Singh, Amanpreet, Lozhkov, Anton, Wang, Thomas, Karamcheti, Siddharth, Rush, Alexander M, Kiela, Douwe, Cord, Matthieu, Sanh, Victor
Year of Publication 21.06.2023
Year of Publication 21.06.2023
Get full text
Journal Article
Mixtral of Experts
Jiang, Albert Q, Sablayrolles, Alexandre, Roux, Antoine, Mensch, Arthur, Savary, Blanche, Bamford, Chris, Chaplot, Devendra Singh, Casas, Diego de las, Hanna, Emma Bou, Bressand, Florian, Lengyel, Gianna, Bour, Guillaume, Lample, Guillaume, Lavaud, Lélio Renard, Saulnier, Lucile, Lachaux, Marie-Anne, Stock, Pierre, Subramanian, Sandeep, Yang, Sophia, Antoniak, Szymon, Scao, Teven Le, Gervet, Théophile, Lavril, Thibaut, Wang, Thomas, Lacroix, Timothée, Sayed, William El
Year of Publication 08.01.2024
Year of Publication 08.01.2024
Get full text
Journal Article
Pixtral 12B
Agrawal, Pravesh, Antoniak, Szymon, Hanna, Emma Bou, Bout, Baptiste, Chaplot, Devendra, Chudnovsky, Jessica, Costa, Diogo, De Monicault, Baudouin, Garg, Saurabh, Gervet, Theophile, Ghosh, Soham, Héliou, Amélie, Jacob, Paul, Jiang, Albert Q, Khandelwal, Kartik, Lacroix, Timothée, Lample, Guillaume, Casas, Diego Las, Lavril, Thibaut, Scao, Teven Le, Lo, Andy, Marshall, William, Martin, Louis, Mensch, Arthur, Muddireddy, Pavankumar, Nemychnikova, Valera, Pellat, Marie, Von Platen, Patrick, Raghuraman, Nikhil, Rozière, Baptiste, Sablayrolles, Alexandre, Saulnier, Lucile, Sauvestre, Romain, Shang, Wendy, Soletskyi, Roman, Stewart, Lawrence, Stock, Pierre, Studnia, Joachim, Subramanian, Sandeep, Vaze, Sagar, Wang, Thomas, Yang, Sophia
Year of Publication 09.10.2024
Year of Publication 09.10.2024
Get full text
Journal Article
Mistral 7B
Jiang, Albert Q, Sablayrolles, Alexandre, Mensch, Arthur, Bamford, Chris, Chaplot, Devendra Singh, Casas, Diego de las, Bressand, Florian, Lengyel, Gianna, Lample, Guillaume, Saulnier, Lucile, Lavaud, Lélio Renard, Lachaux, Marie-Anne, Stock, Pierre, Scao, Teven Le, Lavril, Thibaut, Wang, Thomas, Lacroix, Timothée, Sayed, William El
Year of Publication 10.10.2023
Year of Publication 10.10.2023
Get full text
Journal Article
What Language Model to Train if You Have One Million GPU Hours?
Scao, Teven Le, Wang, Thomas, Hesslow, Daniel, Saulnier, Lucile, Bekman, Stas, Bari, M Saiful, Biderman, Stella, Elsahar, Hady, Muennighoff, Niklas, Phang, Jason, Press, Ofir, Raffel, Colin, Sanh, Victor, Shen, Sheng, Sutawika, Lintang, Tae, Jaesung, Yong, Zheng Xin, Launay, Julien, Beltagy, Iz
Year of Publication 27.10.2022
Year of Publication 27.10.2022
Get full text
Journal Article
Distributed Deep Learning in Open Collaborations
Diskin, Michael, Bukhtiyarov, Alexey, Ryabinin, Max, Saulnier, Lucile, Lhoest, Quentin, Sinitsin, Anton, Popov, Dmitry, Pyrkin, Dmitry, Kashirin, Maxim, Borzunov, Alexander, del Moral, Albert Villanova, Mazur, Denis, Kobelev, Ilia, Jernite, Yacine, Wolf, Thomas, Pekhimenko, Gennady
Year of Publication 18.06.2021
Year of Publication 18.06.2021
Get full text
Journal Article
Training Transformers Together
Borzunov, Alexander, Ryabinin, Max, Dettmers, Tim, Lhoest, Quentin, Saulnier, Lucile, Diskin, Michael, Jernite, Yacine, Wolf, Thomas
Published in arXiv.org (07.07.2022)
Get full text
Published in arXiv.org (07.07.2022)
Paper
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Laurençon, Hugo, Saulnier, Lucile, Wang, Thomas, Akiki, Christopher, del Moral, Albert Villanova, Scao, Teven Le, Von Werra, Leandro, Mou, Chenghao, Ponferrada, Eduardo González, Nguyen, Huu, Frohberg, Jörg, Šaško, Mario, Lhoest, Quentin, McMillan-Major, Angelina, Dupont, Gerard, Biderman, Stella, Rogers, Anna, allal, Loubna Ben, De Toni, Francesco, Pistilli, Giada, Nguyen, Olivier, Nikpoor, Somaieh, Masoud, Maraim, Colombo, Pierre, de la Rosa, Javier, Villegas, Paulo, Thrush, Tristan, Longpre, Shayne, Nagel, Sebastian, Weber, Leon, Muñoz, Manuel, Zhu, Jian, Van Strien, Daniel, Alyafeai, Zaid, Almubarak, Khalid, Vu, Minh Chien, Gonzalez-Dios, Itziar, Soroa, Aitor, Lo, Kyle, Dey, Manan, Suarez, Pedro Ortiz, Gokaslan, Aaron, Bose, Shamik, Adelani, David, Phan, Long, Tran, Hieu, Yu, Ian, Pai, Suhas, Chim, Jenny, Lepercq, Violette, Ilic, Suzana, Mitchell, Margaret, Luccioni, Sasha Alexandra, Jernite, Yacine
Year of Publication 07.03.2023
Year of Publication 07.03.2023
Get full text
Journal Article
OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents
Laurençon, Hugo, Saulnier, Lucile, Tronchon, Léo, Bekman, Stas, Singh, Amanpreet, Lozhkov, Anton, Wang, Thomas, Karamcheti, Siddharth, Rush, Alexander M, Kiela, Douwe, Cord, Matthieu, Sanh, Victor
Published in arXiv.org (21.08.2023)
Get full text
Published in arXiv.org (21.08.2023)
Paper
Pixtral 12B
Agrawal, Pravesh, Antoniak, Szymon, Emma Bou Hanna, Bout, Baptiste, Chaplot, Devendra, Chudnovsky, Jessica, Costa, Diogo, Baudouin De Monicault, Garg, Saurabh, Gervet, Theophile, Ghosh, Soham, Héliou, Amélie, Jacob, Paul, Jiang, Albert Q, Khandelwal, Kartik, Lacroix, Timothée, Lample, Guillaume, Las Casas, Diego, Lavril, Thibaut, Teven Le Scao, Lo, Andy, Marshall, William, Martin, Louis, Mensch, Arthur, Muddireddy, Pavankumar, Valera Nemychnikova, Pellat, Marie, Patrick Von Platen, Raghuraman, Nikhil, Rozière, Baptiste, Sablayrolles, Alexandre, Saulnier, Lucile, Sauvestre, Romain, Shang, Wendy, Soletskyi, Roman, Stewart, Lawrence, Stock, Pierre, Studnia, Joachim, Subramanian, Sandeep, Vaze, Sagar, Wang, Thomas, Yang, Sophia
Published in arXiv.org (10.10.2024)
Get full text
Published in arXiv.org (10.10.2024)
Paper
What Language Model to Train if You Have One Million GPU Hours?
Teven Le Scao, Wang, Thomas, Hesslow, Daniel, Saulnier, Lucile, Bekman, Stas, M Saiful Bari, Biderman, Stella, Hady Elsahar, Muennighoff, Niklas, Phang, Jason, Press, Ofir, Raffel, Colin, Sanh, Victor, Shen, Sheng, Sutawika, Lintang, Tae, Jaesung, Zheng, Xin Yong, Launay, Julien, Beltagy, Iz
Published in arXiv.org (08.11.2022)
Get full text
Published in arXiv.org (08.11.2022)
Paper
Mistral 7B
Jiang, Albert Q, Sablayrolles, Alexandre, Mensch, Arthur, Bamford, Chris, Devendra Singh Chaplot, de las Casas, Diego, Bressand, Florian, Lengyel, Gianna, Lample, Guillaume, Saulnier, Lucile, Lélio Renard Lavaud, Marie-Anne Lachaux, Stock, Pierre, Teven Le Scao, Lavril, Thibaut, Wang, Thomas, Lacroix, Timothée, William El Sayed
Published in arXiv.org (10.10.2023)
Get full text
Published in arXiv.org (10.10.2023)
Paper
Distributed Deep Learning in Open Collaborations
Diskin, Michael, Bukhtiyarov, Alexey, Ryabinin, Max, Saulnier, Lucile, Lhoest, Quentin, Sinitsin, Anton, Popov, Dmitry, Pyrkin, Dmitry, Kashirin, Maxim, Borzunov, Alexander, Albert Villanova del Moral, Mazur, Denis, Kobelev, Ilia, Jernite, Yacine, Wolf, Thomas, Pekhimenko, Gennady
Published in arXiv.org (08.11.2021)
Get full text
Published in arXiv.org (08.11.2021)
Paper
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Laurençon, Hugo, Saulnier, Lucile, Wang, Thomas, Akiki, Christopher, Albert Villanova del Moral, Teven Le Scao, Leandro Von Werra, Mou, Chenghao, Eduardo González Ponferrada, Nguyen, Huu, Frohberg, Jörg, Šaško, Mario, Lhoest, Quentin, McMillan-Major, Angelina, Dupont, Gerard, Biderman, Stella, Rogers, Anna, Loubna Ben allal, De Toni, Francesco, Pistilli, Giada, Nguyen, Olivier, Nikpoor, Somaieh, Maraim Masoud, Colombo, Pierre, de la Rosa, Javier, Villegas, Paulo, Thrush, Tristan, Longpre, Shayne, Nagel, Sebastian, Weber, Leon, Muñoz, Manuel, Zhu, Jian, Daniel Van Strien, Alyafeai, Zaid, Almubarak, Khalid, Minh Chien Vu, Gonzalez-Dios, Itziar, Soroa, Aitor, Lo, Kyle, Dey, Manan, Pedro Ortiz Suarez, Gokaslan, Aaron, Bose, Shamik, Adelani, David, Long, Phan, Tran, Hieu, Yu, Ian, Pai, Suhas, Chim, Jenny, Lepercq, Violette, Ilic, Suzana, Mitchell, Margaret, Luccioni, Sasha Alexandra, Jernite, Yacine
Published in arXiv.org (07.03.2023)
Get full text
Published in arXiv.org (07.03.2023)
Paper