Could ChatGPT get an engineering degree? Evaluating higher education vulnerability to AI assistants

/ Authors

Beatriz Borges, Negar Foroutan, Deniz Bayazit, A. Sotnikova, Syrielle Montariol, Tanya Nazaretzky, Mohammadreza Banaei, Alireza Sakhaeirad, Philippe Servant, Seyed Parsa Neshaei

and 80 more authors

Jibril Frej, Angelika Romanou, Gail Weiss, Sepideh Mamooler, Zeming Chen, Simin Fan, Silin Gao, Mete Ismayilzada, Debjit Paul, Alexandre Schopfer, Andrej Janchevski, Anja Tiede, Clarence Linden, E. Troiani, Francesco Salvi, Freya Behrens, Giacomo Orsi, Giovanni Piccioli, Hadrien Sevel, Louis Coulon, Manuela Pineros-Rodriguez, Marin Bonnassies, Pierre Hellich, Puck van Gerwen, Sankalp Gambhir, Solal Pirelli, Thomas Blanchard, Timoth'ee Callens, Toni Abi Aoun, Yannick Calvino Alonso, Yuri Cho, A. Chiappa, Antonio Sclocchi, 'Etienne Bruno, Florian Hofhammer, Gabriel Pescia, Geovani Rizk, L. Dadi, Lucas Stoffl, Manoel Horta Ribeiro, Matthieu Bovel, Yueyang Pan, Aleksandra Radenovic, Alexandre Alahi, Alexander Mathis, Anne-Florence Bitbol, Boi Faltings, C'ecile H'ebert, D. Tuia, Franccois Mar'echal, George Candea, Giuseppe Carleo, Jean-Cédric Chappelier, Nicolas Flammarion, Jean-Marie Furbringer, Jean-Philippe Pellet, Karl Aberer, Lenka Zdeborov'a, Marcel Salath'e, Martin Jaggi, M. Rajman, Mathias Payer, M. Wyart, Michael Gastpar, Michele Ceriotti, O. Svensson, O. L'eveque, P. Ienne, R. Guerraoui, Robert West, Sanidhya Kashyap, V. Piazza, V. Simanis, Viktor Kuncak, V. Cevher, Philipp Schwaller, S. Friedli, Patrick Jermann, Tanja Kaser, Antoine Bosselut

/ Abstract

Significance Universities primarily evaluate student learning through various course assessments. Our study demonstrates that AI assistants, such as ChatGPT, can answer at least 65.8% of examination questions correctly across 50 diverse courses in the technical and natural sciences. Our analysis demonstrates that these capabilities render many degree programs (and their teaching objectives) vulnerable to potential misuse of these systems. These findings call for attention to assessment design to mitigate the possibility that AI assistants could divert students from acquiring the knowledge and critical thinking skills that university programs are meant to instill.

Journal: Proceedings of the National Academy of Sciences of the United States of America

DOI: 10.1073/pnas.2414955121