Transformer models are gauge invariant: A mathematical connection between AI and particle physics

In particle physics, the fundamental forces are subject to symmetries called gauge invariance. It is a redundancy in the mathematical description of any physical system. In this article I will demonstrate that the transformer architecture exhibits the same properties, and show that the default repre...

Full description

Saved in:
Bibliographic Details
Main Author van Nierop, Leo
Format Journal Article
LanguageEnglish
Published 19.12.2024
Subjects
Computer Science - Learning
Physics - High Energy Physics - Theory
Online AccessGet full text

Cover

More Information
Summary:In particle physics, the fundamental forces are subject to symmetries called gauge invariance. It is a redundancy in the mathematical description of any physical system. In this article I will demonstrate that the transformer architecture exhibits the same properties, and show that the default representation of transformers has partially, but not fully removed the gauge invariance.
DOI:10.48550/arxiv.2412.14543