This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Mattia Atzeni, EPFL, Switzerland and [email protected];
(2) Mrinmaya Sachan, ETH Zurich, Switzerland;
(3) Andreas Loukas, Prescient Design, Switzerland.
Table of Links
- Abstract & Introduction
- Formalizing the Group-Action Learning Problem
- Attention Masks for Core Geometry Priors
- The LATFORMER Architecture
- Experiments
- Related Work
- Conclusion & References
- A. Additional Details on the Model
- B. Additional Experiments and Details on the Experimental Setup
- C. Limitations and Future Work
- D. Deferred Proofs
Formalizing the Group-Action Learning Problem
To build intuition on the kind of basic priors that we aim to infuse in neural networks, Figure 1 shows a task borrowed from ARC (Chollet, 2019). The task entails learning to fill out the yellow patches in the leftmost image (input) so that the resulting image satisfies a 90◦ rotation symmetry. The learner is given only a small set of input-output pairs: the ARC tasks have 3.3 training examples on average. Though the task is challenging for a general neural network (due to the small number of examples), it becomes easier under the prior knowledge of discrete two-dimensional point groups, one of which is the cyclic group of 4-fold rotations C4. Under this prior, it can be solved by the composition of a group action (rotating each image x by some g ∈ C4) and a shallow neural network with a non-linear activation (mapping yellow to zero and taking a pixel-wise max).
More formally, we are interested in helping neural networks learn lattice transformations in a sample efficient manner by infusing knowledge priors in the model. Motivated by ARC, we focus on learning geometric transformations that belong to the symmetry group of a lattice. This pertains to the more general problem of learning group actions given the input and the output of the transformation (group-action learning).
Concretely, we consider input-output transformations involving a group element g taken from some known group G that can be expressed under the general formulation:

Intuitively speaking, whereas in group-action learning one aims to learn functions that involve specific (and datadependent) transformations of our data by actions of the group, in in/equivariant learning the goal is to build models that are oblivious to such actions in a systematic manner.
