The Abstraction and Reasoning Corpus: Formalizing the Group-Action Learning Problem

cover
11 Mar 2024

This paper is available on arxiv under CC 4.0 license.

Authors:

(1) Mattia Atzeni, EPFL, Switzerland and [email protected];

(2) Mrinmaya Sachan, ETH Zurich, Switzerland;

(3) Andreas Loukas, Prescient Design, Switzerland.

Formalizing the Group-Action Learning Problem

To build intuition on the kind of basic priors that we aim to infuse in neural networks, Figure 1 shows a task borrowed from ARC (Chollet, 2019). The task entails learning to fill out the yellow patches in the leftmost image (input) so that the resulting image satisfies a 90◦ rotation symmetry. The learner is given only a small set of input-output pairs: the ARC tasks have 3.3 training examples on average. Though the task is challenging for a general neural network (due to the small number of examples), it becomes easier under the prior knowledge of discrete two-dimensional point groups, one of which is the cyclic group of 4-fold rotations C4. Under this prior, it can be solved by the composition of a group action (rotating each image x by some g ∈ C4) and a shallow neural network with a non-linear activation (mapping yellow to zero and taking a pixel-wise max).

More formally, we are interested in helping neural networks learn lattice transformations in a sample efficient manner by infusing knowledge priors in the model. Motivated by ARC, we focus on learning geometric transformations that belong to the symmetry group of a lattice. This pertains to the more general problem of learning group actions given the input and the output of the transformation (group-action learning).

Concretely, we consider input-output transformations involving a group element g taken from some known group G that can be expressed under the general formulation:

Intuitively speaking, whereas in group-action learning one aims to learn functions that involve specific (and datadependent) transformations of our data by actions of the group, in in/equivariant learning the goal is to build models that are oblivious to such actions in a systematic manner.