Neurogeometry of perception: isotropic and anisotropic aspects

TL;DR

This paper explores the geometrical properties of the visual cortex, highlighting anisotropic edge co-occurrence distributions, and introduces a new non-isotropic cortical connectivity model that explains oblique visual phenomena.

Contribution

It presents a novel non-isotropic cortical connectivity model based on edge co-occurrence histograms, advancing understanding of visual cortex geometry and perception.

Findings

Histograms of edges are biased towards horizontal and vertical directions.

The new model explains oblique phenomena consistent with experimental data.

Geometrical properties of cortical connectivity influence perception patterns.

Abstract

In this paper we first recall the definition of geometical model of the visual cortex, focusing in particular on the geometrical properties of horizontal cortical connectivity. Then we recognize that histograms of edges - co-occurrences are not isotropic distributed, and are strongly biased in horizontal and vertical directions of the stimulus. Finally we introduce a new model of non isotropic cortical connectivity modeled on the histogram of edges - co-occurrences. Using this kernel we are able to justify oblique phenomena comparable with experimental findings.

Equations

$$O_F(x_1,x_2,f)=\int I({\xi }_1,{\xi }_2){\psi }_{(x_1,x_2,f)}({\xi }_1,{\xi }_2)d{\xi }_{1}d{\xi }_2.$$

$$a(x_{1},x_{2},f)=\sigma\Bigg{(}\int K_{F}((x_{1},x_{2},f,\bar{x}_{1},\bar{x}_{2},\bar{f})\Big{(}\frac{a(\bar{x}_{1},\bar{x}_{2},\bar{f})+O_{F}(\bar{x}_{1},\bar{x}_{2},\bar{f})}{2}\Big{)}d\bar{x}_{1}d\bar{x}_{2}d\bar{f}\Bigg{)},$$

$${\Omega }_F=\{(x_1,x_2,f):|O_F(x_1,x_2,f)|\ge h\}.$$

$$(K_F\ast v_i)(x_1,x_2,f)=\int K_F((x_1,x_2,f),({\bar{x}}_1,{\bar{x}}_2,\bar{f}))v_i({\bar{x}}_1,{\bar{x}}_2,\bar{f})d{\bar{x}}_{1}d{\bar{x}}_{2}d\bar{f}$$

$$K_F\ast v_i={\lambda }_i{v}_i$$

$$|O_F(x_1,x_2,f_I(x_1,x_2))|=\underset{f}{\max }|O_F(x_1,x_2,f)|.$$

$$K_{\Pi }(({\xi }_1,{\xi }_2),(x_1,x_2))=K_F(({\xi }_1,{\xi }_2,f_I({\xi }_1,{\xi }_2)),(x_1,x_2,f_I(x_1,x_2))).$$

$$u(\xi_{1},\xi_{2})=\sigma\Bigg{(}\int K_{\Pi}((\xi_{1},\xi_{2}),(x_{1},x_{2}))O_{F}(x_{1},x_{2},f_{I}(x_{1},x_{2}))dx_{1}dx_{2}\Bigg{)}.$$

$${\psi }_{0,LGN}({\xi }_1,{\xi }_2)=\Delta G({\xi }_1,{\xi }_2)$$

$${\psi }_{x_1,x_2,LGN}({\xi }_1,{\xi }_2)={\psi }_{0,LGN}(x_1-{\xi }_1,x_2-{\xi }_2).$$

$$O_{LGN}(x_1,x_2)=\int I({\xi }_1,{\xi }_2){\psi }_{x_1,x_2,LGN}({\xi }_1,{\xi }_2)d{\xi }_{1}d{\xi }_2=$$

$$\int I({\xi }_1,{\xi }_2){\psi }_{0,LGN}(x_1-{\xi }_1,x_2-{\xi }_2)d{\xi }_{1}d{\xi }_2=(\Delta G\ast I)(x_1,x_2)\approx \Delta I(x_1,x_2).$$

$$K_{LGN}(x_1,x_2)=\log (x_1^2+x_2^2).$$

$$u({\xi }_1,{\xi }_2)=\int K_{LGN}({\xi }_1-x_1,{\xi }_2-x_2)O_{LGN}(x_1,x_2)d{x}_{1}d{x}_2$$

$$L_1(u)=\int |\nabla (I-u){|}^{2}d{x}_{1}d{x}_2.$$

$$b=(I+u)/2.$$

$${\psi }_{0,V1}({\xi }_1,{\xi }_2)=e^{-i{\xi }_2-({\xi }_1^2+{\xi }_2^2)}.$$

$${\psi }_{x_1,x_2,\theta ,V1}({\xi }_1,{\xi }_2)={\psi }_{0,V1}(T_{x_1,x_2}{R}_{\theta })({\xi }_1,{\xi }_2))=$$

$$=e^{-i\Big{(}(\xi_{1}-x_{1})\sin(\theta)-(\xi_{2}-x_{2})\cos(\theta)\Big{)}-((\xi_{1}-x_{1})^{2}+(\xi_{2}-x_{2})^{2})}.$$

$$O_{V1}(x_1,x_2,\theta )$$

$$\omega =\sin (\theta )d{x}_1-\cos (\theta )d{x}_2$$

$$X_1=\cos (\theta ){\partial }_x+\sin (\theta ){\partial }_y,X_2={\partial }_{\theta }.$$

$$X_3=[X_1,X_2]=-\sin (\theta ){\partial }_x+\cos (\theta ){\partial }_y.$$

$$v=a_1{X}_1+a_2{X}_2\text{ we will define norm of }v\text{ the quantity }||v||=\sqrt{a_1^2+a_2^2}.$$

$$FPu=X_{1}u+X_2^{2}u.$$

$$K_{V1}(x_1,x_2,\theta ,{\xi }_1,{\xi }_2,\eta )={\Gamma }_{V1}(x_1,x_2,\theta ,{\xi }_1,{\xi }_2,\eta )+{\Gamma }_{V1}({\xi }_1,{\xi }_2,\eta ,x_1,x_2,\theta ).$$

$$K_{V1}\ast u=\lambda u$$

$$|O_I(x_1,x_2,{\theta }_I(x_1,x_2))|=\underset{\theta }{\max }|O_I(x_1,x_2,\theta )|.$$

$$A=\nabla I\approx |O_I(x_1,x_2,{\theta }_I(x_1,x_2))|(-\sin ({\theta }_I(x_1,x_2)),\cos ({\theta }_I(x_1,x_2))).$$

$$\frac{A_1}{\sqrt{A_1^2+A_2^2}}=-\sin ({\theta }_I)\frac{A_2}{\sqrt{A_1^2+A_2^2}}=\cos ({\theta }_I)$$

Full text

\floatsetup

[figure]capposition=bottom

Neurogeometry of perception:

isotropic and anisotropic aspects

G. Citti, A. Sarti 111The authors have been supported by Horizon 2020 Project ref. 777822: GHAIA

1 Introduction

The first attempts to formalize rules of perception go back to the Gestalt psychologists, as for example Wertheimer [83], Kohler [50], Koffka [49]. They formulated some geometric laws accounting for perceptual phenomena, taking into account different features like position but also brightness, orientation and scale. More precisely these laws depend on normalized differences of position, brightness, orientation or scale between the target image and the background. As an example we recall that two circles, with equal luminance, (so that they reflect physically the same amount of light) are perceived of different grey level, if put on different backgrounds: the one with darker background is perceived as brighter than the one with bright background (see figure 1). This phenomenon, also called simultaneous contrast, was known by Goethe, and then it has been deeply studied. A first explanation is that the brightness is related to detection of edges of the region, followed by a filling-in mechanism ([39], [68], [75]). In particular the retinex model was inspired by similar principles [12, 58, 44, 48, 60]. These aspects have to be considered as relatively local properties of perception. A number of computational models have been published, taking into account other aspects of perception as for example distal aspects, or illumination (see for example [5], [11], and [79]).

Here we will consider an other aspect of perception: its globality, since emergence of perceptual units depends on the visual stimulus as a whole. Indeed visual scenes are perceived as constituted by a finite number of figures. The most salient configuration pops up from the ground and becomes a figure (see Merleau-Ponty [59]). Often this happens even in absence of gradients or boundaries in the visual stimulus (see for example the classical example of the Kanizsa triangle [46]). A long history of phenomenology of perception, starting from Brentano, to Husserl until Merleau-Ponty has considered the emergence of foreground and background at the centre of perceptual experience and resulting as an articulation of a global perceptual field.

A number of researchers studied principles of psychology of form in this perspective not only qualitatively but also quantitatively. We recall the first results of Grossberg and Mingolla in [38], the notion of vision field proposed by Parent and Zucker in [63] and the theory of object perception due to Kellman and Shipley in [47] [77, 78]. In the same years Field, Hayes and Hess [34] introduced the notion of association fields, via the following experiment. figure 2(a) it composed by random Gabor patches and a perceptual units obtained via aligned patches, which is also visualized in figure 2(b). Through a series of similar experiments, Field, Hayes and Hess constructed an association field, which describes the complete set of possible subjective boundaries starting from a fixed point. See figure 2(c).

We take the point of view that the geometry of visual perception is induced by the functional geometry of the visual cortex. The first results in this direction have been achieved by Hubel and Wiesel [43] who provided experimental evidence of the presence in the primary cortex of families of cells sensitive to different orientations organized in the so called hypercolumnar module. After that, results of Bosking et al. in [13] and Fregnac Shulz in [36] provided experimental evidence for the organisation of many other features already studied in psychology of perception, like brightness, curvature and scale. In addition, the structure of horizontal connectivity between cells studied by Bosking is a good candidate as a neural correlate for perceptual association fields. This allows to introduce a relation between geometry of the cortex and geometry of vision. Mathematical models in this direction have been proposed by Hoffman in [41], Mumford in [61], Williams and Jacobs in [85] and August Zucker in [6]. They modelled the analogous of the association fields with Fokker-Planck equations. Petitot and Tondut (in [65]) introduced a model of the functional architecture of V1, and described the propagation by a constrained Lagrangian operator, giving a possible explanation of Kanizsa subjective boundaries in terms of geodesics. With these results it became clear the central role of geometry in the description of the organization of the cortex and of its correlate on the visual plane. Citti and Sarti in [21] proposed a model of the functional architecture of V1 as a Lie group of symmetry, with a sub-Riemannian geometry, showing the strict relation between geometric integral curves, association fields, and metric cortical properties. A large literature has been provided on models in the same subriemannian space both for boundary completion [7, 14, 67, 29, 30, 28] and for other perceptual phenomena [74, 72, 35]. This approach has been called neurogeometry (or neuromathematics). For a more complete bibliography and application fields of neuromathematics the reader can refer for example to [22].

These models assume by simplicity that the perception is orientation invariant, allowing a representation in the Lie group of rotation and translation. However it is a classical finding that human subjects perform best on a spatial acuity test when the visual targets are oriented horizontally or vertically. This phenomenon first described by Mach in [55], has been called the oblique effect by Appelle [4]. It applies to different phenomena such as orientation selectivity (Andrews [2]; Campbell and Kulikowski [20]; Orban et al.[62]), orientation discrimination (Bouma and Andriessen [15]; Caelli et al. [19], Regan and Price [66]; Westheimer and Beard [84]), grouping (Beck [8, 9, 10]), and geometric illusions (Green and Hoyle [37] Leibowitz and Toffey [53], Wallace [81]). In particular in Bross et al. [18] the oblique effect is studied on a Kanizsa figure. A Kanizsa square and a Kanizsa diamond are considered. If no edge misalignment is present both the square and the diamond are correctly reconstructed. However with a 6-degree misalignment the square is not perceived, while the diamond is perceived. For higher misalignment neither the square not the diamond are perceived (see figure 3).

Neural correlates of this phenomenon have been provided by Hubel and Wiesel [42], who proved that the distribution of oriented cells who respond to lines is not uniform. Nowadays most studies agree that the oblique effect happens in the visual cortex [57] [56] but the mechanism is not totally clear. However it seems that there are at least two aspects to be considered: the functionality of the visual system (see [54] and [1]), and learning from the environment (e.g. with statistics of natural scenes - see [32] and [40]).

In this paper we first recall the definition of neurogeometical model for a general set of features, focusing in particular on the geometrical properties of horizontal cortical connectivity and showing how they can be considered as a neural correlate of a geometry of the visual plane. In this first geometrical model the geometry of the cortex is expressed as a sub-Riemannian geometry in the Lie group of symmetries of the set of features.

Here, using data taken from [69] we recognize that histograms of edges - co-occurrences are not isotropic distributed, and are strongly biased in horizontal and vertical directions of the stimulus. Finally we introduce a new model of non isotropic cortical connectivity modeled on the histogram of edges - co-occurrences. Using this kernel in the geometrical cortical model previously described, we are able to justify oblique phenomena comparable with the experimental findings of [15] and [18].

2 A General Approach to the Geometry of Perception

Citti and Sarti developed in a few papers [21, 70, 73, 23], a theory of invariant perception in Lie groups, taking into account different features: brightness orientation, scale, curvature, movement. Each of these features have been characterized by a different Lie group, but the different techniques can be presented from an unitary point of view, which we recall here.

2.1 Families of cells and their functionality

The primary visual cortex is the first part of the brain processing the visual signal coming from the retina. The receptive field (RF) of a cortical neuron is the portion of the retina which the neuron reacts to, and the receptive profile (RP) $\psi({\xi})$ is the function that models the activation of a cortical neuron when a stimulus is applied to a point ${\xi}=(\xi_{1},\xi_{2})$ of the retinal plane.

Due to the retinotopic structure, there is an log-polar map between retina and cortical space in V1, which we will discard in first approximation. In addition the hypercolumnar structure organizes the cortical cells of V1 in columns corresponding to different features. As a results we will identify cells in the cortex by means of three parameters $(x_{1},x_{2},f)$, where $(x_{1},x_{2})$ is the position of the point, and $f$ a vector of extracted features. We will denote $F$ the set of features, and consequently the cortical space will be identified with $R^{2}\times F$. In the presence of a visual stimulus the whole hypercolumn fires, giving rise to an output

[TABLE]

Note that the output is a high dimensional function, defined on the cortical space. We denoted it $O_{F}$ to underline the dependence on the family of filters $F$. It is clear that the same image, filtered with a different family of cells, produces a different output.

2.2 Cortical Connectivity and the Geometry of the Cortical Space

The output of a family of cells is propagated in the cortical space $R^{2}\times F$ via the lateral connectivity. We will represent connectivity as a geometric kernel $K_{F}$, acting in the cortical space $R^{2}\times F$ on $O_{F}$ which is now considered a feedforward input to the overall activity. The shape of this kernel will be different in different families of cells and will be compatible with their invariance properties. We will see that there is a strict relation between the structure of set $R^{2}\times F$ of receptive profiles, their functionality and the direction of propagation of cortical activity codified in the kernels $K_{F}$. This allows to introduce a geometry of the considered family of cells, where the distance is not the usual Euclidean one, but it is induced by the strength of connectivity: the distance between couple of points will be considered a decreasing function of the value of the connectivity strength.

Different equations associated to these kernels have been used to describe the propagation of neural activity in the cortical space $R^{2}\times F$. In particular, following an approach first proposed by Ermentraut-Cowan in [31] and Bressloff and Cowan in [17], we consider a mean field equation representing the cortical activity $a(x_{1},x_{2},f)$. In the stationary case the equation becomes

[TABLE]

where $\sigma$ is a sigmoidal functional. Our intent in the following is to show that the RP functions $\psi$ as well as the connectivity kernels $K_{F}$ can be derived from theoretical geometrical considerations.

2.3 Spectral analysis and perceptual units

Bressloff, Cowan, Golubitsky and Thomas in [16], showed that in absence of a visual input $O_{F}$ but in presence of drugs excitation, the eigenvectors of this functional on the whole cortical space can describe visual hallucination phenomena.

In presence of a visual input, the propagation of the signal through horizontal connectivity and its eigenvectors have been studied by Sarti and Citti in [73]. Since eigenvectors are the emergent states of the cortical activity, they individuate the coherent perceptual units in the scene and allow to segment it. Every eigenvector corresponds to a different perceptual unit. Let $\Omega_{F}$ be the set where the feedforward input is bigger than a fixed threshold $h$:

[TABLE]

Calling $K_{F}*v$ the restriction to $\Omega_{F}$ of the linearization of the operator defined in (2) we get up to a change of variable the operator:

[TABLE]

The associated eigenvectors satisfy the equation:

[TABLE]

and correspond to visual units (see [73] for details). This result shows that perceptual units correspond to visual hallucination modulated by the stimulus. Eigenvectors of these operators are functions defined on $\Omega_{F}$ with real values. We will assign the meaning of a saliency index of the objects to the associated eigenvalues. The first eigenvalue will correspond to the most salient object in the image. In addition, the associated eigenvalue allows comparison between different scenes and images. This approach also extends on neural basis the model proposed for image processing by Perona Freeman, [64], Shi Malik, [76], Weiss [82], Coifman Lafon [25].

2.4 Projected geometry on the visual plane

The geometry of visual perception is simply the projection of the active cortical geometry on the visual plane. Even though it is not clear where this phenomenon is codified we will project on the retinal plane the geometry previously introduced in order to describe geometry of vision.

For every cortical point, the cortical activity, suitably normalized can be considered a probability density. Hence its maximum over the fibre $F$ can be considered the most probable value of $f$, and can be considered the feature identified by the system:

[TABLE]

Note that the function $f_{I}$ detect at every point the most salient value of the feature, and clearly depends on the properties of the visual stimulus $I$. This feature selection mechanism induces a geometry on the perceptual plane. The projection of the kernel $K_{F}$ on the perceptual plane will be a kernel $K_{\Pi}$ defined as

[TABLE]

Note that the kernel $K_{\Pi}$ is now depending on the function $I$. Hence it carries the metric induced on the retinal plane by the image $I$.

If we propagate the function $I$ with this kernel, we obtain an activity function on the retinal plane

[TABLE]

In the following we will discard the sigmoid $\sigma$ when the solution remains bounded from above and below.

2.5 The modular structure of the cortex

We can assume that different families of cells depending on different features act in sequence on the same visual stimulus. A family of cells will process the stimulus in a space $R^{2}\times F_{1}$. Then its output will become the input for the second family of cells, (depending for example on orientation) which will process the stimulus in a new space of features $R^{2}\times F_{2}$. In this way interaction between the two families of cells can be described with our model, even if they are selective for different features.

3 Isotropic models inherited by the geometry of the visual cortex

3.1 Brightness perception

We refer to the introduction and to [79] and the references therein for a review on the state of the art. In our model the visual signal is convolved with the RF of the retinal cells, which are modeled as a Laplacian of a Gaussian. After that we apply a distal mechanism induced by the non classical components of RF. Indeed in Lateral Geniculate Nucleus (LGN) e in V1 there is a strong near surround suppression mechanism due to the lateral connection and a far surround suppression mechanism due to feedback connectivity from higher cortices [3]. We postulate that the far surround suppression mechanism is able to generates brightness contextual effects and that it is at the origin of filling in. Indeed the simplest surround function with positive values at the origin and negative at infinity is a negative logaritmic fuction, which coincides with the fundamental solution of the Laplace operator. As a result the output of the mexical hat RPs is convolved with a logaritmic functioin, and provides the reconstruction of the signal up to an harmonic function, which takes into account a global light contrast effect.

3.1.1 The set of receptive profiles

Retinal ganglion cells and LGN cells, have a radially symmetric shape (see [27]), and are ubbsually modeled by Laplacians of Gaussians:

[TABLE]

where $G(\xi_{1},\xi_{2})=e^{-(\xi_{1}^{2}+\xi_{2}^{2})}$ is the Gaussian bell. We will consider by simplicity filters at a fixed scale (see figure 4). As a result there is a single receptive field over each retinal point $(\xi_{1},\xi_{2})$. In other words all filters of this family of cells can be obtained from a fixed one by translation:

[TABLE]

Formally the set of filters will be identified with a copy of the group $R^{2}$, with the standard sum.

In presence of a visual stimulus $I$, the retinotopic map performs a logarithmic deformation, which will be discarded here for simplicity. Then the output defined in (1) reduces to

[TABLE]

[TABLE]

3.1.2 Connectivity patterns and the Euclidean geometry of the space

The output of LGN cells is propagated via the lateral connectivity. Experimentally it has been showed that connectivity is isotropic. As a result both the filters and the connectivity has the same radial symmetry. This is typical of isotropic kernels of the Euclidean geometry, and it is compatible with the fact that the set of cells is simply defined on the $R^{2}$ plane. Hence we can choose as model of connectivity the fundamental solution of the Laplace operator in the isotropic Euclidean metric:

[TABLE]

According to (8) total activity contribution will be modelled as

[TABLE]

Note that we are identifying the set of filters with $R^{2}$, the geometry is the Euclidean one and the propagation is performed with the standard Laplacian on it. This does not means that the perceived image coincide in general with the visual input. Indeed $O_{LGN}(x_{1},x_{2})=\Delta I$, which implies that $u-I$ is an harmonic function. If we impose Neumann boundary conditions, the same harmonicity condition can be expressed as minimum of the functional

[TABLE]

Finally we propose to obtain the brightness of the perceived targets as a mean between the initial image and the activity $u$:

[TABLE]

Also note that the perception of the background is unchanged.

We apply this algorithm to the image in figure 1 left and the results are shown in figure 5. On the left it is represented the output $O_{LGN}$, which is a convolution with a Laplacian of a Gaussian. Then we compute the activity: due to the Neumann boundary condition the background has a constant color, and the circle on the right has gray level brighter than the one on the left. Finally, applying equation (10), we obtain the perceived gray level of the two target circles as a mean of the stimulus and the activity (figure 5 right).

3.2 Geometry of Orientation space

3.2.1 The family of simple cells as the Lie group of rotation and translation

Receptive profiles of simple cells sensitive to orientation have been modeled as Gabor filters by Daugman [26], Jones and Palmer [45]. A suitable choice of the mother function is

[TABLE]

The previously mentioned authors also showed that every simple cell is obtained from the mother one via rotation, translation. If we denote $T_{x_{1},x_{2}}$ a translation of a vector $(x_{1},x_{2})$ and $R_{\theta}$ a rotation of an angle $\theta$, their expression becomes:

[TABLE]

[TABLE]

In this way any simple cell receptive profile is identified by an element of the Lie group of rotation and translation $R^{2}\times S^{1}$ (see fig. 6 left and middle).

The output of this family of cells receptive profiles can be expressed by linear filtering of the stimulus $I$ according to equation (1):

[TABLE]

3.2.2 The subriemannian neurogeometry

It is not surprising that there is a strong relation between the set of cell RPs and their connectivity. Connectivity kernels have been measured by Bosking, who proved that they have an elongated shape ([13], see also figure 6 right), and that the strength of connectivity is maxima between cells with the same orientation. Due to this strong relation, we will define the metric on the space $R^{2}\times S^{1}$, starting from properties of the filters. We introduce 1- form modeled on the expression of the filters (11):

[TABLE]

and choose left invariant vector fields, which lie in its kernel:

[TABLE]

The plane generated by these vector fields will be called Horizontal tangent bundle $HT$. Note that we have introduced only two vector fields in a 3D space, but the orthogonal direction can be recovered as a commutator:

[TABLE]

This condition, called the Hörmander condition, allows to define a subriemannian structure, if we also choose a metric on $HT$. The metric is not unique and the simplest choice is the metric which makes the vector fields orthonormal. In other words for every vector

[TABLE]

We will see in the next section that this choice can justify many perceptual phenomena, but other choices could be considered.

3.2.3 Grouping and emergence of shapes

Differential operators can be defined in terms of the vector fields $X_{1}$ and $X_{2}$, which play the same role of a derivative in the Euclidean setting. In particular when studying grouping we can consider the Fokker-Planck operator

[TABLE]

Due to the Hörmander condition, it is known that this operator has a fundamental solution $\Gamma_{V1}$, which is strictly positive. However it is not symmetric in general, and in order to model horizontal connectivity we need to symmetrize it

[TABLE]

An isosurface of this kernel is depicted in figure 7.

To segment a stimulus into perceptual units we use then this kernel with the spectral technique introduced in (5). We can apply the eigenvalue equation to the operator associated to $K_{V1}$:

[TABLE]

A few results have been depicted in figure 8. The first image on the left is the Kanizsa square. Due to the strong orientation properties of the kernel, collinear elements tend to be grouped. Hence the L-junctions pops up as the first eigenvector (highlighted in red). This can be considered the key element at the basis of perception of the square (the second eigenvector, that is not visualized, corresponds to the 4 arcs of circle). Note that the technique is invariant with respect to rotation, and the a L-junctions in Kanizsa diamond are detected with the same accuracy (second image in figure 8). In the last image we show that not only inducers of classical geometrical object such as squares or diamonds are correctly detected, but also inducers of completely arbitrary shapes can emerge (third image in figure 8). Indeed the geometric process of grouping is based on the internal and global coherence of the elements in the image, not on a pre-existing list of possible shapes. As such it introduces a geometric intrinsic notion of shape based on the property of the cortex.

3.2.4 Completion in the orientation geometry

In order to project on the image plane, we first apply the equation (6) restricted to this setting:

[TABLE]

The cells are selective to the orientation since it can be proved that $\theta_{I}(x_{1},x_{2})$ is the direction of the level line of $I$ at the point $(x_{1},x_{2})$. As a consequence the gradient of $I$ can be approximated as follows

[TABLE]

Dividing by the norm of $A$ we obtain

[TABLE]

and the 2D projection of the vector field $X_{1}$ is expressed in terms of $A$ as

[TABLE]

The 2D projection of the Kernel defined in (7), becomes in this case

[TABLE]

This kernel, depicted in figure 9, can be considered a good model of the association fields introduced by Field Hayes, Hess, in [34].

Since the $K_{V1}$ expresses propagation in the direction $X_{1}$ and $X_{2}$, convolution with its projection $K_{\Pi}$ expresses propagation in the direction of the vector field $X_{1,A}$. Propagation along the kernel can be expressed as minimizer of the functional

[TABLE]

3.3 Combined contrast - orientation geometry

In this section we will study the join action of LGN cells and cells in V1, taking into account feedforward, horizontal, and feedback connectivity. We propose a complete Lagrangian, sum of three terms. The first two have already been introduced: the functional $L_{1}$ defined in (9) expresses the contrast variable considered as the analogous of a particle term in a gauge field functional and the functional $L_{2}$ defined in (14) which describes a strongly anisotropic process and it is performed with respect to a subriemannian metric. Finally we will introduce an interaction term coupling the two terms as in the usual Lagrangian field theories.

The last term describes the interaction between the particle $u(x,y)$ and the field $A(x,y)$.

[TABLE]

The resulting functional is then

[TABLE]

The Euler Lagrange equations of the functional (16) are obtained by variational calculus:

[TABLE]

In order to solve the system we first solve the particle equation, assuming that $A=0$. Then, a first estimate $u_{1}$ of $u$ is obtained as a solution of the first equation

[TABLE]

The function $u_{1}$ select boundaries (see figure 10(a)) Then we apply the grouping algorithm and find the inducers of the square (see figure 10(b)). With this estimated second member and the condition $A_{0}=0$ we find the solution of the equation for the vector $A$:

[TABLE]

The diverge of the vector solution is depicted in fig 10(c). Finally we recover the value of $u$ using as forcing term $\Delta I+div(A)$,

[TABLE]

applying to the function $u$ the mean as in (10) we obtain

[TABLE]

which is the last image represented in figure 10.

The same combined algorithm is also applied to the a Kanizsa diamond and to a non symmetric shape in figure 11.

Due to the fact that the model is invariant by rotation symmetry the completion of the Kanizsa square is completely equivalent to the completion of the Kanizsa diamond up to a global rotation. We will question in the following this rotation invariance based both on perceptual evidence and image statistics results.

3.4 The model applied to other features

The previous model can be extended to the description of families of cell sensitive to other features, as for example movement. This has been studied in [24], where grouping and completion has been achieved in a five-dimensional space, depending on position $(x_{1},x_{2})$, orientation, time and apparent velocity. We also refer to [35] where other classical illusion only dependent on orientation have been considered (Hering, Wundt, Ehrenstein, Zollner illusions). We expect that better results could be obtained with the algorithm we present here, able to combine the grouping effect with the orientation selectivity.

An other feature which could be considered is the disparity feature, which allows reconstruction a 3D image from 2 flat projections on the retina of the eyes. With our approach these aspects of perception of images have to be developed in the group $SE(3)$ of rigid motion of the 3D space. This phenomena is probably related to the distal properties of brightness perception. Indeed according to [79] a flat image exhibiting various reflectance distributions, observed under uniform illumination, can be interpreted as a scene with different level of illumination, giving the impression of a 3D object, and giving rise to illusions of perception of brightness.

4 Anysotropic aspects of the geometry induced by learned kernels

Studies of visual acuity in humans were among the first investigations to uncover preferences for vertical and horizontal stimuli. Emsley (192S) noticed acuity differences among subjects asked to resolve line gratings. Maximal acuity occurred when the gratings were in horizontal or vertical orientations. We will see that this anisotropy can be explained in terms of kernel learned by natural images.

4.1 Statistics of images

An intriguing hypothesis is that association fields induced by horizontal connectivity have been learned by the history of visual stimuli that a subject has perceived. In this case the specific connectivity pattern would result from some statistical property of natural images.

A specific study to assess the existence of this relation has been proposed in [69]. Research has been focused on the statistics of edges in natural images and particularly in the statistics of co-occurence of couples of edges taking into account its relative position and orientation. The statistics has been estimated analyzing a number of natural images from the image database: http://hlab.phys.rug.nl/imlib/index.html, which has been many times used in literature to compute natural image statistics It consists on 4000 high-quality gray scale digital images, 1536 $\times$ 1024 pixel and 12 bits in depth. Images have been preprocessed by linear filtering with a set of oriented edge detection kernels (Gabor filters) and performing non maximal suppression. A list of pixels corresponding to edges with their respective orientations has been obtained by thresholding and binarization. If we fix the orientations $\theta_{c}$ and $\theta_{p}$, a $2\times 2$ histogram can be computed counting how many edges have orientation $\theta_{c}$ at the central pixel and orientation $\theta_{p}$ at the position $(\Delta x,\Delta y)$. An example is depicted in figure 12, left. If we consider all possible orientations $\theta_{c}$ of the central pixel and $\theta_{p}$ of the other pixels, we obtain a four dimensional histogram $(\Delta x,\Delta y,\theta_{c},\theta_{p})$ (see fig. 12, right).

4.2 Anysotropic Connectivity kernels

In [69] a 3D histogram $(\Delta x,\Delta y,\Delta\theta)$ is obtained where the third coordinate is the relative orientation $\Delta\theta=\theta_{p}-\theta_{c}$. This kernel, invariant by rotation and translation was compared with the one obtained with the Lie group theory.

Here we keep now completely independent any orientation in order to study possible anisotropy of the statistic in the different directions.

We focus in particular on the histogram with the central orientation horizontal and the one with the central orientation diagonal. This means that we consider separately the first and the third row of the histogram.

We have selected two histograms each one constituted by 16 $128\times 128$ images. Hence each of them can be considered a $128\times 128\times 16$ array, discretization of a 3D kernel. We will denote the two histograms respectively associated with the horizontal or diagonal central orientation as follows:

[TABLE]

We visualized each of them in figure 13 by slices and figure 14 in 3D: it is clear that they are different, even though they have some similarity with the 3D kernel in figure 7. In particular if we assume that cortical connectivity is modulated by the geometry of the perceived images, then the connectivity kernel will be different for different directions.

4.3 Anisotropic association fields

We can project the kernels $K_{hor}$ and $K_{diag}$ on the 2D plane, in order to understand their properties. We obtain respectively

[TABLE]

The two kernels are depicted in figure 15. We can see that they have completely different properties. The horizontal one takes higher values, and has a longer action. Moving from the diagonal the values change very rapidly. Hence the kernel has a very clear ability to discriminate the horizontal direction with respect to any other one.

The diagonal one takes lower values, and has a shorter action, and does not change very much far from the diagonal. As a consequence the kernel assume comparable values for diagonal and slightly off diagonal segments.

4.4 Difference in horizontal versus diagonal orientation detection

Here we would like to related the geometry of the non isotropic association fields we have found and the difference in orientation detection for stimuli aligned in horizontal as compared to stimuli in oblique orientations.

We start with a few horizontal segments, which can be evaluated with the horizontal association field. Since the intensity of the kernel is very high, it takes the same values on aligned horizontal segment (see figure 16 above left), and keep the same value even if these are taken apart (see figure 16 above middle). On the other side the values change very rapidly while moving from the horizontal, hence the kernel takes values very different on non aligned segments, leading to a a very high discrimination (see figure 16 above right) in accordance with psycophysical results.

When we repeat the experiment on diagonal segments the situation is very different. The kernel tends to group aligned diagonal segments (see figure 16 bottom), but since the intensity of the kernel decreases rapidly with the distance, the kernel is not able to group distant segments, even if they are diagonal. Finally the values of the diagonal kernel change very slowly while moving from the diagonal, hence the kernel takes the same values on diagonal misaligned segments, leading to a low discrimination, in accordance with the so called oblique effect.

4.5 Oblique effect in the Kanizsa square

The oblique effect was studied in Kanizsa square with misaligned edges in [18]. The problem refers to the ability to perceive stimuli lying in the horizontal orientation with greater efficiency than stimuli situated in any other orientation. It was proved that the stimuli in the oblique orientation is perceived as square for greater degrees of misalignment than those in the cardinal axis orientation (see figure 3).

Here we simulate the experiment with our model and for misalignment at 0, 6, and 12 degrees for the Kanizsa squares and diamonds (see figure 17). We first express the propagation with the two different kernels $K_{\Pi hor}$ and $K_{\Pi diag}$ in terms of fundamental solution of differential equations. The horizontal kernel is strongly directional, so that it will be represented as a pure subriemannian operator. Consequently the Lagrangian operator will be exactly the one introduced in (16):

[TABLE]

The diagonal kernel is weaker and less directional, so that we will add a Riemannian coefficient to better model it. Consequently the Lagrangian operator will be:

[TABLE]

The minimizers of the two functionals are computed and results are presented in figure 18: as in the perceptual experiment, both the Kanizsa square and diamond are completed if there is no misalignment (left image). In presence of a misalignment of 6 degrees, due to the strongly directional properties of the kernel, the horizontal Kanizsa square is not reconstructed, while the strongly diffusive effect of the diagonal kernel does not allow to appreciate the misalignment, and the diamond is much better completed in accordance with experimental results (middle image). Finally for a misalignment of 12 degrees, neither the square nor the diamond are reconstructed (right image).

5 Conclusion

We presented here a general model, which unifies a number of models of the neurogeometry of perception. Every family of cells, is parametrized with coordinates $R^{2}\times F$, where $R^{2}$ represents the position on the retinal plane and $F$ the extracted feature. The cortical connectivity is expressed via a kernel $K_{F}$, whose eigenvectors can describe the emerge of perceptual units [73] and represent the distribution of neural activity in un the primary cortex. Cortical connectivity kernels, projected on the 2D plane induces a geometry of perception on the visual plane. Propagation with these kernels is responsible for perceptual completion. The model is first applied to contrast and orientation perception using kernels invariant with respect to suitable Lie group laws following the presentation of [23]. These models are able to justify modal completion as for example the Kanzsa square. However they discard the ability of the visual system to perceive stimuli lying in the horizontal orientation with greater efficiency than in any other orientation. In order to take into account this aspect, we applied the model with non isotropic kernels learned by statistical of images, recovering anisotropy in the geometry of perception in different orientations. A more radical heterogeneity able to account for different geometries locally defined will be considered in the future. A preliminary study has been presented in [71].