The multidegree of the multi-image variety
Laura Escobar, Allen Knutson

TL;DR
This paper computes the multidegree of the multi-image variety, a mathematical model for multiple rational cameras, by analyzing its cohomology class within a product of Grassmannians.
Contribution
It provides the first explicit calculation of the multidegree of the multi-image variety, linking algebraic geometry with multi-view camera models.
Findings
Computed the cohomology class of the multi-image variety.
Derived its multidegree in the Plücker embedding.
Established a connection between algebraic geometry and multi-camera imaging models.
Abstract
The multi-image variety is a subvariety of Gr(1,P^3)^n that models taking pictures with n rational cameras. We compute its cohomology class in the cohomology of Gr(1,P^3)^n, and from there its multidegree as a subvariety of (P^5)^n under the Pl\"ucker embedding.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Numerical Analysis Techniques · Medical Image Segmentation Techniques
∎
11institutetext: Laura Escobar 22institutetext: University of Illinois at Urbana-Champaign, Department of Mathematics, Urbana, IL, 61801, USA
22email: [email protected] 33institutetext: Allen Knutson 44institutetext: Cornell University, Department of Mathematics, Ithaca, NY, 14850, USA,
44email: [email protected]
The multidegree of the multi-image variety
Laura Escobar and Allen Knutson
Abstract
The multi-image variety is a subvariety of that models taking pictures with rational cameras. We compute its cohomology class in the cohomology of , and from there its multidegree as a subvariety of under the Plücker embedding.
1 Introduction
Multi-view geometry studies the constraints imposed on a three-dimensional scene from various two-dimensional images of the scene. Each image is produced by a camera. Algebraic vision is a recent field of mathematics in which the techniques of algebraic geometry and optimization are used to formulate and solve problems in computer vision. One of the main objects studied by this field is a multi-view variety. Roughly speaking, a multi-view variety parametrizes all the possible images that can be taken by a fixed collection of cameras. See AST ; PST ; THP for more details on multi-view varieties. See SRTGB for a survey on various camera models.
The work PST presents a new point of view to study the multi-view varieties. A photographic camera maps a point in the scene to a point in the image. PST defines a geometric camera, which maps a point in the scene not to a point, but to a viewing ray. More precisely, a photographic camera is a map or , whereas a geometric camera is a map such that the image of each point is a line containing the point. The viewing ray corresponding to a point is the line in this point gets mapped to. An important assumption is that light travels along rays which means that the image of a geometric camera is two-dimensional.
We now illustrate these definitions using the example of pinhole cameras or cameræ obscuræ, see Figure 1. As a geometric camera, a pinhole camera maps a point to the line connecting to the focal point. This map is rational since it is undefined at the focal point. As a photographic camera, a pinhole camera maps a point to the intersection of and the plane at the back of the camera. Notice that, as a photographic camera, there are many pinhole cameras corresponding to a given focal point. All of these cameras are equivalent up to a projective transformation. The essential part of a pinhole camera is the mapping of the scene points to viewing rays, i.e. its modeling as a geometric camera.
A multi-image variety is the Zariski closure of the image of a map
[TABLE]
where each is a geometric camera. The multi-view variety is a multi-image variety such that each is a pinhole camera. Let be the closure of the image of inside the -th . Then under some assumptions, (PST, , Theorem 5.1) shows that
[TABLE]
where is the concurrent lines variety consisting of ordered -tuples of lines in that meet in a point . The concurrent lines and multi-image varieties are embedded into using the Plücker embedding.
The multidegree of a variety embedded into a product of projective spaces is the polynomial whose coefficients give the numbers (when finite) of intersection points in the variety intersected with a product of general linear subspaces. The present paper verifies the conjectured formula (PST, , Equation (11)) for the multidegree of and computes the multidegree of the multi-image variety. To do so, we describe using a projection of a partial flag variety and use Schubert calculus to compute the cohomology classes of and in the cohomology ring of . We then push forward these formulæ into to obtain the multidegrees.
We now describe the organization of this paper. In §2 we define the main objects of study: the multi-image variety and the concurrent lines variety. We present the main theorem which computes the multidegrees of these objects. The main tool to prove the main theorem is Schubert calculus. In §3 we give a brief introduction to Schubert calculus for . In §4 we compute the cohomology class of the multi-image variety and the concurrent lines variety in terms of the Schubert cycles in . We prove the main theorem by taking the pushforward of these equations to the cohomology ring of . In §5 we refine these results to a computation of the -class for the concurrent lines variety.
2 The multi-image variety
The Grassmannian consists of -dimensional planes inside . A congruence is a two-dimensional family of lines in , i.e. a surface in . The bidegree of a congruence is a pair of nonnegative integers such that the cohomology class of in has the form
[TABLE]
The first integer is called the order and counts the number of lines in that pass through a general point of , and is called the class and counts the number of lines in that lie in a general plane of . The focal locus of consists of the points in that do not belong to distinct lines of .
Example 1
A congruence with bidegree consists of all lines in that contain a fixed point. A geometric camera for such a congruence represents a pinhole camera where the fixed point is the focal point.
Example 2
A two-slit camera assigns to the unique line passing through and intersecting two fixed lines , see Figure 2. Its focal locus is . These cameras correspond to the congruences with bidegrees .
The study of congruences started with Kum which classified those of order one. They were studied by many mathematicians during the second half of the 19th century; see the book Jes for some of these results.
Remark 1
Congruences are studied the article KNT . In particular, (KNT, , §2) discusses congruences and their bidegrees. The bidegrees of curves in are discussed in the article NFSY .
Consider a rational map
[TABLE]
where the closure or equals , each is a congruence and for all . Each map is defined everywhere except on the focal locus of . In the language of algebraic vision such a map means taking pictures with rational cameras where each is the -th image plane. An important assumption is that light travels along rays which means that is -dimensional.
Let be congruences of bidegree . Note that is the unique line in passing through . The multi-image variety of is the Zariski closure of . The concurrent lines variety consists of ordered -tuples of lines in that meet in a point . By (PST, , Theorem 5.1), if the focal loci of the congruences are pairwise disjoint then the multi-image equals the intersection
[TABLE]
Most of the cameras studied in computer vision are associated with congruences of order . However, cameras of higher order also appear in computer vision, see SRTGB and (PST, , §7). As an example of a camera associated to a congruence of bidegree let us discuss a non-central panoramic camera, see Figure 3. Consider a circle obtained by rotating a point about a vertical axis . There are two lines in passing through a general point and intersecting both and . The congruence consisting of all lines intersecting both and has bidegree . A physical realization of a non-central panoramic camera consists of a sensor on the circle taking measurements pointing outwards. This orientation of the sensor yields a map , i.e. it assigns only one line to a point .
The concurrent lines and multi-image varieties are embedded into using the Pücker embedding
[TABLE]
The multidegree of a variety embedded into a product of projective spaces is a homogeneous polynomial whose term indicates that there are intersection points when is intersected with the product of general linear subspaces, where . The degree of this polynomial equals the codimension of in . An equivalent definition of the multidegree of is its cohomology class in the cohomology ring of . The built-in command multidegree in the software Macaulay2 M2 computes the multidegree of from its defining ideal. We refer to the book (MS, , §8.5) for more details on multidegrees.
The main theorem of this paper computes the multidegree of the multi-image variety:
Theorem 2.1
The multidegree of the concurrent lines variety in equals
[TABLE]
Let be the bidegree of for . The multidegree of in equals
[TABLE]
where we distribute accordingly whenever .
In particular, the multidegree of the multi-image variety of where the bidegree of is equals
[TABLE]
Remark 2
Ponce-Sturmfels-Trager (PST, , equation (11)) had conjectured Formula 2 for the multidegree of , based on experimental evidence from Macaulay2.
Remark 3
Aholt-Sturmfels-Thomas (AST, , Corollary 3.5) gives an equation for the multidegree of the multi-image variety for the congruences with bidegree . This equation coincides with the equation of Theorem 2.1.
To prove this theorem, we first use Schubert calculus to obtain the cohomology classes of and the multi-image variety in the cohomology ring of ; see Theorems 4.1 and 4.2. Using this formula we then describe the multidegrees of these varieties and prove Theorem 2.1. In the next section we introduce our notation for Schubert varieties and review the part of Schubert calculus that we need.
3 Schubert varieties
In this section we review Schubert varieties in keeping as the main example. For further details we recommend the book Ful .
Fix a coordinate system for . Let denote the coordinate subspace of spanned by the coordinates and consider the standard flag
[TABLE]
The Schubert variety in corresponding to the subset is
[TABLE]
Example 3
The Schubert varieties in are
[TABLE]
[TABLE]
Consider the Schubert cells defined by replacing by in Equation (3). Since is a disjoint union of the Schubert cells, and they are each contractible, the classes for form a basis for the cohomology ring of . The ring operation is the cup product
[TABLE]
where is defined by using instead of in Equation 3. (Unlike , this variety intersects transversely, while having the same cohomology class as .) One also gets a basis for the cohomology ring of products of Grassmannians via the Künneth isomorphism.
Example 4
Note that for any since . Any line containing is not contained in and therefore . We have that since there is a unique line containing the points and . On general Grassmannians, computing can be done using the “Pieri rule” for special classes, see Equation (4), and the “Littlewood-Richardson rule” LR for arbitrary dimensions.
Schubert varieties stratify . The poset of their inclusions is most easily described when Schubert varieties are indexed using partitions. A partition of into parts is a list such that . There is a bijection between -subsets of and partitions with at most parts such that which is given by
[TABLE]
Partitions can be visualized in the following way. Given , we draw a figure made up of squares sharing edges that has squares in the first row, squares in the second row starting out right below the beginning of the first row, and so on. This figure is called a Young diagram. See Figure 4 for an example. We say that if for all , i.e. if the diagram of lies inside the one for .
From now on we will index Schubert cells and varieties by partitions. With this indexing set we have the following facts:
- •
,
- •
iff , and
- •
The Pieri rule: suppose that and is any partition. Then
[TABLE]
where the sum is over all partitions such that , , and .
Example 5
The Schubert varieties in are ordered by containment in the poset in Figure 5.
This poset is ranked by the dimensions of the Schubert varieties. By the Pieri rule, we have that and .
Remark 4
We can rewrite Equation 1 for the cohomology class of a congruence as
[TABLE]
Remark 5
In the article (KNT, , §6), Schubert varieties and the intersection theory of are also discussed.
4 Computing the multidegrees
Let be the partial flag manifold consisting of pairs where is a point in and is a line through . Such a pair is called a (partial) flag in . Define to be the subvariety of consisting of lists of flags such that the point is the same in all of them. Consider the diagonal
[TABLE]
then is the preimage of under the projection induced by
[TABLE]
For example, consists of pairs of flags of the form . The following is straightforward:
Proposition 1
The concurrent lines variety is the image of under the projection induced by
[TABLE]
The intersection consists of the lists of flags such that , and this intersection is transverse. We can write as the transverse intersection
[TABLE]
From this description we deduce the following Theorems which give the cohomology classes of and the multi-image variety.
Theorem 4.1
The class of the concurrent lines variety in the cohomology ring of is
[TABLE]
Moreover, can be written as
[TABLE]
Remark 6
Equation (7) is more natural than Equation (8), in that (7) is correct in -equivariant cohomology, whereas (8) is only correct in ordinary cohomology.
Example 6
Consider . In -equivariant cohomology we have that
[TABLE]
In ordinary cohomology we have that
[TABLE]
Proof
From equation (6) we have that
[TABLE]
in the cohomology of . Identifying using the Künneth isomorphism, we have
[TABLE]
and hence
[TABLE]
Pulling this back to , we get essentially the same formula
[TABLE]
Pushing forward this class under the projection map , we get . Not all terms of the formula above survive when we project to by forgetting :
- •
The image of under equals . Since for the dimension of drops under , any factor pushes down to [math].
- •
The other terms push down to .
It follows that
[TABLE]
Note that for any we have that and . Therefore the terms corresponding to and push down to . ∎
Theorem 4.2
For , let be a general congruence with class . Then
[TABLE]
Example 7
Let be three congruences with bidegrees . Then
[TABLE]
Proof
Let be a surface of class . By Theorem 4.1,
[TABLE]
Using the computations from Examples 4 and 5 we have that:
- •
When , then ,
and the factor is
- •
When , then ,
and the factor is .
- •
When , then ,
giving .
The result is
[TABLE]
∎
Remark 7
When we can drop the terms with in the equation for .
Let us now describe the classes in under the Plücker embedding . To do so we describe their equations inside . The degrees of general Schubert varieties were computed by Schubert in Sch .
A line passing through the points is uniquely determined by the minors of the matrix with rows and . Let denote the minor of the columns and . The Plücker embedding associates the vector of the minors to each line in .
- •
is defined by the Plücker relation . Therefore its degree is and its codimension , so .
- •
The condition is equivalent to . So is the complete intersection . Therefore it has degree and codimension , and so .
- •
Similarly, the condition is equivalent to . Therefore is the intersection
[TABLE]
which becomes the complete intersection . Therefore it has degree and codimension , and so .
- •
is the intersection
[TABLE]
which becomes the complete intersection . As just above, .
- •
is the intersection
[TABLE]
so the complete intersection . Therefore it has degree and codimension , and so .
- •
is the complete intersection . Therefore it has degree , codimension and .
We summarize these computations in Table 1.
Remark 8
The small case relevant for this paper is convenient but misleading: already in one meets Schubert varieties that are not complete intersections in the Plücker embedding, making it less straightforward to compute their degrees.
Proof (of Theorem 2.1)
We compute the multidegrees of and by using Table 1 to specialize
[TABLE]
in the -th component of Equation (8). For we obtain
[TABLE]
Note that given such that , there are exactly two different possibilities: either
for all but three indices at which , or 2. 2.
for all but two indices at which and .
In the first case, we have a term of the form . In the second case we have a term of the form .
Similarly, for we obtain
[TABLE]
The formula in the theorem is deduced similarly as in the case. ∎
5 -theory
We conclude this paper by computing the -class of the concurrent lines variety . Our reference for -polynomials is (MS, , §8.5), but we include some words of motivation here.
5.1 A -class example, and the definition
Consider the diagonal , the space of pairs
[TABLE]
This equation degenerates to (preserving the homology class), which vanishes on the union . Therefore the homology class is the sum of the classes of the two components. (We used the analogous calculation in the proof of Theorem 4.1.)
If are subvarieties (of some manifold) of the same dimension, then the equation we just used in homology doesn’t hold for the “-classes” we’re about to define: rather, it obeys a sort of inclusion-exclusion formula
[TABLE]
(the s to be defined below). Put another way, -theory cares about the overcounting of the intersection, even though its dimension is smaller than that of the components. In this sense, homology just sees the top-dimensional part of a -class. Very precisely, there is a filtration on and an isomorphism .
We only need to define the -theory of , which we do now. Begin with the abelian group generated by (isomorphism classes of) finitely generated -graded modules over the polynomial ring in variables , where acts homogeneously with weight . We impose two types of relations on this group, the first coming from exact sequences of multigraded modules, and the second for each , coming from modules annihilated by . The resulting set of equivalence classes, “-classes”, we call . Actually Grothendieck derived the name “-theory” from the German word “klasse”, so -class is redundant.
The multigraded modules just described define sheaves on , i.e. a place for local functions on to act. (By Liouville’s theorem, the only global functions are constant.) In particular, for a closed subscheme (defined by some multigraded ideal), we know how to multiply local functions on by local functions on , giving us a sheaf we call with corresponding -class . In the example above the two -equivalent modules are just the quotients by the ideals .
Defined as above, when is smooth the abelian group has a somewhat non-obvious product, but the only part of it we will need is
[TABLE]
for example if are smooth and . In , any two hyperplanes define the same class , hence , and it turns out that .
While it will be nice that our methods serve to compute this finer invariant of , our motivation for introducing -classes is more concrete. For a subscheme, the data of is exactly the same information as the Hilbert polynomial of , which one can compute from a Gröbner basis for ’s ideal. In particular, a basis for ’s ideal is Gröbner if and only if the scheme defined by the leading monomials of the has the same Hilbert function as , whose stable behavior is captured by the -class .
Let us return to the example of , whose -class we can now write as
[TABLE]
from the inclusion-exclusion formula. We will prefer to write . We again have, and are using here, a Künneth isomorphism .
5.2 The -class of
Using a degeneration much like the one we had for , we find the -class to be
[TABLE]
From this formula we can follow arguments similar to those of §4 to compute the -class of in . Letting denote the resulting formula is
[TABLE]
This is an analogue of the Hilbert polynomial calculation of (AST, , theorem 3.6). There are two important differences: theirs concerns a rational map whereas ours is about a rational map . Also, their class in is multiplicity-free in the sense of Brion , which is what shows that every degeneration of their variety will be reduced (and Cohen-Macaulay), i.e. that they can have a universal Gröbner basis with squarefree initial terms (AST, , §2). Our class in is not multiplicity-free (thanks to those coefficients from Theorem 4.1).
However, Equation (8) shows that is multiplicity-free in the sense of Brion when considered as a subvariety of , and as such every degeneration of it inside that ambient space (not the larger space ) is reduced and Cohen-Macaulay.
Acknowledgements.
This article was initiated during the Apprenticeship Weeks (22 August-2 September 2016), led by Bernd Sturmfels, as part of the Combinatorial Algebraic Geometry Semester at the Fields Institute for Research in Mathematical Sciences. The authors would like to thank their anonymous referees as well as Jenna Rajchgot and Bernd Sturmfels. LE was supported by the Fields Institute for Research in Mathematical Sciences.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1(1) Chris Aholt, Bernd Sturmfels, and Rekha Thomas. A Hilbert scheme in computer vision. Canad. J. Math. , 65(5):961–988, 2013.
- 2(2) Michel Brion. Multiplicity-free subvarieties of flag varieties. Contemporary Math. 331, 13–23, Amer. Math. Soc., Providence, 2003.
- 3(3) William Fulton. Young tableaux , volume 35 of London Mathematical Society Student Texts . Cambridge University Press, Cambridge, 1997.
- 4(4) Daniel R. Grayson and Michael E. Stillman. Macaulay 2, a software system for research in algebraic geometry. Available at http://www.math.uiuc.edu/Macaulay 2/ .
- 5(5) Charles M. Jessop: A Treatise on the Line Complex , Cambridge University Press, 1903, (American Mathematical Society, 2001).
- 6(6) Kathlén Kohn, Bernt Ivar Utstøl Nødland, and Paolo Tripoli: Secants, bitangents, and their congruences, op. cit .
- 7(7) Ernst Kummer: Über die algebraischen Strahlensysteme, insbesondere über die der ersten und zweiten Ordnung , Abh. K. Preuss. Akad. Wiss. Berlin (1866) 1–120.
- 8(8) Dudley E. Littlewood and Archibald R. Richardson. Group characters and algebra. Philos. Trans. Royal Soc. London. , 233:99–141, 1934.
