Distribution of degrees of freedom over structure and motion of rigid bodies

TL;DR

This paper investigates the minimal data requirements for recovering structure and motion of rigid bodies from multiple frames, establishing the minimum points and frames needed and presenting solutions for different data configurations.

Contribution

It determines the minimum number of points and frames required for structure and motion recovery and provides closed-form and linear solutions for these cases.

Findings

3 frames and 3 points are the minimum for recovery

Linearization possible with 4 points or 4 frames

No recovery from only 2 frames regardless of points

Abstract

This paper is concerned with recovery of motion and structure parameters from multiframes under orthogonal projection when only points are traced. The main question is how many points and/or how many frames are necessary for the task. It is demonstrated that 3 frames and 3 points are the absolute minimum. Closed-form solution is presented. Furthermore, it is shown that the task may be linearized if either four points or four frames are available. It is demonstrated that no increase in the number of points may lead to recovery of structure and motion parameters from two frames only. It is shown that instead the increase in the number of points may support the task of tracing the points from frame to frame.

Tables

Table 1. Table 1 : Distances of points of a 3 point rigid body.

edge:	PQ (a)	QR (b)	RP (c)
length: real	2	3	4
length: squared	4	9	16

Table 2. Table 2 : Distances of points in a 4 point rigid body.

edge:	PQ	QR	RP
length: real	2	3	3.562
length: squared	4	9	12.6878
edge:	TP	TQ	TR
length: real	7.07107	7.43303	5.82734
length: squared	50	55.25	33.9578

Table 3. Table 3 : The coefficient matrix.

x1	x2	x3	x4	x5
-16.9258	0	0	-3.65832	3.43508
0	-0.423132	0	0	11.5049
0	0	1.82806	9.25371	0
-5.18936	0	0	-9.99644	9.04194
0	-11.1085	0	0	3.03494
0	0	-3.25323	-4.82034	0

Equations

$$a^4+b^4+c^4-2a^2{b}^2-2a^2{c}^2-2b^2{c}^2+{a_i}^4+{b_i}^4+{c_i}^4-2{a_i}^2{b_i}^2-2{a_i}^2{c_i}^2-2{b_i}^2{c_i}^2$$

$$+2(-{a_i}^2+{b_i}^2+{c_i}^2)a^2+2(+{a_i}^2-{b_i}^2+{c_i}^2)b^2+2(+{a_i}^2+{b_i}^2-{c_i}^2)c^2=0$$

$$(2(-{a_1}^2+{b_1}^2+{c_1}^2)-2(-{a_3}^2+{b_3}^2+{c_3}^2))a^2+(2({a_1}^2-{b_1}^2+{c_1}^2)$$

$$-2({a_3}^2-{b_3}^2+{c_3}^2))b^2+(2({a_1}^2+{b_1}^2-{c_1}^2)-2({a_3}^2+{b_3}^2-{c_3}^2))c^2$$

$$(({a_1}^4+{b_1}^4+{c_1}^4-2{a_1}^2{b_1}^2-2{a_1}^2{c_1}^2-2{b_1}^2{c_1}^2)$$

$$-({a_3}^4+{b_3}^4+{c_3}^4-2{a_3}^2{b_3}^2-2{a_3}^2{c_3}^2-2{b_3}^2{c_3}^2))=0,$$

$$(2(-{a_2}^2+{b_2}^2+{c_2}^2)-2(-{a_3}^2+{b_3}^2+{c_3}^2))a^2+(2({a_2}^2-{b_2}^2+{c_2}^2)$$

$$-2({a_3}^2-{b_3}^2+{c_3}^2))b^2(2({a_2}^2+{b_2}^2-{c_2}^2)-2({a_3}^2+{b_3}^2-{c_3}^2))c^2$$

$$(({a_2}^4+{b_2}^4+{c_2}^4-2{a_2}^2{b_2}^2-2{a_2}^2{c_2}^2-2{b_2}^2{c_2}^2)$$

$$-({a_3}^4+{b_3}^4+{c_3}^4-2{a_3}^2{b_3}^2-2{a_3}^2{c_3}^2-2{b_3}^2{c_3}^2))=0,$$

$$d_{a,1}=(2(-{a_1}^2+{b_1}^2+{c_1}^2)-2(-{a_3}^2+{b_3}^2+{c_3}^2)),$$

$$d_{b,1}=(2({a_1}^2-{b_1}^2+{c_1}^2)-2({a_3}^2-{b_3}^2+{c_3}^2)),$$

$$d_{c,1}=(2({a_1}^2+{b_1}^2-{c_1}^2)-2({a_3}^2+{b_3}^2-{c_3}^2)),$$

$$d_{Cst,1}=(({a_1}^4+{b_1}^4+{c_1}^4-2{a_1}^2{b_1}^2-2{a_1}^2{c_1}^2-2{b_1}^2{c_1}^2),$$

$$-({a_3}^4+{b_3}^4+{c_3}^4-2{a_3}^2{b_3}^2-2{a_3}^2{c_3}^2-2{b_3}^2{c_3}^2)),$$

$$d_{a,2}=(2(-{a_2}^2+{b_2}^2+{c_2}^2)-2(-{a_3}^2+{b_3}^2+{c_3}^2)),$$

$$d_{b,2}=(2({a_2}^2-{b_2}^2+{c_2}^2)-2({a_3}^2-{b_3}^2+{c_3}^2)),$$

$$d_{c,2}=(2({a_2}^2+{b_2}^2-{c_2}^2)-2({a_3}^2+{b_3}^2-{c_3}^2)),$$

$$d_{Cst,2}=(({a_2}^4+{b_2}^4+{c_2}^4-2{a_2}^2{b_2}^2-2{a_2}^2{c_2}^2-2{b_2}^2{c_2}^2),$$

$$-({a_3}^4+{b_3}^4+{c_3}^4-2{a_3}^2{b_3}^2-2{a_3}^2{c_3}^2-2{b_3}^2{c_3}^2)),$$

$$a^2=\frac{(-d_{c,1}{c}^2-d_{Cst,1})\cdot d_{b,2}-(-d_{c,2}{c}^2-d_{Cst,2})\cdot d_{b,1}}{d_{a,1}\cdot d_{b,2}-d_{a,2}\cdot d_{b,1}},$$

$$b^2=\frac{d_{a,1}\cdot (-d_{c,2}{c}^2-d_{Cst,2})-d_{a,2}\cdot (-d_{c,1}{c}^2-d_{Cst,1})}{d_{a,1}\cdot d_{b,2}-d_{a,2}\cdot d_{b,1}}.$$

$$A_c=\frac{(-d_{c,1})\cdot d_{b,2}-(-d_{c,2})\cdot d_{b,1}}{d_{a,1}\cdot d_{b,2}-d_{a,2}\cdot d_{b,1}},A_{Cst}=\frac{(-d_{Cst,1})\cdot d_{b,2}-(-d_{Cst,2})\cdot d_{b,1}}{d_{a,1}\cdot d_{b,2}-d_{a,2}\cdot d_{b,1}},$$

$$B_c=\frac{d_{a,1}\cdot (-d_{c,2})-d_{a,2}\cdot (-d_{c,1})}{d_{a,1}\cdot d_{b,2}-d_{a,2}\cdot d_{b,1}},B_{Cst}=\frac{d_{a,1}\cdot (-d_{Cst,2})-d_{a,2}\cdot (-d_{Cst,1})}{d_{a,1}\cdot d_{b,2}-d_{a,2}\cdot d_{b,1}}.$$

$$a^2=A_c{c}^2+A_{Cst},b^2=B_c{c}^2+B_{Cst}.$$

$$(A_c{c}^2+A_{Cst}{)}^2+(B_c{c}^2+B_{Cst}{)}^2+c^4-2(A_c{c}^2+A_{Cst})(B_c{c}^2+B_{Cst})$$

$$-2(A_c{c}^2+A_{Cst})c^2-2(B_c{c}^2+B_{Cst})c^2+(a_3^4+b_3^4+c_3^4-2a_3^2{b}_3^2-2a_3^2{c}_3^2-2b_3^2{c}_3^2)$$

$$+2(-a_3^2+b_3^2+c_3^2)(A_c{c}^2+A_{Cst})+2(a_3^2-b_3^2+c_3^2)(B_c{c}^2+B_{Cst})+2(a_3^2+b_3^2-c_3^2)c^2=0.$$

$${a_i}^4+{b_i}^4+{c_i}^4-2{a_i}^2{b_i}^2-2{a_i}^2{c_i}^2-2{b_i}^2{c_i}^2-{a_1}^4-{b_1}^4-{c_1}^4+2{a_1}^2{b_1}^2+2{a_1}^2{c_1}^2$$

$$+2{b_1}^2{c_1}^2=2a^2({a_i}^2-{b_i}^2-{c_i}^2-{a_1}^2+{b_1}^2+{c_1}^2)+2b^2(-{a_i}^2+{b_i}^2-{c_i}^2+{a_1}^2-{b_1}^2$$

$$+{c_1}^2)+2c^2(-{a_i}^2-{b_i}^2+{c_i}^2+{a_1}^2+{b_1}^2-{c_1}^2)$$

Full text

Distribution of degrees of freedom over

structure and motion of rigid bodies

Mieczyslaw A. Kłopotek

Institute of Computer Science, Polish Academy of Sciences

e-mail: klopotek@ipipan.waw.pl

Abstract.  This paper is concerned with recovery of motion and structure parameters from multiframes under orthogonal projection when only points are traced. The main question is how many points and/or how many frames are necessary for the task. It is demonstrated that 3 frames and 3 points are the absolute minimum. Closed-form solution is presented. Furthermore, it is shown that the task may be linearized if either four points or four frames are available. It is demonstrated that no increase in the number of points may lead to recovery of structure and motion parameters from two frames only. It is shown that instead the increase in the number of points may support the task of tracing the points from frame to frame.

1 Introduction

Recovery of a three-dimensional structure from a single view of even the simplest scene consisting of a single object seems to be next to impossible. A number of additional assumptions seems to be necessary for a successful recovery. Some of clues mentioned below (and many other) or their combinations proved helpful in the past:

• •

model restrictions (the object belongs to one of parametric classes to be identified),

• •

surface property assumptions (shades, texture etc.),  [5, 3],

• •

usage of synchronized pairs of views (stereoscopic images of various types),

• •

usage of longer sequences of non-synchronized frames,

• •

assumptions restricting the pattern of motion (e.g. rotation around a fixed direction etc.), [6, 7], etc.

This paper is concerned with recovery of motion and structure parameters from multiframes under orthogonal projection when only points are traced from frame to frame (a finite number of them). We assume that the body is rigid that is that the intrinsic (3D) distances between the traced points are retained from frame to frame. We assume further that the only properties we extract from frames are the distances between projections of traced points (and not for example their ”textures”, shades, colors, light reflections etc.). We shall call such a body a ”point body”. We assume that the pattern of motion of the body is unrestricted between the frames. The term ”unrestricted” means that we do not assume any particular pattern of motion, e.g. rotation around a fixed axis, or orbiting around an attractive center in gravity field etc., though we do not forbid such a motion. Still one shall be concious of the fact that motion of the body reflecting various sides of it is vital for reconstruction. E.g. the pattern of motion of motionlessness is not suitable for recovery purposes from multiframes, and also pure shifts of the body as well as pure rotations around an axis orthogonal to the projection plane is not the case, because then we would have to do with recovery from a single frame.

Though the problem may look fairly simplified, we shall say that similar problems have already been studied in the past, e.g.  [6], has been concerned with bodies consisting of two traceable point rotating around a fixed direction. On the other hand it may be still of practical relevance. Fixing traceable points at military vehicles is used to trace the motion of own troops. In this case the geometry of the rigid body is known and only the motion may be of interest. But assume the reverse situation. We want to trace the enemy troops where we only know that vehicles are marked but the geometry of marking is not known. Here we will have then to do with the complete problem of recovery of both structure and motion. The main question is how many points and/or how many frames are necessary for the task. In this paper, it is demonstrated that 3 frames and 3 points are the absolute minimum. First, a closed-form solution is presented in section 2. In sections 3 and 4 it is shown that the task may be linearized if either four points or four frames are available. In section 5 we are concerned with the problem what kind of information may be gained if only two frames are considered. It is demonstrated that no increase in the number of points may lead to recovery of structure and motion parameters from two frames only. It is shown, however, that instead the increase in the number of points may support the task of tracing the points from frame to frame. The paper ends with a brief discussion and some concluding remarks.

2 Three points and three frames

It is an interesting question to investigate the possibility of reconstruction of structure and motion from multiframes under orthogonal projection. As mentioned in  [8], it is possible to recover them from three traceable points and three images having a quadratic equation system, which may be simplified to a linear one if four images or four frames are available.

2.1 Degrees of freedom for orthogonal projection

Each point of the body introduces 3 df in the first frame minus one df for the whole body as there exists no possibility of determining the initial depth of the body in the space. The motion introduces for each subsequent frame 5 df only (three for rotations and two for translation), because the motion in the direction orthogonal to the projection plane has no impact on the image. In general, with $p$ points forming the rigid body traced over $k$ frames we have $-1+3*p+5*(k-1)$ degrees of freedom.

On the other hand, within each image each traced point provides us with two pieces of information: its x and its y position within the frame. Hence we have at most $k*2*p$ pieces of information available from k images. Thus we need at least to have the balance $-1+3*p+5*(k-1)\leq k*2*p$ to achieve recoverability.

Let us consider some combinations of parameters:

• •

for $k=3$ frames, $p=3$ points we get $-1+3*p+5*(k-1)=18=k*2*p=18$

• •

for $k=2$ frames, $p=4$ points we get $-1+3*p+5*(k-1)=-1+12+5=16=k*2*p=2*2*4=16$.

2.2 Structure and motion for 3 point correspondences

Let us briefly sketch the procedure of recovery of a three-point structure from multiframes.

Let $P,\,Q,\,R$ be the traced points of a rigid body, and $P_{i},\,Q_{I},\,R_{i}$ their respective projections within the ${i}^{th}$ frame. Let $a,\,b,\,c,\,a_{i},\,b_{i},\,c_{i}$ denote the lengths of straight line segments $PQ,\,QR,\,RP,\,P_{i}Q_{i},\,Q_{i}R_{i},\,R_{i}P_{i}$, respectively. Then for each frame one of the following relationships holds: Either: $\sqrt{{a}^{2}-{a_{i}}^{2}}+\sqrt{{b}^{2}-{b_{i}}^{2}}-\sqrt{{c}^{2}-{c_{i}}^{2}}=0$ or $\sqrt{{a}^{2}-{a_{i}}^{2}}-\sqrt{{b}^{2}-{b_{i}}^{2}}+\sqrt{{c}^{2}-{c_{i}}^{2}}=0$ or $-\sqrt{{a}^{2}-{a_{i}}^{2}}+\sqrt{{b}^{2}-{b_{i}}^{2}}+\sqrt{{c}^{2}-{c_{i}}^{2}}=0$ (which is easily seen from geometrical relationships, presented analytically and graphically by Kłopotek [10]). So we have three equations, for $i=1,\,2,$ and $3$, in three unknowns, $a,\,b,\,c$. As any of the above relationships gives after a twofold squaring:

[TABLE]

which is quadratic in ${a}^{2},\,{b}^{2},\,{c}^{2}$, hence solvable by exploitation of proper methods.

In an experiment we used a partial linearization approach. From formulas for $i=1$ and $i=2$ subtracted with one for $i=3$:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

denoting

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

we calculated the quantity ${a}^{2}$ and ${b}^{2}$ as follows:

[TABLE]

[TABLE]

Let us introduce notation:

[TABLE]

[TABLE]

So we have simply:

[TABLE]

We can now substitute these expressions into the equation for $i=3$:

[TABLE]

[TABLE]

[TABLE]

It is immediately obvious that the above equation is quadratic in ${c}^{2}$ and hence solvable by elementary methods. Then $a$, $b$ and $c$ can be calculated from previous equations and by square-rooting.

A few comments are necessary at this point. The above equation (and hence the original problem, as other variables are uniquely determined by $c$) may have none, one or two solutions (or infinitely many - if the three points happen to be collinear or two frames prove to be identical up to rotation). No solution may be attributed to some measurement errors (or to the fact that the three traced points do not in fact constitute a rigid body). The definite solutions need to be checked on physical feasibility, that is:

• •

neither ${c}^{2}$ nor ${a}^{2}$ nor ${b}^{2}$ can be negative,

• •

neither ${c}^{2}$ nor ${a}^{2}$ nor ${b}^{2}$ can be shorter than their respective projections in frames 1,2 and 3.

• Exmp. 1.

The 3 point rigid body with geometry given in tab. 1 has been rotated in space.

Three projections of that body are shown in fig. 1.

Two solutions proved to be feasible:

• –

${a}^{2}$= 4, ${b}^{2}$= 9, ${c}^{2}$= 16 and,

• –

${a}^{2}$= 2.6849, ${b}^{2}$= 8.33902, ${c}^{2}$= 16.2189.

We made also a study of impact of measurement errors.

Assuming errors of up to 0.1 % we got e.g. two solution:

a${}^{2}=$ 4.00152, b${}^{2}=$ 8.98252, c${}^{2}=$ 16.009, and a${}^{2}=$ 0.8868, b${}^{2}=$ 3.69478, c${}^{2}=$ 13.9678.

The first solution approximates the correct solution well (error below 0.2%).

Assuming errors of up to 1. % we got e.g. two solution:

a= 4.25684, b= 9.10636, c= 15.9228, and a= 1.09062, b= 6.57993, c= 12.2893.

The first solution approximates the correct solution not too well (error below 5%).

Assuming errors of up to 10. % we got e.g. two solution:

a= 5.11852, b= 10.8971, c= 15.3443, and a= 3.18443, b= 5.14897, c= 13.0865. The first solution deviates more than 15 % from the correct solution. $\Diamond$

3 Three points and four frames

Kłopotek, [10], simplified the equation system (1) for 3 traceable points by using four instead of three frames and subtracting the twofold squared equation for the first frame from those of the other ones. So one obtains three equations of the form for i=2, 3, 4:

[TABLE]

[TABLE]

[TABLE]

which are linear in ${a}^{2},\,{b}^{2},\,{c}^{2}$, hence solvable by exploitation of respective methods. (No linear dependence is introduced as a new frame is exploited unless the motion has a very special form.)

It should be noted that compared to the case of three frames with three points we gain the uniqueness of the solution (previously two solutions could prove correct for the given frames).

A numerical example can be found in [10].

In an analysis of measurement errors we assumed the following values of a = 2, b = 3, c = 3.562, hence a${}^{2}=$ 4, b${}^{2}=$ 9, c${}^{2}=$ 12.6878.

Assuming error level of up to 0.1% we got e.g.: a${}^{2}=$ 4.00805, b${}^{2}=$ 8.99673, c${}^{2}=$ 12.6841

which seems to be quite satisfactory.

Assuming error level of up to 1.% we got e.g.: a${}^{2}=$ 4.08991, b${}^{2}=$ 8.99485, c${}^{2}=$ 12.6539

which is not bad.

Assuming error level of up to 10.% we got the worst case: a${}^{2}=$ 1.02533, b${}^{2}=$ 3.78813, c${}^{2}=$ 9.84095 which is desastrous. However, the average performance was with 20% from deviation of correct values.

4 Four points and three frames

Let $P$, $Q$, $R$, $T$ be the traced points of a rigid body, and $P_{i}$, $Q_{I}$, $R_{i}$, $T_{i}$ their respective projections within the ${i}^{th}$ frame. Let $a$, $b$, $c$, $d$, $g$, $f$, $a_{i}$, $b_{i}$, $c_{i}$, $d_{i}$, $g_{i}$, $f_{i}$ denote the lengths of straight line segments $PQ$, $QR$, $RP$, $TR$, $TQ$, $TP$, $P_{i}Q_{i}$, $Q_{i}R_{i}$, $R_{i}P_{i}$, $T_{i}R_{i}$, $T_{i}Q_{i}$, $T_{i}P_{i}$, respectively. Then for each frame three relationships hold:

[TABLE]

and

[TABLE]

and

[TABLE]

Then from (2) we obtain the linear equation system for i=2, 3:

[TABLE]

[TABLE]

[TABLE]

and

[TABLE]

[TABLE]

[TABLE]

and

[TABLE]

[TABLE]

[TABLE]

This linear equation system is easily solved.

Again we obtain always (at most) a single solution instead of two as may be the case with three frames and three points only.

• Exmp. 2.

The 4 point rigid body, geometry of which is given in tab. 2, has been rotated in space.

Three projections are shown in the fig. 2.

We denote edges as:

[TABLE]

Then we obtain the equation system (as a matrix) in variables x1-x6 in tab. 3

The solution of the above equation system is:

x1 = 4, x2 = 9, x3 = 12.6878, x4 = 50, x5 = 55.25, x6 = 33.9578,

which means perfect agreement with the intrinsic rigid body.

The linearity has clearly its price: that is the sensitivity to measurement errors. If we have random errors of up to 0.1 % of the real value, then we get still reasonable results, e.g. in a test run we had:

x1 = 4.00643, x2 = 9.16415, x3 = 12.8076, x4 = 50.6951, x5 = 56.0627, x6 = 34.4038

On average, errors for edge lengths (square roots of the above) did not exceed 1 %. However random measurement errors of up to 1. % of the real value, lead to serious deterioration of results, e.g. in a test run we had:

x1 = 4.09934, x2 = 11.2053, x3 = 14.2781, x4 = 58.6152, x5 = 65.3898, x6 = 39.4439,

which means errors of well above 20 %. $\Diamond$

5 Handling 2 frames

So let us now consider a rigid body with four points over two frames (i=1, 2). Let us consider the equation system (2). Please notice that we have also a fourth relationship related to the triangle ABC: $\sqrt{{a}^{2}-{a_{i}}^{2}}=\sqrt{{b}^{2}-{b_{i}}^{2}}+\sqrt{{c}^{2}-{c_{i}}^{2}}$ but we make no use of it as it is linearly dependent on the three previous ones.

In this way we got 6 equations (3 for each of the two frames) in six variables a, b, c, d, g, f. The respective twofold squaring leads to quadratic equations. However, we cannot solve this equation system because, as we demonstrate below, they are dependent - see next subsection. Thereafter, in a subsection to follow, we show how unsolvability of this equation system may be exploited for point identification problem. The last subsection discusses consequences for recovery of curves from two frames.

5.1 Two frames - insufficient for recovery

Let us consider the following scene in 3D, consisting of three parallel lines p1, q1, r1 and three parallel lines p2, q2, r2 (see fig. 3). Let p1, p2 meet at P, let q1 meet q2 and r1 meet r2. Then planes q1/q2 and r1/r2 are parallel. Let n be a line crossing P and orthogonal to plane q1/q2 and hence to r1/r2. Let us rotate the body p2, q2, r2 around the n-axis. Let the rotated images be p2’, q2’, r2’ (see fig. 4). Then p2’ still crosses p1 at P, q2’ lies in the plane q1/q2 - hence unless parallel to q1 it meets q1 (say at Q’) and r2’ lies in r1/r2 and hence meets somewhere r1 (say at R’). Let a plane $\pi$ be orthogonal to p2 q2 r2. Let P2, Q2, R2 be points of intersection of $\pi$ and p2 q2 r2 respectively. But let us consider a plane $\pi^{\prime}$ orthogonal to p2’q2’r2’. n is then parallel to this plane. Let P2’, Q2’, R2’ be points of intersection of $\pi^{\prime}$ and p2’ q2’ r2’ respectively. Distances of n to p2’q2’r2’ are the same as to p2q2r2. So are the angles between image of n2 and P2’Q2’, Q2’R2’, R2’P2’ and the angles between image of n2 and P2Q2, Q2R2, R2P2. But r1 is orthogonal to n and so to its image in the plane P2’Q2’R2’. Hence the angles between image of r1 and P2’Q2’, Q2’R2’, R2’P2’ and the angles between image of r1 and P2Q2, Q2R2, R2P2 are pairwise identical. This implies that the lines on which points lie in the second image are determined from the three points. So the forth point T2 does carry only one piece of information in the second image instead of two. Hence there exists no possibility of recovery of 3-D structure from two images.

To convince the reader that the difference between structure of the two recovered triangles PQR and PQ’R’ is meaningful, we give spacial coordinates of points P, Q, R, Q’, R’: P(0.0, 0.0, 0.0), Q(3.46537, 2.0000, -2.0000), Q’(4.63902, 2.0000, -2.0000), R(.68697, 5.0000, 4.0000), R’(4.37296, 5.0000, 4.0000).

5.2 Point identity problem with two frames

The question seems at this point to be justified what happens with the one degree of freedom left unused. Let us consider what this freedom means geometrically. Three traced points ensure that for every other point of the first image we can identify the line which it lies on. This means a point T with its image T1 in the first frame must have its image lying on a concrete line t2 in the second frame. But if T2 does not lie on the pre-specified line? Than two things may have happened. Either T is not a part of a rigid body containing P, Q and R, or …. the identities of P2, Q2 and R2 have been assigned incorrectly.

But the latter means that if we have a set of projected points S1 and a set of projected points S2 of which we know that they are projections of a set of points belonging to a rigid body, but the identities are not ascribed, then we may be capable of assigning identity relations among points of the set S1 and the set S2. For this purpose we may select four points from the set S1 and try allocating to them points of the set S2. In all, if n is the cardinality of the set S2 (equal to the cardinality of the set S1) we may have to try $n\cdot(n-1)\cdot(n-2)\cdot(n-3)$ combinations of points. (In case of n=4 we have 432*1=24 combinations). First three points are then used to identify the line on which the forth point should lie in the second frame, and the distance between the line and the real position of projected point will be used to evaluate the goodness (or in fact the badness) of fit. The identity assignment minimizing the distance may be considered as the best.

The detailed procedure has been run in an implementation as follows: Let us denote the four traced points with P,Q,R,T and their distances as:

[TABLE]

Assuming the length of c to be some number, we calculate b as a function of c (from the triangle with edges a,b,c). Details are given in Appendix A. Substituting constant expressions -as in Appendix A - with constant symbols $f_{cb}f_{b}\Delta_{b},f_{b^{2}}$ we get the simple expression:

[TABLE]

Obviously, it is quadratic in $b{{}^{2}}$ and assuming knowledge of$c$ we introduce the notation $\Delta_{b}=({c}^{2}*f_{cb}+f_{b})^{2}-4*f_{b^{2}}*({c}^{2}*f_{c}+{c}^{4}*f_{c^{2}}+f_{Cst}),$ yielding: $b=\sqrt{(-({c}^{2}*f_{cb}+f_{b})\pm\sqrt{\Delta_{b}})/(2*f_{b^{2}}).}$

The a priori selected value for c can on the one hand grow towards infinity (which may result in the lost of precision), however it must fulfill several requirements for its minimal value. First of all it must be at least as long as its longest projection. Second, it shall not lead to negative $\Delta_{b}$ and also the resulting values of b and a as function of c shall be positive (and larger than their projections).

Then one proceeds as follows: Let us establish the coordinate system with axes $RP$, $RQ$, $RP\times RQ$, assuming that point R lies in the first and in the second frame plane. Let T1 denote the projection of the point T into the first frame, T2 - into the second. Let us denote with $T_{a}$ the point of intersection of the plane spanned by points RPQ, and with $T_{b}$ the point of intersection of the plane spanned by points RPQ and shifted by one unit in the direction $RP\times RQ$. It is easy to determine then coordinates of $T_{a}$ and $T_{b}$ in he coordinate system $RP$, $RQ$, $RP\times RQ$. Notice that projections $T1_{a}$ and $T1_{b}$ of points $T_{a}$ and $T_{b}$ onto the first frame coincide with $T1$. We can now simply identify projections $T2_{a}$ and $T2_{b}$ of points $T_{a}$ and $T_{b}$ onto the second frame. Points $T2$, $T2_{a}$ and $T2_{b}$ should be collinear in the second frame. If it is not the case, then something is wrong about assigning correspondences between points $P1,Q1,R1,T1$ and $P2,Q2,R2,T2$. Another combination of assignment of point-to-point-correspondences should be tried. (In practice, as we always run at risk of numeric rounding errors, we try to minimize the distance between the point $T2$ and the line $T2_{a}T2_{b}$).

As we have seen, b can in fact take one of two distinct values, hence the whole procedure is to be repeated twice and the lower value of the distance of the point $T2$ to the line $T2_{a}T2_{b}$ shall be taken.

• Exmp. 3.

Let us take the following two frames with four projected points on each.

Frame 1:

(5.301154,2.639265), (4.713916,1.319633), (0.000000,0.000000), (4.952014,0.000000).

Frame 2:

(4.509076,1.042773), (4.642879,0.521386), (0.000000,0.000000), (5.732019,0.000000) (the third point in each frame has been shifted to the origin of the coordinate system), see the fig. 5.

Let us demonstrate several attempts to match the points in these frames The first two attempts are the correct ones, differing only by the sign in the formula for calculating b, see the fig. 6.

And now some of other conceivable assignments of correspondences. We see that the assumed point T2 is distant from the line $T2_{a}T2_{b}$, see the fig. 7. $\Diamond$

5.3 Linearization with 5 points

Five points allow for a linearization as we can consider the position of intersection of the line SS1 connecting S with its projection S1 in the first frame with the planes PQT and QRP. Let call them $S_{a}$ and $S_{b}$ resp. The projections of S, $S_{a}$, $S_{b}$ coincide in the first frame. Coordinates of $S_{a}$ in terms of vectors TP, TQ and $S_{b}$ in terms of vectors RP, RQ are easily obtained from the first frame. Easily we can find projections of these points onto the second frame (using vectors T2P2, T2Q2, R2P2, R2Q2). Let us call these projections $S2_{a}$, $S2_{b}$. $S2_{a}$ and $S2_{b}$. form the line which should contain the projection S2 of point S in the second frame. In a straightforward manner we can calculate the discrepancy from this assumption.

The gain form linearization in this way is that we are free from considerations of solution requirements for quadratic equations. We have always to solve linear equation systems only. The loss is the increased combinatorial complexity as we have to match five points instead of four.

5.4 Recovery of curves from two frames

So far we have considered only finite sets of points. We have just demonstrated that for finite sets of tracable points two frames are insufficient for recovery of structural and motion parameters. But can a break-through be achieved if we have to do with infinite sets of points. Here we must carefully outline the frontier between various classes of tasks. If we have to do with surfaces, with shadows, textures etc., there exist various approaches handling the problem. But we want to concentrate here on objects where surface cannot be percepted (or percepted properly) - on curves. Objects consisting of wires can be viewed as such objects under some circumstances. Further let us assume that we are not aware of the category the object belongs to. If we knew it is a circle or somme opther object with weel-studied invariant properties, we could be helped just with this external information. But let us assume that the object we actually traced is a true any-shaped smooth 3-D curve (or a combination of such curves). Furthermore let us assume we were able to establish point-to-point correspomndence of all (!!!) points of both curves. Let us then assuma we made a guess of the intrinsic shape of the projected object so that it fits both of them.

Can we enjoy our guess? The answer is: No. We can take the very same procedure as in the first subsection of this section (with rotation around a properly selected straight line) and obtain another 3-D curve fiiiting both of our projections , usually totally different from the first one. In fact, we can rotate and rotate and obtain an infinite family of such curves, each of them fitting both projections. This is also true of any imperfect fit. Assume, due to some digitalization (much or less random) errors we cannot fit both images, but we run an approximating procedure yielding a best fit. If then we cannot attribute errors of the digital image to tthe distance between the image and the projected object, then we have no way to achieve unique identification of the 3-D curve.

6 Discussion

In this paper we have demonstrated that for orthogonal projection of rigid point bodies at least three frames and three points are necessary to recover structure and motion. It has been shown that the complexity of the task is that of solving a quadratic equation in one variable with postprocessing to meet physical constraints. The problem under consideration may be linearized, however, if we can trace four points over three frames or three points over four frames.

From a degrees-of-freedom argument it became visible that the amount of information that two frames with four traced points may provide enough information to recover structure and motion from two frames. However, it has been demonstrated that this is impossible because the rigid body assumption imposes internal dependence between the point projections so that information provided by the forth point and any further traced point cannot be consumed for purposes of recovery of structure and motion. This result extends to infinite sets of points: to objects constructed of a set of smooth true 3-D curves.

Instead, four points over two frames may solve identification problem of points between consecutive frames or alternatively the problem of belonging to the same rigid body. That is, in the first case, if we have two frames with four (or more) points each and we know that these points belong to the same rigid body, but we do not know the exact point to point correspondence, then we can exploit the unused information (not consumable for recovery of structure and motion) for purposes of identification of point-to-point correspondences. Alternatively, in the second case, when we have sets of points in two frames where the point-to-point-correspondence between frames is known, then we can exploit the unused information (not consumable for recovery of structure and motion) to decide, which points belong to the same rigid body.We have seen also that a fifth point can linearize the otherwise quadratic task.

It is worth mentioning at this point that several papers claimed for analogous problem under perspective projection that structure and motion can be recovered from two frames when 9, 7 or 5 points,  [1, 2, 4], or 4 points and 1 line,  [9], are traced. In  [12] the four- point-and-a-line claim of  [9] has been rejected on the basis of degrees-of-freedom argument and also by explicit construction. It seems to be worth investigating whether the 9, 7, and 5 points claims are valid or not, that is whether the internal constraints of a rigid body do not exhibit same properties as in case of four points under orthogonal projection.

7 Conclusions

• •

For orthogonal projection of rigid point bodies at least three frames and three points are necessary to recover structure and motion, when the motion is arbitrary (under no control of the observer).

• •

Solution to the structure and motion problem under these circumstances requires solving a quadratic equation, yielding sometimes two feasible solutions.

• •

If four frames or four points are available, then the problem can be linearized, giving unique solution (if the motion pattern is not degenerate).

• •

It is impossible to recover structure or motion from two frames whatever number of traced points is available. The result extends to infinite sets of points in case the traced object consists of smooth 3D curves (but has no surfaces).

• •

If a rigid body consists of at least four points, then we can solve the problem of point tracing for any two consecutive frames alone from knowledge which points of two frames belong to the body (without explicit knowledge of point-to-point correspondence)

• •

Alternatively, if a rigid body consists of at least four points, then we can solve the problem of belonging to a rigid body for any two consecutive frames alone from explicit knowledge of point-to-point correspondence.

• •

The solution of the above two problems involves solution of quadratic equation, but may be linearized if five points are available instead.

• •

It is worth investigating whether these results extend to perspective projections.

Appendix A

Let us denote the four traced points with P,Q,R,T and their distances as:

[TABLE]

Assuming the length of c can be some number, we can calculate b as a function of c (from the triangle with edges a,b,c). We take an equation system of the form given by equation (1), with i=1,2.

Both equations can be considered as quadratic in ${a}^{2}$. In the first step we subtract both equations getting an equation linear in ${a}^{2}$. We calculate ${a}^{2}$ from this equation and substitute the result into one of the original equations just eliminating the variable $a$. The result is an equation which is quadratic both in ${b}^{2}$ and in ${c}^{2}$. It can be expressed in a simple manner if we adopt the denotation:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Substituting constant expressions with constant symbols $f_{cb}f_{b}\Delta_{b},f_{b^{2}}$, we get the simple expression: ${b}^{4}*f_{b^{2}}+{b}^{2}*({c}^{2}*f_{cb}+f_{b})+({c}^{2}*f_{c}+{c}^{4}*f_{c^{2}}+f_{Cst})=0.$ Obviously, it is quadratic in $b{{}^{2}}$ and assuming knowledge of$c$ we introduce the notation $\Delta_{b}=({c}^{2}*f_{cb}+f_{b})^{2}-4*f_{b^{2}}*({c}^{2}*f_{c}+{c}^{4}*f_{c^{2}}+f_{Cst})$ yielding: $b=\sqrt{(-({c}^{2}*f_{cb}+f_{b})\pm\sqrt{\Delta_{b}})/(2*f_{b^{2}})}.$