Visual SLAM: Why Bundle Adjust?

TL;DR

This paper proposes replacing bundle adjustment in visual SLAM with rotation averaging and a quasi-convex formulation, simplifying the process and improving robustness during slow or rotational movements.

Contribution

It introduces a novel SLAM optimization approach that avoids the complexities of bundle adjustment by focusing on rotation averaging and efficient position estimation.

Findings

Simplifies SLAM by removing the need for full bundle adjustment.

Improves robustness during slow or rotational motions.

Achieves globally optimal camera position estimation.

Abstract

Bundle adjustment plays a vital role in feature-based monocular SLAM. In many modern SLAM pipelines, bundle adjustment is performed to estimate the 6DOF camera trajectory and 3D map (3D point cloud) from the input feature tracks. However, two fundamental weaknesses plague SLAM systems based on bundle adjustment. First, the need to carefully initialise bundle adjustment means that all variables, in particular the map, must be estimated as accurately as possible and maintained over time, which makes the overall algorithm cumbersome. Second, since estimating the 3D structure (which requires sufficient baseline) is inherent in bundle adjustment, the SLAM algorithm will encounter difficulties during periods of slow motion or pure rotational motion. We propose a different SLAM optimisation core: instead of bundle adjustment, we conduct rotation averaging to incrementally optimise only camera orientations. Given the orientations, we estimate the camera positions and 3D points via a quasi-convex formulation that can be solved efficiently and globally optimally. Our approach not only obviates the need to estimate and maintain the positions and 3D map at keyframe rate (which enables simpler SLAM systems), it is also more capable of handling slow motions or pure rotational motions.

Figures

Equations

$$\begin{array}{rlrl}& \underset{\{{\mathbf{X}}_i\},\{({\mathbf{R}}_j,{\mathbf{t}}_j)\}}{\min }& & \sum _{i,j}{\Vert {\mathbf{u}}_{i,j}-f({\mathbf{X}}_i\mid {\mathbf{R}}_j,{\mathbf{t}}_j)\Vert }_2^2,\end{array}$$

$${\mathsf{Z}}_{0:t}=\{{\mathsf{Z}}_0,{\mathsf{Z}}_1,\dots ,{\mathsf{Z}}_t\}.$$

$$\begin{array}{rlrl}& \underset{\{{\mathbf{R}}_j\}}{\min }& & \sum _{j,k\in \mathsf{N}}{\Vert {\mathbf{R}}_{j,k}-{\mathbf{R}}_k{\mathbf{R}}_j^{-1}\Vert }_F^2,\end{array}$$

$$\begin{array}{rlrl}& \underset{\{{\mathbf{X}}_i\},\{{\mathbf{t}}_j\}}{\min }& & \underset{i,j}{\max }\Vert {\mathbf{z}}_{i,j}-f({\mathbf{X}}_i\mid {\mathbf{R}}_j,{\mathbf{t}}_j){\Vert }_2,\end{array}$$

$${\mathbf{P}}_j=\mathbf{K}\left[{\mathbf{R}}_j{\mathbf{t}}_j\right]$$

$$f({\mathbf{X}}_i\mid {\mathbf{R}}_j,{\mathbf{t}}_j):=\frac{{\mathbf{R}}_j^{(1:2)}{\mathbf{X}}_i+{\mathbf{t}}_j^{(1:2)}}{{\mathbf{R}}_j^{(3)}{\mathbf{X}}_i+{\mathbf{t}}_j^{(3)}},$$

$$P_0:$$

$$\frac{{\Vert {\mathbf{A}}_{i,j}\left[\begin{array}{c}{\mathbf{X}}_i\\ {\mathbf{t}}_j\end{array}\right]\Vert }_2}{{\mathbf{b}}_j^{\top }\left[\begin{array}{c}{\mathbf{X}}_i\\ {\mathbf{t}}_j\end{array}\right]}\le \gamma ,\forall i,j,$$

$${\mathbf{b}}_j^{\top }\left[\begin{array}{c}{\mathbf{X}}_i\\ {\mathbf{t}}_j\end{array}\right]\ge 0,\forall i,j,$$

$$\gamma \ge 0,$$

$${\mathbf{A}}_{i,j}=\left[\begin{array}{c}{\mathbf{S}}_{i,j}{\mathbf{I}}_{2\times 2}-{\mathbf{z}}_{i,j}\end{array}\right],{\mathbf{b}}_j={\left[{\mathbf{R}}_j^{(3)}001\right]}^{\top },\text{and}$$

$${\mathbf{S}}_{i,j}={\mathbf{R}}_j^{(1:2)}-{\mathbf{z}}_{i,j}{\mathbf{R}}_j^{(3)}.$$

$${\Vert {\mathbf{A}}_{i,j}\left[\begin{array}{c}{\mathbf{X}}_i\\ {\mathbf{t}}_j\end{array}\right]\Vert }_2-\gamma {\mathbf{b}}_j^{\top }\left[\begin{array}{c}{\mathbf{X}}_i\\ {\mathbf{t}}_j\end{array}\right]\le 0$$

$$P_1:$$

$${\Vert {\mathbf{A}}_{i,j}\left[\begin{array}{c}{\mathbf{X}}_i\\ {\mathbf{t}}_j\end{array}\right]\Vert }_2\le \gamma {\mathbf{b}}_j^{\top }\left[\begin{array}{c}{\mathbf{X}}_i\\ {\mathbf{t}}_j\end{array}\right],\forall i,j,$$

$$\gamma \ge 0,$$

$${\mathbf{C}}_j=-{\mathbf{R}}_j^{\top }{\mathbf{t}}_j$$

$$\angle ({\mathbf{t}}_{j,k},{\mathbf{C}}_k-{\mathbf{C}}_j)\le \alpha \forall j,k,$$

$${\mathbf{t}}_{j,k}=({\mathbf{K}}^{-1}{\mathbf{R}}_j{)}^{\top }{\mathbf{t}}_{j,k}^{(E)},$$

$$P_3:$$

$${\Vert {\mathbf{B}}_{i,j}\left[\begin{array}{c}{\mathbf{X}}_i\\ {\mathbf{C}}_j\end{array}\right]\Vert }_2\le \gamma {\mathbf{c}}_j^{\top }\left[\begin{array}{c}{\mathbf{X}}_i\\ {\mathbf{C}}_j\end{array}\right],\forall i,j,$$

$${\Vert {\mathbf{D}}_{j,k}\left[\begin{array}{c}{\mathbf{C}}_j\\ {\mathbf{C}}_k\end{array}\right]\Vert }_2\le {\mathbf{e}}_{j,k}^{\top }\left[\begin{array}{c}{\mathbf{C}}_j\\ {\mathbf{C}}_k\end{array}\right],\forall j,k,$$

$$\gamma \ge 0,$$

$${\mathbf{B}}_{i,j}=\left[\begin{array}{c}{\mathbf{S}}_{i,j}-{\mathbf{S}}_{i,j}\end{array}\right],$$

$${\mathbf{D}}_{j,k}=\left[\begin{array}{c}{\mathbf{Z}}_{j,k}^{(1:2)}-{\mathbf{Z}}_{j,k}^{(1:2)}\end{array}\right],$$

$${\mathbf{Z}}_{j,k}{\mathbf{t}}_{j,k}=[001{]}^{\top }.$$

Full text

We propose a different SLAM optimisation core: instead of bundle adjustment, we conduct rotation averaging to incrementally optimise only camera orientations. Given the orientations, we estimate the camera positions and 3D points via a quasi-convex formulation that can be solved efficiently and globally optimally. Our approach not only obviates the need to estimate and maintain the positions and 3D map at keyframe rate (which enables simpler SLAM systems), it is also more capable of handling slow motions or pure rotational motions.

I INTRODUCTION

Let $\mathbf{u}_{i,j}$ be the 2D coordinates of the $i$-th scene point as seen in the $j$-th image $\mathsf{Z}_{j}$. Given a set $\{\mathbf{u}_{i,j}\}$ of observations, structure-from-motion (SfM) aims to estimate the 3D coordinates $\mathsf{X}=\{\mathbf{X}_{i}\}$ of the scene points and 6DOF poses $\{(\mathbf{R}_{j},\mathbf{t}_{j})\}$ of the images $\{\mathsf{Z}_{j}\}$ that agree with the observations. The bundle adjustment (BA) formulation is

[TABLE]

where $f(\mathbf{X}_{i}\mid\mathbf{R}_{j},\mathbf{t}_{j})$ is the projection of $\mathbf{X}_{i}$ onto $\mathsf{Z}_{j}$ (assuming calibrated cameras). In practice, not all $\mathbf{X}_{i}$ are visible in every $\mathsf{Z}_{j}$, thus some of the $(i,j)$ terms are dropped. For ease of exposition, we follow [1] and regard the image set $\{\mathsf{Z}_{j}\}$ as inputs to BA, bearing in mind that the effective inputs are the observations $\{\mathbf{u}_{i,j}\}$ and the visibility matrix.

As a non-linear least squares problem, (1) is usually solved by gradient descent methods, e.g., Levenberg-Marquardt, which require initialisation for all unknowns. Thus, apart from the images $\{\mathsf{Z}_{j}\}$, the total inputs to a BA instance typically include the initial values for $\{(\mathbf{R}_{j},\mathbf{t}_{j})\}$ and $\mathsf{X}$.

BA is justifiable in the maximum likelihood sense if the errors due to the uncertainty in localising the feature points $\{\mathbf{u}_{i,j}\}$ are Normally distributed. However, it is not obvious that available feature detectors satisfy this property [2, 3, 4]. While this does not reduce the usefulness of BA, its statistical validity should not be taken for granted.

I-A BA-SLAM

Roughly speaking, monocular feature-based SLAM [5] (henceforth, “SLAM”) is the execution of SfM incrementally to process streaming input images $\mathsf{Z}_{0:t}$, where

[TABLE]

Several influential works [6, 7, 1, 8] have cemented the importance of BA in SLAM. Algorithm 1, which is adapted from [1, Table 1], describes a SLAM optimisation core based on BA over keyframes. Specifically:

• •

In Step 5, new scene points are “spawned” if the current frame $\mathsf{Z}_{t}$ does not adequately observe the map $\mathsf{X}$.

• •

In Step 7 (a.k.a. local mappping), BA is used to estimate the camera trajectory and 3D map in the current time window. Often, local mapping is preceded by camera tracking to accurately initialise the current pose $(\mathbf{R}_{t},\mathbf{t}_{t})$. See [1, Sec. 5.3] or [8, Sec. V] for examples.

• •

In Step 9 (a.k.a. loop closure), a system-wide BA is executed to reoptimise all the variables and redistribute accumulated drift errors. Implicit in Algorithm 1 is the introduction of covisibility information between $\mathsf{Z}_{t}$ and older keyframes, prior to BA. Often, Step 9 is preceded by pose graph optimisation [9, 10, 11, 12] to give a more accurate initialisation of the poses.

Note that Algorithm 1 is merely a “basic recipe” for SLAM. In practice, “what will make or break a real-time SLAM system are all the (often heuristic) nitty-gritty details” [13], e.g., how to select features/keyframes, how to update the covisibility graph, how to select/merge/prune 3D points, etc. However, since our focus is on optimisation, Algorithm 1 is sufficient to capture the core algorithmic elements of SLAM systems based on BA, such as ORB-SLAM [8].

I-B Why do we want an alternative to BA-SLAM?

I-B1 High system complexity

Besides computing the trajectory and map in the current vicinity, Step 7 in BA-SLAM plays a more basic role: incrementally estimating the variables in small BA “chunks” to serve as initialisation for the system-wide BA in Step 9. Note that since (1) is amenable to only locally optimal solutions, without good initial values for the large number of variables (poses and 3D points), Step 9 will converge to poor solutions.

Therefore, unavoidably all variables must be estimated as accurately as possible and updated at keyframe rate throughout the lifetime of the algorithm—we argue that this increases the complexity of SLAM systems. For example, while Algorithm 1 shows only the creation of new 3D points (Step 5), in a practical system (e.g., [8]) a host of other heuristics are required for map maintenance, e.g., map point selection, point culling, map updating and aggregation. Many of these heuristics contain a number of thresholds, which, if not tuned carefully, will lead to system failure.

I-B2 Difficulties due to pure rotational motion

More fundamentally, since the estimation of 3D points (which require sufficient baseline) are essential, it is unavoidable that a system based on BA-SLAM will encounter numerical issues during periods of pure rotational motion or slow motion [14, Sec. 7.1], and will require special treatment to deal with this problem [15, 16, 17, 18]. This issue is particularly acute at the start of the sequence where the camera is usually slow moving111Pioneers of monocular SLAM [5] call the deliberate motion to initialise a SLAM algorithm the “SLAM wiggle”.. For example, in ORB-SLAM, elaborate map initialisation heuristics [8, Sec. IV] (cf. Step 1 in Algorithm 1) are used to combat inaccuracies due to insufficient translations. However, experienced users of ORB-SLAM cite its difficulty to initialise on challenging image sequences.

II L-infinity SLAM

Towards simpler visual SLAM systems, we propose a novel optimisation core called L-infinity SLAM; see Algorithm 2. A main distinguishing factor is that the online effort (Steps 4 and 8) are devoted to estimating only the camera orientations via rotation averaging [19, 20]. Given the orientations, a separate optimisation via the known rotation problem [21, 22] (Steps 5 and 9) is conducted to obtain the camera positions and 3D map—since rotation averaging can be done independently from position and map estimation, Steps 5 and 9 can be performed in a lower priority thread.

II-A Rotation averaging and known rotation problem

Given a set of relative rotations $\{\mathbf{R}_{j,k}\}$ between pairs of overlapping images $\{\mathsf{Z}_{j},\mathsf{Z}_{k}\}$, the goal of rotation averaging is to estimate the absolute rotations $\{\mathbf{R}_{j}\}$ that are consistent with the relative rotations (Sec. III-A will provide details on estimating relative rotations in our work). Following [20], we chose the chordal metric for rotations, which yields the rotation averaging formulation

[TABLE]

where $\mathsf{N}$ is the covisibility graph. In Step 3 of L-infinity SLAM, the covisibility graph $\mathsf{N}_{\text{win}}$ in the window is used, while in Step 8, the system-wide covisibility graph $\mathsf{N}_{\text{sys}}$ (updated to account for loop closure) is used. Sec. III will describe the specific algorithm for (3).

Given a set of absolute camera orientations $\{\mathbf{R}_{j}\}$, the known rotation problem (KRot) [21] optimises the camera positions $\{\mathbf{t}_{j}\}$ and 3D points $\{\mathbf{X}_{i}\}$ as

[TABLE]

subject to cheirality constraints (details in Sec. III). Observe that unlike (1) which minimises the sum of squared reprojection errors, (4) minimises the maximum reprojection error, which can be viewed as the $\ell_{\infty}$-norm of the vector of reprojection errors (leading to the name “L-infinity SLAM”).

At this juncture, it is vital to note that (4) is quasi-convex, which is amenable to efficient global solution [21, 22]. In our work, a novel variant of KRot is proposed specifically for the loop closure optimisation in Step 9; details in Sec. III.

II-B Benefits of L-infinity SLAM

II-B1 Simplicity

As alluded to above, tracking and loop-closing in L-infinity SLAM estimate only orientations. Since positions and 3D map are obtained via an independent optimisation problem that can be solved globally optimally, the results of Steps 5 and 9 do not affect the results of subsequent instances222Their results can be used to warm start the subsequent instances, but this is an optional computational consideration.. Therefore, there is no need to accurately calculate positions and 3D map on-the-fly and maintain/propagate them. Note that in Algorithm 2, Steps 5 and 9 are shown mainly to make the overall functionality of L-infinity SLAM equivalent to BA-SLAM. Contrast this to the equivalent steps in BA-SLAM (Steps 7 and 9 in Algorithm 1), whose resulting quality are vital at all times to ensure correct operation.

A signficant advantage of the processing flow of L-infinity SLAM is that many tasks related to map maintenance (e.g., feature/map point selection, point culling, map updating and aggregation) can be done in a low priority thread, or even offline if there is no need for on-the-fly position and map estimation (e.g., the application in [23]). This has the potential to significantly simplify visual SLAM systems.

II-B2 Handling pure rotation motion

It is well-known that under epipolar geometry, camera orientation can be estimated independently from the translation [24]. Hence, since the online routines in L-infinity SLAM estimate orientations only, a real-time system based on L-infinity SLAM is less likely to encounter difficulties due to pure or close-to-pure rotational motions. Potential numerical issues due to insufficient baselines between camera views can be handled in the low-priority thread that estimates position and 3D map. Sec. IV-B will provide results that illustrate this advantage of L-infinity SLAM over BA-SLAM.

II-C Concerns on global optimality and outliers

Some readers may find it disconcerting that in L-infinity SLAM the estimation of the variables are detached. First, note that we have not claimed that L-infinity SLAM is globally optimal in all variables. Second, there is ample evidence [25, 20] that rotation averaging algorithms are capable of producing highly-accurate orientation estimates, independently from positions and 3D points. Since the quasiconvex estimation for positions and 3D points is globally optimal, the overall quality of L-infinity SLAM will be high, as we will demonstrate in Sec. IV.

Also, a common impression of $\ell_{\infty}$ estimation is its sensitivity to outliers. Note, however, that both the $\ell_{\infty}$ and $\ell_{2}$ norms have a breakdown point of 0 [26], hence both norms are equally susceptible to outliers. In practical BA-SLAM systems, a typical remedy is to pass the $\ell_{2}$ residual through an isotropic robust norm (e.g., Cauchy norm). Likewise, there are efficient and theoretically justified techniques to identify and remove outliers in $\ell_{\infty}$ estimation [27, 28]. Hence, outliers do not present a problem for L-infinity SLAM.

III Algorithmic details

In this section, we describe the details of the core optimisation routines in L-infinity SLAM. Consider a calibrated camera with $\mathbf{K}$ the $\mathbb{R}^{3\times 3}$ camera intrinsic matrix. Let

[TABLE]

be the projection matrix of the j-th image with assumed known rotation matrix $\mathbf{R}_{j}$ in $SO(3)$ and unknown translation vector $\mathbf{t}_{j}$ in $\mathbb{R}^{3}$. For simplicity, we assume $\mathbf{K}=\mathbf{I}_{3\times 3}$. For an arbitrary $\mathbf{K}$, derivations are still valid if camera extrinsics are recovered by applying $\mathbf{K}^{-1}$ to $\mathbf{R}_{j}$ and $\mathbf{t}_{j}$, which now form the three first columns and the last column of $\mathbf{P}_{j}$.

III-A Estimating relative motions

L-infinity SLAM estimates camera rotations from relative camera rotations $\mathbf{R}_{j,k}$ in the covisibility graph $\mathsf{N}_{\text{win}}$. We simply estimate $\mathbf{R}_{j,k}$ from the essential matrix which can be decomposed into $\mathbf{R}_{j,k}$ and a relative translation direction $\mathbf{t}_{j,k}^{(E)}$ ($\|\mathbf{t}_{j,k}^{(E)}\|=1$). A weakness of this decomposition is the need of images with sufficient displacement; however, other methods can be used to estimate relative motions [24, 29, 30]. In the case of low displacement, we estimated $\mathbf{R}_{j,k}$ by rotationally aligning backprojected feature rays by using a rotation only variant of Trimmed ICP [31].

III-B Rotation averaging

Several methods exist to solve (3) [32, 19, 33, 20]. Here we adopt the robust method of [33] which uses an iteratively reweighted least-squares approach in $SO(3)$. The method in [33] is simpler than BA as, for example, no linearisation and no estimation of a damping factor is required.

III-C Known rotation problem

By referring to $\mathbf{R}_{j}^{(1:2)}$ as the first two rows of $\mathbf{R}_{j}$, and to $\mathbf{R}_{j}^{(3)}$ as the third row of $\mathbf{R}_{j}$ (similarly for $\mathbf{t}^{(1:2)}$ and $\mathbf{t}^{(3)}$), the projection of $\mathbf{X}_{i}$ onto the $j$-th image is given by

[TABLE]

and the known rotation problem (4) can be rewritten by adding an extra variable $\gamma$ as

[TABLE]

where

[TABLE]

Cheirality constraints (7c) impose to $\mathbf{X}_{i}$ to lie in front of the cameras in which $\mathbf{X}_{i}$ is visible.

Intuitively, $\gamma$ defines the sublevel sets of the objective in (4), i.e., the maximum over the LHS of (7b). For a fixed $\gamma$,

[TABLE]

defines a convex set hence the objective is quasi-convex and $(P_{0})$ is a quasi-convex problem. For a detailed proof of $(P_{0})$ being a quasi-convex problem the reader can refer to [34].

III-C1 Solving the known rotation problem

$(P_{0})$ can be rewritten as

[TABLE]

in which the Cheirality constraints (7c) are implicit in (11b) as both the LHS of (11b) and $\gamma$ are non-negative. If $\gamma$ is fixed, constraints (11b) became second-order cones which allows to solve ($P_{1}$) by using bisection trough SOCP feasibility tests [21]. Several other methods solve $(P_{1})$ through SOCP sub-problems. [35] shown that Gugat’s algorithm [36] outperforms among this type of methods (bisection, Brent’s algorithm[37], and Dinkelbach’s algorithm [38, 22]). Recently, Zhang et al. [39] presented a method, named Res-Int, which outperformed existent methods by alternating between pose estimation and triangulation to efficiently partition the problem into small sub-problems without compromising global optimality. As a result, Res-Int solves $(P_{1})$ in about 3 seconds for moderate size input (around 15 images and 3000 3D points).

We use Res-Int as the known_rotation_prob routine in Line 5 in Algorithm 2 since its superior performance. Although Res-Int converges in a few seconds for moderate problems (as those optimised for a moving window in Algorithm 2), its performance is still inadequate for medium to large size problems ($>10,000$ 3D points, $>100$ images) that Algorithm 2 optimises when detecting a loop. For such problems, Res-Int could take from a few minutes to hours to converge (see [39, Table 2]). To efficiently address loop closure, we propose a new formulation that incorporates relative camera translation directions (obtained from the essential matrix) to alleviate the size of the problem but still produce an accurate result.

III-D Known rotation problem with translation direction constraints

Solving KRot in Step 9 in Algorithm 2 can be excessively time-consuming as a loop can be detected at an advanced stage generating a large input size. Instead, we propose to address loop closure over a sample of the input with a formulation that incorporates camera translation directions.

Inspired by the quasi-convex approach of Sim and Hartley [40] to estimate the camera translations from $\mathbf{t}_{j,k}^{(E)}$ and known camera rotations, we constrained camera positions

[TABLE]

in the known rotation problem $(P_{1})$ to agree up to an angular threshold

[TABLE]

to

[TABLE]

which is the relative translation direction in world coordinates (we explicitly apply $\mathbf{K}^{-1}$ to $\mathbf{R}_{j}$ in (14) in case $\mathbf{K}\neq\mathbf{I}_{3\times 3}$ and therefore $\mathbf{R}_{j}$ is not a rotation matrix).

We observed that the method of [40] was unable to produce satisfactory results (see results in Sec. IV) for loop closure, arguably since no structural information is optimised thus camera positions were not sufficiently constrained. On the other hand, adding angle constraints (13) to $(P_{1})$ it allows to efficiently solve loop closure ($<100$ s) with a sparse set of 3D points (300 points for result in Fig. 2b). Our proposition yields the following problem.

[TABLE]

where

[TABLE]

and $\mathbf{Z}_{j,k}$ is a rotation matrix such that

[TABLE]

Similarly to the method in [40], $(P_{3})$ is valid for $\alpha<$$$. For derivation details of the camera translation direction constraints (15c) please refer to [40].

IV Results

Here we compare L-infinity SLAM (Algorithm 2) against BA-SLAM (Algorithm 1) on real data with precise ground truth. The used dataset, provided by Maptek333https://www.maptek.com/, was captured with a system equipped with a high precision INS (refer to [23] for system’s details). Mounted on a truck, a forward looking camera captured a video together with inertial measurements providing the ground truth for the camera poses.

Experiments were run on a PC with a quad-core 2.5GHz Intel core i7 CPU and 16GB of RAM. We implemented L-infinity SLAM and BA-SLAM in MATLAB with the following optimisation routines:

• •

BA: implemented in C++ using the Ceres solver [41].

• •

rotation_averaging: code provided in [33].

• •

known_rotation_prob: code provided in [39].

• •

krot_tdc: $(P_{3})$ implemented in MATLAB using SeDuMi [42].

IV-A Results for the Maptek dataset

We sampled the full sequence (1833 frames) into 358 keyframes. We detected the occurrence of a loop by using provided ground truth in both BA-SLAM and L-infinity SLAM. Since the moving camera describes a two-loop sequence (see the ground truth in Fig. 2), after completing the first loop (at frame 790), a loop is detected for each consecutive keyframe. To solve loop closure in L-infinity SLAM, we fed krot_tdc with 300 uniformly sampled feature tracks (we used the same sample size for loop closure in BA-SLAM). krot_tdc accurately solved loop closure (see Fig 1) in 90.54 s in a MATLAB single-thread implementation. Since its quasi-convex nature, krot_tdc does not have to be invoked at each loop detection; here we invoked krot_tdc at the last keyframe only. On the contrary, BA-SLAM could be incapable of fixing the drift produced if invocations of BA are skipped (e.g., failing in detecting a loop on a real-world system). We set a window size equal to 10.

To compare BA-SLAM and L-infinity SLAM, Fig. 1 plots the camera position error and camera rotation error for both methods. BA-SLAM was unable to complete the sequence –the camera got disconnected at frame 1356, i.e., no feature track existed for the visited keyframe. This disconnection was a consequence of the outlier removal heuristic used for BA-SLAM (removing a feature track if the distance of the camera position to any visible 3D point is above a threshold; 150 m for this experiment) which failed by eliminating inliers when BA was unable to produce an accurate result. We observed that BA is prone to fail on reduced data input, hence BA-SLAM needs to keep a large number of feature tracks. The $l_{\infty}$ optimisation approach of L-infinity SLAM admits less risky outlier removal strategies (e.g. eliminating the support set) without the need of keeping a large number of tracks. As result, BA-SLAM achieved lower camera position ($<0.67~{}m$) and camera rotation ($<$$$) errors than BA-SLAM.

We also ran BA after krot_tdc on same feature tracks. The camera position and rotation error tend to be lowered; however, BA produced significant error in several camera poses as depicted in Fig. 1.

IV-A1 krot_tdc vs the camera recovering method in [40]

We ran the method in [40] with the same relative translation directions $\mathbf{t}_{j,k}^{(E)}$ and camera rotations used to solve loop closure. As depicted in Fig. 3, recovering camera positions is unachievable from $\mathbf{t}_{j,k}^{(E)}$ measurements only. In addition, Fig. 4 shows known_rotation_prob failed on producing an accurate result when solving loop closure on the same tracks we used with krot_tdc.

IV-A2 Map reconstruction

Since we reconstructed 300 scene points only when solving loop closure, we used the quasi-convex method in [39] to triangulate all scene points. Fig. 6 shows the scene reconstruction and the camera positions.

IV-B Low speed and high rotational motion

We tested L-Infinity SLAM against ORB-SLAM with a video recorded with a smart-phone by a pedestrian while turning right; Fig. 5a displays two frames of the video. ORB-SLAM failed to build an initial map (see Fig. 5c), arguably, since the low baseline of frames from the walking speed sequence with rotational motion (see camera poses in Fig. 5b). Unlike ORB-SLAM, the quasi-convex formulation of L-Infinity SLAM does not required any initial map to track camera motions. As result, only L-infinity SLAM produced a reconstruction (see Fig. 5b).

IV-C Runtime of online routines

To compare the efficiency L-infinity SLAM against BA-SLAM for rotation only camera motions, we measured the runtime of incremental rotation averaging and incremental BA, which are the fundamental optimisation routines for this problem. We used a window size equal to 10 and plotted the runtimes for the first loop of the Maptek dataset in Fig. 7 (in log scale). Rotation averaging is an order of magnitude faster than BA which indicates the superior efficiency of L-infinity SLAM over BA-SLAM for rotation only problems.

V CONCLUSIONS

We presented L-infinity SLAM to be a simpler alternative to SLAM systems based on bundle adjustment. Driven by globally optimal quasi-convex optimisation, there is no need to maintain an accurate map and camera motions at key-frame rate as demanded by systems based on bundle adjustment. Instead, the online effort is devoted to efficiently estimating camera orientations through rotation averaging. To efficiently solve loop closure, we proposed a variant of the known rotation problem which incorporates relative translation directions to accurately solve camera drifts when optimising over a sample of feature tracks. Also, L-infinity SLAM is a simple and efficient alternative for applications requiring estimating slow motions or only rotational motions. We hope L-infinity SLAM can motivate future research on quasi-convex optimisation in the SLAM community.

ACKNOWLEDGEMENT

This work was supported by ARC Grants LP140100946 and CE140100016.