Dynamic Matrix Inverse: Improved Algorithms and Matching Conditional Lower Bounds

TL;DR

This paper introduces improved algorithms for dynamic matrix inverse, explores their tightness under conjectures, and applies these results to enhance algorithms for various graph and matrix problems, advancing understanding of their computational complexity.

Contribution

The paper presents new algorithms for dynamic matrix inverse and its variants, and establishes conjectural lower bounds, advancing the understanding of their computational limits and applications.

Findings

Faster algorithms for dynamic matrix inverse and related problems.

Conditional lower bounds based on the Online Matrix-Vector conjecture.

Applications to improved dynamic algorithms for graph problems like reachability and maximum matching.

Abstract

The dynamic matrix inverse problem is to maintain the inverse of a matrix undergoing element and column updates. It is the main subroutine behind the best algorithms for many dynamic problems whose complexity is not yet well-understood, such as maintaining the largest eigenvalue, rank and determinant of a matrix and maintaining reachability, distances, maximum matching size, and $k$-paths/cycles in a graph. Understanding the complexity of dynamic matrix inverse is a key to understand these problems. In this paper, we present (i) improved algorithms for dynamic matrix inverse and their extensions to some incremental/look-ahead variants, and (ii) variants of the Online Matrix-Vector conjecture [Henzinger et al. STOC'15] that, if true, imply that these algorithms are tight. Our algorithms automatically lead to faster dynamic algorithms for the aforementioned problems, some of which are also tight under our conjectures, e.g. reachability and maximum matching size (closing the gaps for these two problems was in fact asked by Abboud and V. Williams [FOCS'14]). Prior best bounds for most of these problems date back to more than a decade ago [Sankowski FOCS'04, COCOON'05, SODA'07; Kavitha FSTTCS'08; Mucha and Sankowski Algorithmica'10; Bosek et al. FOCS'14]. Our improvements stem mostly from the ability to use fast matrix multiplication ``one more time'', to maintain a certain transformation matrix which could be maintained only combinatorially previously (i.e. without fast matrix multiplication). Oddly, unlike other dynamic problems where this approach, once successful, could be repeated several times (``bootstrapping''), our conjectures imply that this is not the case for dynamic matrix inverse and some related problems. However, when a small additional ``look-ahead'' information is provided we can perform such repetition to drive the bounds down further.

Figures

Equations

$$A^{(t)}=A^{(0)}{T}^{(0,t)}$$

$$(A^{(t)}{)}^{-1}=(T^{(0,t)}{)}^{-1}(A^{(0)}{)}^{-1},$$

$$T^{(0,t)}=T^{(0,t-1)}{T}^{(t-1,t)}$$

$$T^{(0,t)}$$

$$(T^{(0,t)}{)}^{-1}$$

$$T^{(0,t)}$$

$$(T^{(0,t)}{)}^{-1}$$

$$T^{(0,t)}=T^{(0,t^{\prime })}{T}^{(t^{\prime },t)},\text{mbox}thus(T^{(0,t)}{)}^{-1}=(T^{(t^{\prime },t)}{)}^{-1}(T^{(0,t^{\prime })}{)}^{-1}.$$

$$T^{(0,t)}=T^{(0,t-1)}+e_j^{\top }v,\text{mbox}thus(by\text{eqref}eq:TransformViaTransform)T^{(t^{\prime },t)}=T^{(t^{\prime },t-1)}+e_j^{\top }[(T^{(0,t^{\prime })}{)}^{-1}v].$$

$$\displaystyle T^{(t^{\prime},t)}=\left(\begin{array}[]{c|c}C_{1}&0\\ \hline\cr C_{2}&\mathbb{I}\end{array}\right)\hskip 50.0pt(T^{(t^{\prime},t)})^{-1}=\left(\begin{array}[]{c|c}C_{1}^{-1}&0\\ \hline\cr-C_{2}C_{1}^{-1}&\mathbb{I}\end{array}\right)$$

$$S^{(t^{\prime },t)}:=T^{(0,t)}-T^{(0,t^{\prime })}.$$

$$T^{(0,t)}=T^{(0,t^{\prime })}+S^{(t^{\prime },t)},\text{mbox}thus(by\text{eqref}eq:TransformViaTransform)T^{(t^{\prime },t)}=\mathbb{I}+(T^{(0,t^{\prime })}{)}^{-1}{S}^{(t^{\prime },t)}.$$

$$A^{(t)}=A^{(0)}{T}^{(0,t^{\prime })}{T}^{(t^{\prime },t)},\text{mbox}thus(A^{(t)}{)}^{-1}=(T^{(t^{\prime },t)}{)}^{-1}(T^{(0,t^{\prime })}{)}^{-1}(A^{(0)}{)}^{-1}.$$

$$A^{(t_1)}\left[\mathbb{I}+(A^{(t_1)}{)}^{-1}(A^{(t_2)}-A^{(t_1)})\right]=A^{(t_1)}+(A^{(t_2)}-A^{(t_1)})=A^{(t_2)}$$

$$T^{(t^{\prime },t)}$$

$$=\mathbb{I}+(A^{(t^{\prime })}{)}^{-1}(A^{(t)}-A^{(t-1)}+A^{(t-1)}-A^{(t^{\prime })})$$

$$=(A^{(t^{\prime })}{)}^{-1}(A^{(t)}-A^{(t-1)})+\mathbb{I}+(A^{(t^{\prime })}{)}^{-1}(A^{(t-1)}-A^{(t^{\prime })})$$

$$=(A^{(t^{\prime })}{)}^{-1}(A^{(t)}-A^{(t-1)})+T^{(t^{\prime },t-1)}$$

$$M^{(t)}=\left(\begin{array}{cc}\mathbb{I}& 0\\ 0& M_{I^{(t)},I^{(t)}}^{(t)}\end{array}\right)(M^{(t)}{)}^{-1}=\left(\begin{array}{cc}\mathbb{I}& 0\\ 0& (M_{I^{(t)},I^{(t)}}^{(t)}{)}^{-1}\end{array}\right)$$

$$T^{(t^{\prime },t)}$$

$$\forall 0\le i\le l,0\le t:$$

$$\forall t_1,t_2,i_1,i_2\text{ such that }{t}_1\le t_2\le t_2+i_2\le t_1+i_2:$$

$$\omega :=\inf \{s\mid \text{we can multiply two n×n matrices in }O(n^s)\text{ arithmetic operations}\}$$

$$\displaystyle\left(\begin{array}[]{cccccc}\mathbb{I}&A_{1}&0&\cdots&\cdots&0\\ 0&\mathbb{I}&A_{2}&0&&\vdots\\ \vdots&\ddots&\ddots&\ddots&\ddots&\vdots\\ \vdots&&0&\mathbb{I}&A_{s-1}&0\\ \vdots&&&0&\mathbb{I}&A_{s}\\ 0&\cdots&\cdots&\cdots&0&\mathbb{I}\end{array}\right)$$

$$\displaystyle\left(\begin{array}[]{cccccc}\mathbb{I}&-A_{1}&A_{1}A_{2}&-A_{1}A_{2}A_{3}&\cdots&\prod_{i=1}^{s}-A_{i}\\ 0&\mathbb{I}&-A_{2}&A_{2}A_{3}&-A_{2}A_{3}A_{4}&\vdots\\ \vdots&\ddots&\ddots&\ddots&\ddots&\vdots\\ \vdots&&0&\mathbb{I}&-A_{s-1}&A_{s-1}A_{s}\\ \vdots&&&0&\mathbb{I}&-A_{s}\\ 0&\cdots&\cdots&\cdots&0&\mathbb{I}\end{array}\right)$$

$$\det (A+u{v}^{\top })=\det (A)(1+v^{\top }{A}^{-1}u).$$

$$\displaystyle\left(\begin{array}[]{cc}Q&R\\ S&T\end{array}\right)^{-1}=\left(\begin{array}[]{cc}Q^{-1}+Q^{-1}R(T-SQ^{-1}R)^{-1}SQ^{-1}&-Q^{-1}R(T-SQ^{-1}R)^{-1}\\ -(T-SQ^{-1}R)^{-1}SQ^{-1}&(T-SQ^{-1}R)^{-1}\end{array}\right)$$

$$\displaystyle\tilde{A}=\left(\begin{array}[]{ccc}A&X&0\\ Y&0&\mathbb{I}\\ 0&\mathbb{I}&\mathbb{I}^{(k)}\end{array}\right)$$

$$T^{(t-1,t)}=\mathbb{I}+(A^{(t-1)}{)}^{-1}(A^{(t)}-A^{(t-1)})$$

$$\mathcal{I}$$

$$\displaystyle\tilde{A}:=\left(\begin{array}[]{cccccc}\mathbb{I}&A_{1}&0&\cdots&\cdots&0\\ 0&\mathbb{I}&A_{2}&0&&\vdots\\ \vdots&\ddots&\ddots&\ddots&\ddots&\vdots\\ \vdots&&0&\mathbb{I}&A_{s-1}&0\\ \vdots&&&0&\mathbb{I}&A_{s}\\ 0&\cdots&\cdots&\cdots&0&\mathbb{I}\end{array}\right)$$

$$\displaystyle\tilde{A}:=\left(\begin{array}[]{cccccc}\mathbb{I}&A^{(1,1)}&A^{(1,2)}&\cdots&\cdots&A^{(1,k)}\\ 0&\mathbb{I}&A^{(2,2)}&&&\vdots\\ \vdots&\ddots&\ddots&\ddots&&\vdots\\ \vdots&&0&\mathbb{I}&A^{(s-1,s-1)}&A^{(s-1,s)}\\ \vdots&&&0&\mathbb{I}&A^{(s,s)}\\ 0&\cdots&\cdots&\cdots&0&\mathbb{I}\end{array}\right)$$

$$\displaystyle\tilde{B}:=\left(\begin{array}[]{cccccc}\mathbb{I}&B^{(1,1)}&B^{(1,2)}&\cdots&\cdots&B^{(1,k)}\\ 0&\mathbb{I}&B^{(2,2)}&&&\vdots\\ \vdots&\ddots&\ddots&\ddots&&\vdots\\ \vdots&&0&\mathbb{I}&B^{(s-1,s-1)}&B^{(s-1,s)}\\ \vdots&&&0&\mathbb{I}&B^{(s,s)}\\ 0&\cdots&\cdots&\cdots&0&\mathbb{I}\end{array}\right)$$

Full text

1. 1 Introduction

2. 1.1 Our Algorithmic Results (Details in Sections 4, C and 6)

3. 1.2 Our Conditional Lower Bounds (Details in Section 5)

4. 1.3 Other Related Work

5. 2 Overview of Our Algorithms

6. 3 Preliminaries

7. 4 Dynamic Matrix Inverse

8. 4.1 Transformation Maintenance implies Dynamic Matrix Inverse

9. 4.2 Explicit Transformation Maintenance

10. 4.3 Implicit Transformation Maintenance

11. 4.4 Combining the Transformation Maintenance Algorithms

12. 4.5 Applications

13. 5 Conditional Lower Bounds

14. 5.1 Column Update, Row Query

15. 5.2 Element Update, Row Query

16. 5.3 Element Update, Element Query

17. 5.4 Discussion on Super-Linear Bounds for Dynamic Matrix Inverse

18. 6 Look-Ahead Setting

19. 6.1 Short Sequence of Updates

20. 6.2 Long Sequence of Updates

21. 7 Open Problems

22. A Runtime Analysis, Balancing Terms

23. B Worst-Case Standard Technique

24. C Applications

25. C.1 Algebraic Applications

26. C.2 Polynomial Matrices

27. C.3 Graph Applications

28. D Bit-Length Increase for the Matrix Product Reduction

29. E Lower Bounds

30. E.1 Lower Bounds assuming $o(n^{\omega})$ pre-processing

31. E.2 Implications of Mv and OMv results

32. E.3 OMv-based Lower Bounds

33. E.4 Amortized Lower Bounds

34. E.5 Column Update Induces $\Omega(nt)$ Changes in the Inverse

1 Introduction

In the dynamic matrix inverse problem, we want to maintain the inverse of an $n\times n$ matrix $A$ over any field, when $A$ undergoes some updates. There were many variants of this problem considered [San04, San07, LS15, CLS18]: Updates can be element updates, where we change the value of one element in $A$, or column updates, where we change the values of all elements in one column.111There are other kinds of updates which we do not consider in this paper, such as rank-1 updates in [LS15, CLS18]. The inverse of $A$ might be maintained explicitly or might be answered through an element query or a row/column query; the former returns the value of a specified element of the inverse, and the latter answers the values of all elements in a specified row/column of the inverse. The goal is to design algorithms with small update time and query time, denoting the time needed to handle each update and each query respectively. Time complexity is measured by the number of field operations.222 Later when we consider other kinds of dynamic problems, such as dynamic graphs, the time refer to the standard notion of time in the RAM model. Variants where elements are polynomials and where some updates are known ahead of time (the look-ahead setting) were also considered (e.g. [SM10, Kav14, KMW98, Yan90]).

Dynamic matrix inverse algorithms played a central role in designing algorithms for many dynamic problems such as maintaining matrix and graph properties. Its study can be traced back to the 1950 algorithm of Sherman and Morrison [SM50] which can be used to maintain the inverse explicitly in $O(n^{2})$ time. The previous best bounds are due to Sankowski’s FOCS’04 paper [San04] and its follow-ups [San07, SM10, San05]. Their time guarantees depend on how fast we can multiply matrices. For example, with the state-of-the-art matrix multiplication algorithms [GU18, Gal14], Sankowski’s algorithm [San04] can handle an element update and answer an element query for matrix inverse in $O(n^{1.447})$ time. Consequently, the same update time333See Footnote 2. can be guaranteed for, e.g., maintaining largest eigenvalues, ranks and determinants of matrices undergoing entry updates and maintaining maximum matching sizes, reachability between two nodes ($st$-reachability), existences of a directed cycle, numbers of spanning trees, and numbers of paths (in directed acyclic graphs; DAG) in graphs undergoing edge insertions and deletions.444We note that while the update and query time for the matrix inverse problem is defined to be the number of arithmetic operations, most of time the guarantees translate into the same running time in the RAM model. Exceptions are the numbers of spanning trees in a graph and numbers of paths in a DAG, where the output might be a very big number. In this case the running time is different from the number of arithmetic operations. (Unless specified otherwise, all mentioned update times are worst-case (as opposed to being amortized555Amortized time is not the focus of this paper, and we are not aware of any better amortized bounds for problems we consider in this paper).) See Sections 1.1 and 2 for lists of known results for dynamic matrix inverse and Figures 4 and 5 for lists of applications.

Is the $O(n^{1.447})$ bound the best possible for above problems? This kind of question exhibits the current gap between existing algorithmic and lower bound techniques and our limited understanding of the power of algebraic techniques in designing dynamic algorithms. First of all, despite many successes in the last decade in proving tight bounds for a host of dynamic problems (e.g. [HKN+15, AW14, Pat10]), conditional lower bounds for most of these problems got stuck at $\Omega(n)$ in general. Even for a very special case where the preprocessing time is limited to $o(n^{\omega})$ (which is too limited as discussed in Section 1.3), the best known conditional lower bound of $\Omega(n^{\omega-1})=\Omega(n^{1.3728639})$ [AW14] is still not tight ([AW14] mentioned that “closing this gap is a very interesting open question”). Note that while the upper bounds might be improved in the future with improved rectangular matrix multiplication algorithms, there will still be big gaps even in the best-possible scenario: even if there is a linear-time rectangular matrix multiplication algorithm, the upper bounds will still be only $O(n^{1+1/3})$, while the lower bound will be $\Omega(n)$.

Secondly, it was shown that algebraic techniques – techniques based on fast matrix multiplication algorithms initiated by Strassen [Str69] – are inherent in any upper bound improvements for some of these problems: Assuming the Combinatorial Boolean Matrix Multiplication (BMM) conjecture, without algebraic techniques we cannot maintain, e.g., maximum matching size and $st$-reachability faster than $O(n^{2})$ per edge insertion/deletion [AW14]666More precisely, assuming BMM, no “combinatorial” algorithm can maintain maximum matching size and $st$-reachability in $O(n^{2-\epsilon})$ time, for any constant $\epsilon>0$. Note that “combinatorial” a vague term usually used to refer as an algorithm that does not use subcubic-time matrix multiplication algorithms as initiated by Strassen [Str69]. We note that this statement only holds for algorithms with $O(n^{3-\epsilon})$ preprocessing time, which are the case for Sankowski’s and our algorithms.. Can algebraic techniques lead to faster algorithms that may ideally have update time linear in $n$? If not, how can we argue lower bounds that are superlinear in $n$ and, more importantly, match upper bounds from algebraic algorithms?

In this paper, we show that it is possible to improve some of the existing dynamic matrix inverse algorithms further and at the same time present conjectures that, if true, imply that they cannot be improved anymore.

1.1 Our Algorithmic Results (Details in Sections 4, C and 6)

Algorithms in the Standard Setting (Details in Sections 4 and C).

We present two faster algorithms as summarized in Section 1.1. With known fast matrix multiplication algorithms [GU18, Gal14], our first algorithm requires $O(n^{{1.407}})$ time to handle each entry update and entry query, and the second requires $O(n^{{1.529}})$ time to handle each column update and row query.