Spectral theory for non-unitary twists

TL;DR

This paper develops a spectral theory framework for non-unitary twists of representations of Lie groups, establishing a trace formula applicable to a broad class of locally compact groups.

Contribution

It introduces a complete filtration for the $G$-representation on $L^2( ext{quotient space}, ext{twist})$ with irreducible quotients, extending spectral analysis to non-unitary cases.

Findings

Established a complete filtration with irreducible quotients.

Proved a trace formula for non-unitary twists.

Extended spectral theory to arbitrary locally compact groups.

Abstract

Let $G$ be a Lie-group and $\Ga\subset G$ a cocompact lattice. For a finite-dimensional, not necessarily unitary representation $\om$ of $\Ga$ we show that the $G$-representation on $L^2(\Ga\bs G,\om)$ admits a complete filtration with irreducible quotients. As a consequence, we show the trace formula for non-unitary twists and arbitrary locally compact groups.

Equations

$$H\cong \underset{\begin{array}{c}\leftarrow \\ N\end{array}}{\lim }H/N,$$

$$\Gamma x_i{U}^2\cap \Gamma x_j{U}^2=\varnothing$$

$${\varphi }_j=c_j{1}_{\Gamma x_j{V}^2},$$

$$c_j{1}_{\Gamma x_{j}V}\le R(f){\varphi }_j.$$

$$1={\Vert {\varphi }_j\Vert }^2=c_j^2\mathrm{vol}(\Gamma \backslash \Gamma x_j{V}^2)\le n{c}_j^2\mathrm{vol}(\Gamma \backslash \Gamma x_{j}V)$$

$${\Vert R(f){\varphi }_j\Vert }^2\ge {\Vert c_j{1}_{\Gamma x_{j}V}\Vert }^2\ge \frac{c_j^2}{n}\mathrm{vol}(\Gamma x_j{V}^2)=\frac{1}{n}.$$

$$\Gamma =\mathcal{O}⋊{\mathcal{O}}^1.$$

$${\mathrm{Hom}}_G(V_{\pi },L^2(\Gamma \backslash G)$$

$$V(\Delta ,\sigma )=\bigcup _{n\in \mathbb{N}}\ker (\Delta -\sigma {)}^n\subset V^{\infty }$$

$$v=\sum _{\sigma \in \mathrm{Spec}(\Delta )}{v}_{\sigma },$$

$$V(\Delta ,{\sigma }_0{)}^{\prime }=\overline{\underset{\sigma \ne {\sigma }_0}{⨁}V(\Delta ,\sigma )}$$

$$TR(\Delta -\lambda )v=R(\Delta -\lambda )Tv=v$$

$$T=({\sigma }_0-\lambda )(\Delta -\lambda {)}^{-1}$$

$$(\Delta -\lambda {)}^{-1}$$

$$=(\lambda -{\sigma }_0)(\Delta -\lambda {)}^{-1}(\Delta -{\sigma }_0{)}^{-1}+(\Delta -{\sigma }_0{)}^{-1}.$$

$$\Vert (\Delta -\lambda {)}^{-1}\Vert \le \frac{\Vert (\Delta -{\sigma }_0{)}^{-1}\Vert }{1-({\sigma }_0-\lambda )\Vert (\Delta -{\sigma }_0{)}^{-1}\Vert },$$

$$T=\left(1+\sum _{j=1}^{d-1}\frac{S^j}{({\sigma }_0-\lambda {)}^j}\right)=\left(1+N\right)$$

$$T^n=(1+N{)}^n=\sum _{k=0}^{d-1}(\genfrac{}{}{0ex}{}{n}{k})N^k.$$

$$F_1^{\prime }\subset F_1\subset F_2^{\prime }\subset F_2\subset \cdots \subset F_l^{\prime }\subset F_l$$

$$S(\Delta ,\lambda )\cong V_1(\Delta ,\lambda )/V_0(\Delta ,\lambda ).$$

$$\mathrm{tr}R(f)=\sum _{\pi \in \tilde{G}}{N}_{\Gamma ,\omega }(\pi )\mathrm{tr}\pi (f).$$

$$(\Delta -\xi {)}^{-n}\underset{\in U}{\underbrace{(\Delta -{\lambda }_0{)}^{n}v}}=(\Delta -\xi {)}^{-n}\sum _{\lambda }{w}_{\lambda }=\sum _{\lambda }(\Delta -\xi {)}^{-n}{w}_{\lambda }\in U.$$

$$v=(\Delta -\xi {)}^{-n}(\Delta -\xi {)}^{n}v=\sum _{\lambda }(\Delta -\xi {)}^{-n}{w}_{\lambda }^{\xi },$$

$$F_1^{\prime }\subset F_1\subset F_2^{\prime }\subset F_2\subset \cdots \subset F_l^{\prime }\subset F_l$$

$$\sum _{\pi \in \tilde{G}}{N}_{\Gamma ,\omega }(\pi )\mathrm{tr}\pi (f)=\sum _{[\gamma ]}\mathrm{vol}({\Gamma }_{\gamma }\backslash G_{\gamma }){\mathcal{O}}_{\gamma }(f)\mathrm{tr}\omega (\gamma ),$$

$${\mathcal{O}}_{\gamma }(f)={\int }_{G_{\gamma }\backslash G}f(x^{-1}\gamma x)dx.$$

$$\sum _{[\gamma ]}\mathrm{vol}({\Gamma }_{\gamma }\backslash G_{\gamma }){\mathcal{O}}_{\gamma }(f)\mathrm{tr}\omega (\gamma ).$$

$$G=\underset{\begin{array}{c}\leftarrow \\ N\end{array}}{\lim }G/N.$$

$$0=F_0\subset \cdots \subset F_k=V$$

$${\Gamma }^{\infty }(E)\cong (C^{\infty }(G)\otimes V_{\chi }{)}^{\Gamma }.$$

$${\Gamma }^{\infty }(E)(\tau )\cong V_{\tau }\otimes {\mathrm{Hom}}_K(V_{\tau },{\Gamma }^{\infty }(E)),$$

Full text

1. 1 Trace class representations
2. 2 The spectral filtration
3. 3 Admissible representations
4. 4 The spectral theorem
5. 5 Semisimple Lie groups

Introduction

For unitary representations of locally compact groups there is a general spectral theory, expressing such representations as direct integrals of irreducibles or even, if the representation is, say, trace class, as direct sums of irreducibles. For non-unitary representations there is no spectral theory in general. In this paper we introduce a spectral theory for representations, non-unitarily induced from cocompact lattices. These representations occur naturally in extensions of the trace formula [mueller]. In this paper we use the spectral analysis of the group Laplacian to deduce that for a Lie group these representations admit complete filtrations with irreducible graded steps.

1 Trace class representations

Let $G$ be a locally compact group. For the convenience of the reader we briefly recall the definition of the space $C_{c}^{\infty}(G)$ of test functions on $G$.

Definition 1.1**.**

First, if $L$ is a Lie group, then $C_{c}^{\infty}(L)$ is defined as the space of all infinitely differentiable functions of compact support on $L$. The space $C_{c}^{\infty}(L)$ is the inductive limit of all $C_{K}^{\infty}(L)$, where $K\subset L$ runs through all compact subsets of $L$ and $C_{K}^{\infty}(L)$ is the space of all smooth functions supported inside $K$. The latter is a Fréchet space equipped with the supremum norms over all derivatives. Then $C_{c}^{\infty}(L)$ is equipped with the inductive limit topology in the category of locally convex spaces as defined in [Schaef], Chap II, Sec. 6.

Next, suppose the locally compact group $H$ has the property that $H/H^{0}$ is compact, where $H^{0}$ is the connected component. Let $\mathcal{N}$ be the family of all normal closed subgroups $N\subset H$ such that $H/N$ is a Lie group with finitely many connected components. We call $H/N$ a Lie quotient of $H$. Then, by [MZ], the set $\mathcal{N}$ is directed by inverse inclusion and

[TABLE]

where the inverse limit runs over the set $\mathcal{N}$. So $H$ is a projective limit of Lie groups. The space $C_{c}^{\infty}(H)$ is then defined to be the sum of all spaces $C_{c}^{\infty}(H/N)$ as $N$ varies in $\mathcal{N}$. Then $C_{c}^{\infty}(H)$ is the inductive limit over all $C_{c}^{\infty}(L)$ running over all Lie quotients $L$ of $H$ and so $C_{c}^{\infty}(H)$ again is equipped with the inductive limit topology in the category of locally convex spaces.

Finally to the general case. By [MZ] one knows that every locally compact group $G$ has an open subgroup $H$ such that $H/H^{0}$ is compact, so $H$ is a projective limit of connected Lie groups in a canonical way. A Lie quotient of $H$ then is called a local Lie quotient of $G$. We have the notion $C_{c}^{\infty}(H)$ and for any $g\in G$ we define $C_{c}^{\infty}(gH)$ to be the set of functions $f$ on the coset $gH$ such that $x\mapsto f(gx)$ lies in $C_{c}^{\infty}(H)$. We then define $C_{c}^{\infty}(G)$ to be the sum of all $C_{c}^{\infty}(gH)$, where $g$ varies in $G$. Then $C_{c}^{\infty}(G)$ is the inductive limit over all finite sums of the spaces $C_{c}^{\infty}(gH)$. Note that the definition is independent of the choice of $H$, since, given a second open group $H^{\prime}$, the support of any given $f\in C_{c}(G)$ will only meet finitely many left cosets $gH^{\prime\prime}$ of the open subgroup $H^{\prime\prime}=H\cap H^{\prime}$. It follows in particular, that $C_{c}^{\infty}(G)$ is the inductive limit over a family of Fréchet spaces. This concludes the definition of the space $C_{c}^{\infty}(G)$ of test functions.

Remark 1.2**.**

1. (a)

Note that the inductive limit topology in the category of locally convex spaces differs from the inductive limit topology in the category of topological spaces, as is made clear in [Gloeck]. 2. (b)

Note that for a linear functional $\alpha:C_{c}^{\infty}(G)\to{\mathbb{C}}$ to be continuous, it suffices, that for any local Lie quotient $L$ of $G$ and any compact subset $K\subset L$ and any sequence $f_{n}\in C_{K}^{\infty}(L)$ with $f_{n}\to 0$ in the Fréchet space $C_{K}^{\infty}(L)$ and every $g\in G$ the sequence $\alpha(L_{g}f_{n})$ tends to zero, where $L_{g}f(x)=f(g^{-1}x)$. This is deduced from [Schaef], Chap II, Sec. 6.1. 3. (c)

If a locally compact group $G$ is a projective limit of Lie groups $G_{j}=G/N_{j}$, then it follows from [Neeb]*Cor. 12.3, that every irreducible continuous representation $\pi$ factors through some $G_{j}$. This reduces many issues related to distribution characters to the case of Lie groups.

Definition 1.3**.**

A representation $(\pi,V_{\pi})$ of a locally compact group $G$ is called a compact representation, if $\pi(f)$ is a compact operator for every $f\in C_{c}^{\infty}(G)$. It is called a trace class representation, if $\pi(f)$ is trace class for every $f\in C_{c}^{\infty}(G)$. We say that $G$ is a trace class group, if every irreducible unitary representation is trace class. See [DvD] for more on trace class groups.

Definition 1.4**.**

Let $G$ be a unimodular locally compact group and let $\Gamma\subset G$ be a discrete subgroup. Then there exists a non-vanishing, $G$-invariant Radon measure on the quotient $\Gamma\backslash G$, which is unique up to scaling [HA2] and the induced representation $R_{g}\phi(x)=\phi(xg)$ of $G$ on the Hilbert space $L^{2}(\Gamma\backslash G)$ is unitary.

Note that if $G$ admits a cocompact discrete subgroup $\Gamma$, then $G$ is unimodular and $\Gamma$ is a lattice.

Proposition 1.5**.**

For a unimodular locally compact group $G$ and a discrete subgroup $\Gamma\subset G$ the following are equivalent:

1. (a)

$\Gamma\backslash G$* is compact.* 2. (b)

The representation of $G$ on $L^{2}(\Gamma\backslash G)$ is trace class. 3. (c)

The representation of $G$ on $L^{2}(\Gamma\backslash G)$ is compact.

Proof.

(a)$\Rightarrow$(b) is the classical trace formula argument and can be found in [HA2], Chapter 9.

(b)$\Rightarrow$(c) is trivial.

(c)$\Rightarrow$(a): Assume that $L^{2}(\Gamma\backslash G)$ is a compact representation, but $\Gamma\backslash G$ is not compact. Then for every compact unit-neighborhood $U\subset G$ and every compact set $K\subset G$ there exists $x\in G$ such that $\Gamma xU\cap\Gamma K=\emptyset$, for otherwise the element $\Gamma x$ of $\Gamma\backslash G$ lies in the compact set $\Gamma\backslash\Gamma KU^{-1}$.

Applying this iteratedly, one obtains a sequence $x_{1},x_{2},\dots\in G$ such that

[TABLE]

for all $i\neq j$. Fix a symmetric unit-neighborhood $V$ such that $V^{3}\subset U$ and let

[TABLE]

where $c_{j}>0$ is such that $\left\|{\phi_{j}}\right\|^{2}=\int_{\Gamma\backslash G}|\phi_{j}(x)|^{2}\,dx=1$. Fix some $f\in C_{c}^{\infty}(G)$ with support in $V$ and such that $f\geq 0$ and $\int_{G}f(x)\,dx=1$. Now $\operatorname{supp}(R(f)\phi_{j})\subset\Gamma x_{j}V^{3}\subset\Gamma x_{j}U$ and therefore the supports of $R(f)\phi_{j}$ for varying $j$ are disjoint, hence these vectors in the Hilbert space $L^{2}(\Gamma\backslash G)$ are pairwise orthogonal. As $R(f)$ is a compact operator, the sequence $(R(f)\phi_{j})$ must have a convergent subsequence, but as the vectors are pairwise orthogonal, there must exist a subsequence with $\left\|{R(f)\phi_{j_{k}}}\right\|\to 0$. Since the integral of $f$ is 1, we have

[TABLE]

Now assume that $V^{2}\subset\bigcup_{l=1}^{n}Vz_{l}$ with $z_{l}\in G$, then $\Gamma x_{j}V^{2}\subset\bigcup_{l=1}^{n}\Gamma x_{j}Vz_{l}$ and so

[TABLE]

and

[TABLE]

So there is no subsequence with $\left\|{R(f)\phi_{j_{k}}}\right\|\to 0$. ∎

Remark 1.6**.**

(Counterexample) In [DvD], last Remark of Section 4, it is asked, whether any locally compact group $G$ admitting a cocompact lattice must be trace class. We now give a counterexample. Let $G=\operatorname{M}_{2}({\mathbb{R}})\rtimes\operatorname{SL}_{2}({\mathbb{R}})$, where $\operatorname{M}_{2}({\mathbb{R}})$ is the space of real $2\times 2$ matrices. Let $D$ be a quaternion division algebra over ${\mathbb{Q}}$ which splits at infinity. Fix a splitting $D\hookrightarrow\operatorname{M}_{2}({\mathbb{R}})$ and thus consider $D$ a ${\mathbb{Q}}$-subalgebra of $\operatorname{M}_{2}({\mathbb{R}})$. Fix an order $\mathcal{O}\subset D$ (see [Reiner]), and let $\mathcal{O}^{1}\subset\mathcal{O}^{\times}$ be the subgroup of all elements of determinant 1. Set

[TABLE]

Since $\mathcal{O}$ is a cocompact lattice in $\operatorname{M}_{2}({\mathbb{R}})$ and $\mathcal{O}^{1}$ is a cocompact lattice in $\operatorname{SL}_{2}({\mathbb{R}})$, the group $\Gamma$ is a cocompact lattice in $G$.

Next we need to show that $G$ is not trace class. In [DvD], Proposition 1.9, it is shown that $H={\mathbb{R}}^{2}\rtimes\operatorname{SL}_{2}({\mathbb{R}})$ is not trace class. Let $N\subset G$ be the set of all elements of the form $(A,1)$, where the matrix $A$ has zeros in the first column. Then $N$ is closed and normal in $G$ and $G/N\cong H$. So the irreducible representation of $H$, which is not trace class, induces an irreducible representation of $G$, which is not trace class.

Remark 1.7**.**

It seems to be an open question whether for any lattice $\Gamma$ the spectral multiplicities in the discrete spectrum of $L^{2}(\Gamma\backslash G)$ are finite. In other words, let $\Gamma$ be a lattice in the locally compact group $G$ and let $(\pi,V_{\pi})\in\widehat{G}$ be an irreducible unitary representation, is it true that

[TABLE]

is finite-dimensional?

2 The spectral filtration

By a representation we shall mean a continuous representation on a Banach space.

Definition 2.1**.**

Let $L$ be a linearly ordered set. For $a<b$ in $L$ we consider the closed interval $[a,b]$ of all $x\in L$ with $a\leq x\leq b$. The elements $a<b$ are called neighbored, if $[a,b]=\{a,b\}$, i.e., if there is no element between them.

A linearly ordered set $C$ is called complete, if every subset of $C$ possesses a supremum and an infimum.

For a given linearly ordered set $L$ there is a uniquely defined completion $C$, which is a complete ordered set which contains $L$ as a substructure such that $L$ is dense in $C$ in the sense that every $c\in C$ is the supremum or the infimum of a subset of $L$.

Definition 2.2**.**

A sub-tower is a linearly ordered set $L$, such that every $x\in L$ has a neighbor. A tower is a linearly ordered set which is the completion of a sub-tower. In particular, a tower $L$ contains a minimum $\min(L)$ and a maximum $\max(L)$.

Definition 2.3**.**

Let $L$ be a tower. Let $(R,V)$ be a representation of a locally compact group $G$ on a Banach space $V$. A complete $L$-filtration on $(R,V)$ is a family of closed, $G$-stable subspaces $(F_{i})_{i\in L}$ such that the following hold:

1. (a)

$F_{\min(L)}=0$ and $F_{\max(L)}=V$, 2. (b)

if $i\leq j$, then $F_{i}\subset F_{j}$, 3. (c)

if $i<j$ are neighbored, then $F_{j}/F_{i}$ is irreducible, 4. (d)

if $b\in L$ has no lower neighbor, then $F_{b}$ is the closure of $\bigcup_{j<b}F_{j}$, 5. (e)

if $a$ has no upper neighbor, then $F_{a}=\bigcap_{j>a}F_{j}$.

Definition 2.4**.**

Let $(F_{j})_{j\in L}$ be a complete $L$-filtration of $(R,V)$ for the tower $L$. For a given irreducible representation $(\pi,V_{\pi})$ of $G$ we define the multiplicity $m_{L}(\pi)$ to be the number of pairs $i<j$ in $L$ such that the representation on $F_{j}/F_{i}$ is isomorphic to $\pi$. This multiplicity may be zero, a natural number, or infinity.

Definition 2.5**.**

Let $(R,V)$ be a representation. A subquotient of $(R,V)$ is a representation of the form $P/Q$, where $Q\subset P$ are closed, $G$-stable subspaces of $V$.

A representation $(R,V)$ is called discrete, if every subquotient has an irreducible subquotient. This means that for any two closed, $G$-stable subspaces $Q\subset P$ there exist closed, $G$-stable subspaces $Q\subset U\subset W\subset P$ such that $W/U$ is irreducible.

Lemma 2.6**.**

Let $(R,V)$ be a discrete representation of the locally compact group $G$.

1. (a)

There exists a complete filtration $\mathcal{F}$ of $V$ for some tower $L$. 2. (b)

If additionally the representation $R$ is trace class, the multiplicities are finite.

Proof.

(a) A filtration $\mathcal{F}$ of $(R,V)$, indexed by a sub-tower, is called admissible, if every $i\in L$ has at least one neighbor $j$ such that $F_{i}/F_{j}$ or $F_{j}/F_{i}$ respectively, is irreducible. We apply the Lemma of Zorn to the set of all admissible filtrations $\mathcal{F}$, where we say that $(\mathcal{F},L)\leq(\mathcal{F}^{\prime},L^{\prime})$ if $L$ is a subset of $L^{\prime}$ and the filtration steps of $\mathcal{F}$ and $\mathcal{F}^{\prime}$ agree on $L$. We get a maximal admissible filtration. We complete $L$ by Dedekind cuts. If $D\subset L$ is a Dedekind cut, i.e., a subset with the property $x<y\in D\Rightarrow x\in D$, then we set $F_{D}=\overline{\bigcup_{i\in D}F_{i}}$. If $D\in L$ already, i.e., there exists $\alpha\in L$ such that $D=\{i\in L:i\leq\alpha\}$, then $F_{D}=F_{\alpha}$, so this filtration extends $\mathcal{F}$. If $D$ has no neighbor, then it is not in $L$ and $F_{D}$ is the closure of $\bigcup_{j<D}F_{j}$ by definition. On the other hand, we have $F_{D}=\bigcap_{j>D}F_{j}$, since otherwise there would be an irreducible subquotient between $F_{D}$ and this intersection, which would contradict the maximality of $\mathcal{F}$. Next if $D$ has an upper neighbor, but no lower, then we get $F_{D}=\overline{\bigcup_{j<D}F_{j}}$ again by maximality and likewise, we get $F_{D}=\bigcap_{j>D}F_{j}$ if $D$ has a lower neighbor, but no upper. This shows that there exists a complete filtration.

(b) Assume the representation to be trace class. By choosing an orthonormal basis which is compatible with the filtration, we see that the trace of $R(f)$ equals the trace on the associated graded representation. This implies finiteness of the multiplicities. ∎

3 Admissible representations

In this section, we assume that $G$ is a Lie group. Choose a left-invariant metric on $G$ and let $\Delta$ denote the Laplace operator for this metric. We call such a $\Delta$ a group-Laplacian. Let $\mathfrak{g}_{\mathbb{R}}$ be the real Lie algebra of $G$ and $\mathfrak{g}$ its complexification. The universal enveloping algebra $U(\mathfrak{g})$ can be identified with the algebra of left-invariant differential operators on $G$, so $\Delta$ can be viewed as an element of $U(\mathfrak{g})$.

By a representation of $G$ we mean a group homomorphism $R:G\to\operatorname{GL}(V)$ to the group of invertible bicontinuous linear operators on some Banach space $V$ such that the map $G\times V\to V$, $(g,v)\mapsto\pi(g)v$ is continuous. The space of smooth vectors $V^{\infty}$ then is defined as the space of all $v\in V$ such that $G\to V$, $x\mapsto R(x)v$ is infinitely differentiable. The universal enveloping algebra $U(\mathfrak{g})$ acts on the dense subspace $V^{\infty}$ of smooth vectors.

Definition 3.1**.**

A representation $(R,V)$ of $G$ is called $\Delta$-admissible, if

1. (a)

there is a dense subset $\Lambda_{R}\subset{\mathbb{C}}$, such that for each $\lambda\in\Lambda_{R}$ the operator $R(\Delta-\lambda)^{-1}$ is defined and extends to a continuous operator on the space $V$. For every $G$-stable closed subspace $U\subset V$ one has $(\Delta-1)^{-1}U\subset U$, 2. (b)

for each $\sigma\in{\mathbb{C}}$ the generalized eigenspace

[TABLE]

is finite-dimensional, 3. (c)

the set $\operatorname{Spec}_{R}(\Delta)$ of all $\sigma\in{\mathbb{C}}$ with $V(\Delta,\sigma)\neq 0$ has no accumulation point in ${\mathbb{C}}$, 4. (d)

every $v\in V$ can be written as absolutely convergent sum

[TABLE]

each $v_{\sigma}\in V(\Delta,\sigma)$ is uniquely determined and the projection map $v\mapsto v_{\sigma}$ is continuous, 5. (e)

for every $\sigma_{0}\in\operatorname{Spec}(\Delta)$ the space

[TABLE]

satisfies $V=V(\Delta,\sigma_{0})\oplus V(\Delta,\sigma_{0})^{\prime}$ and the operator $\Delta-\sigma_{0}$ has a bounded inverse on $V(\Delta,\sigma_{0})^{\prime}$.

The condition (a) needs explaining: We request that there exists a continuous operator $T$ on $V$ which preserves $V^{\infty}$ as well as every $G$-stable closed subset and satisfies

[TABLE]

for every $v\in V^{\infty}$. We denote this operator by $R(\Delta-\lambda)^{-1}$.

We find it convenient to leave out the $R$ in the notation, so we occasionally write $(\Delta-\lambda)$ instead of $R(\Delta-\lambda)$ and the same for the inverses.

Lemma 3.2**.**

Let $(R,V)$ be $\Delta$-admissible and let $U\subset V$ be a closed $G$-stable subspace, then $U$ is $\Delta$-admissible.

Proof.

The only part of the definition which needs proving, is part (d). More precisely we need to show that if $u\in U$ and $u=\sum_{\sigma}u_{\sigma}$ is the spectral decomposition in $V$, then $u_{\sigma}\in U$ for every $\sigma$. For this let $\sigma_{0}\in\operatorname{Spec}_{R}(\Delta)$ and let $\lambda\in\Lambda_{R}$ be closer to $\sigma_{0}$ than any other $\sigma\in\operatorname{Spec}_{R}(\Delta)$. Then the operator

[TABLE]

has eigenvalue 1 on $V(\Delta,\sigma_{0})$ and eigenvalue of absolute value $<1$ on $V(\Delta,\sigma)$ for every $\sigma_{0}\neq\sigma\in\operatorname{Spec}_{R}(\Delta)$. We write $u=u_{\sigma_{0}}+u^{\sigma_{0}}$, where $u^{\sigma_{0}}=\sum_{\sigma\neq\sigma_{0}}u_{\sigma}$. We first show that $T^{n}u^{\sigma_{0}}$ tends to [math] as $n\to\infty$. For this note that on the space $V(\Delta,\sigma_{0})^{\prime}$ one has

[TABLE]

Taking operator norms on both sides and using the triangle inequality we infer that for small values of $|\sigma_{0}-\lambda|$ we have

[TABLE]

where we mean the operator norm on the space $V(\Delta,\sigma_{0})^{\prime}$. It follows that for $\lambda$ close enough to $\sigma_{0}$ the operator norm of $T$ on $V(\Delta,\sigma_{0})^{\prime}$ is $<1$, which implies that $T^{n}u^{\sigma_{0}}$ tends to zero.

On $V(\Delta,\sigma_{0})$ we write $\Delta=\sigma_{0}-S$ where $S$ is nilpotent. So

[TABLE]

where $N=\sum_{j=1}^{d-1}\frac{S^{j}}{(\sigma_{0}-\lambda)^{j}}$ is again nilpotent and $d=\dim V(\Delta,\sigma_{0})$. Then on $V(\Delta,\sigma_{0})$ we have

[TABLE]

it follows that $T^{n}\binom{n}{d-1}^{-1}$ tends to $N^{d-1}$ as $n\to\infty$, which implies that $N^{d-1}u_{\sigma_{0}}$ lies in $U$. Next $\left(T^{n}-\binom{n}{d-1}N^{d-1}\right)\binom{n}{d-2}^{-1}$ tends to $N^{d-2}$ which implies that $N^{d-2}u_{\sigma_{0}}$ lies in $U$. We repeat until we reach $N^{0}u_{\sigma_{0}}=u_{\sigma_{0}}\in U$ as claimed. ∎

Definition 3.3**.**

Let $(R,V)$ be a representation of the locally compact group $G$ and let $\pi$ be an irreducible representation of $G$. A $\pi$-filtration in $V$ is a sequence

[TABLE]

of closed, $G$-stable subspaces such that $F_{j}/F_{j}^{\prime}\cong\pi$ for each $j$.

Theorem 3.4** (Spectral theorem).**

Let $(R,V)$ be a $\Delta$-admissible representation of the Lie group $G$.

1. (a)

If $V_{0}\subset V_{1}$ are closed $G$-stable subspaces, then the sub-quotient $S=V_{1}/V_{0}$ is $\Delta$-admissible as well. Each spectral value $\lambda$ of $S$ is a spectral value of $V$, more precisely, one has

[TABLE]

If $m(S,\lambda)=m(V,\lambda)$ for all $\lambda$, then $S=V$. 2. (b)

Let $(R,V)$ be $\Delta$-admissible and $\pi$ an irreducible representation of $G$. Then all maximal $\pi$-filtrations have the same finite length. We call this length $N_{\Gamma,\omega}(\pi)\in{\mathbb{N}}_{0}$ the multiplicity of $\pi$ in $R$. 3. (c)

If $f\in C_{c}(G)$ is such that the operator $R(f)=\int_{G}f(x)R(x)\,dx$ is trace class, then $\pi(f)$ is trace class for every $\pi\in\widetilde{G}$ with $N_{\Gamma,\omega}(\pi)>0$ and one has

[TABLE] 4. (d)

The representation $(R,V)$ is discrete, so there exists a complete filtration on $(R,V)$.

Proof.

(a) A submodule is admissible, so it remains to show that a quotient is admissible. So let $U\subset V$ be a closed $G$-stable subspace. We claim that for $\lambda_{0}\in{\mathbb{C}}$ the map $V(\Delta,\lambda_{0})\to V/U$ induces an isomorphism $V(\Delta,\lambda_{0})/U(\Delta,\lambda_{0})\cong(V/U)(\Delta,\lambda_{0})$. The injectivity is clear. For the surjectivity let $v+U$ be in $(V/U)(\Delta,\lambda_{0})$, then $(\Delta-\lambda_{0})^{n}v\in U$ for some $n$. Write $v=\sum_{\lambda\in{\mathbb{C}}}v_{\lambda}$ as in the definition of admissibility. We claim that $v-v_{\lambda_{0}}$ lies in $U$. Let $\xi\in{\mathbb{C}}\smallsetminus\operatorname{Spec}_{R}(\Delta)$. Write $(\Delta-\lambda_{0})^{n}v=\sum_{\lambda}w_{\lambda}\in U$, then each $w_{\lambda}$ lies in $U$ and

[TABLE]

which implies $(\Delta-\xi)^{-n}w_{\lambda}\in U$ the uniqueness of the $\lambda$-expansion.

For $\lambda\neq\lambda_{0}$ we let $\xi$ tend to $\lambda_{0}$ and find $(\Delta-\lambda_{0})^{-n}w_{\lambda}\in U$. Next let $(\Delta-\xi)^{n}v=\sum_{\lambda}w_{\lambda}^{\xi}$ and note that $w_{\lambda}^{\xi}$ depends continuously on $\xi$. As

[TABLE]

we deduce $v_{\lambda}=(\Delta-\xi)^{-n}w_{\lambda}^{\xi}$ by uniqueness. For $\lambda\neq\lambda_{0}$ we let $\xi$ tend to $\lambda_{0}$ and we can deduce $v_{\lambda}=(\Delta-\lambda_{0})^{-n}w_{\lambda}\in U$. This implies $v-v_{\lambda_{0}}\in U$ as claimed. The rest of part (a) is clear.

For (b),(c) and (d) we argue that for an admissible representation the property (d) implies (b) and (c). To see that (d) implies (b) we consider a maximal $\pi$-filtration

[TABLE]

and a complete $G$-stable $L$-filtration $(S_{j})_{j\in L}$ with irreducible quotients. We claim that there must exist indices $\nu_{1}<\nu_{1}^{\prime}<\dots<\nu_{l}<\nu_{l}^{\prime}$ in $L$ such that $S_{\nu_{i}}/S_{\nu_{i}^{\prime}}\cong\pi$ and $S_{\nu_{i}^{\prime}}/S_{\nu_{i-1}}$ has no $\pi$-sub-quotient, so that $l$ equals the number of $\pi$-sub-quotients within the given $L$-filtration and this independent of the chosen maximal $\pi$-filtration. If the $L$-filtration is finite, this is the classical Jordan-Hölder Theorem. We reduce the present case to a finite filtration as follows: We choose a $\lambda\in\operatorname{Spec}_{\pi}(\Delta)$. Then $(S_{j}^{\lambda}=S_{j}\cap V(\Delta,\lambda))_{j}$ is a filtration of this finite dimensional space. There must exist two neighboring indices $i_{1}<j_{1}$ such that $S_{i_{1}}^{\lambda}=0$ and $S_{j_{1}}^{\lambda}\neq 0$. Repeating we find indices $i_{1}<j_{1}<i_{2}<j_{2}<\dots<i_{k}<j_{k}$ such that $i_{\nu}$ and $j_{\nu}$ are neighbored for each $\nu$ and $S_{j_{\nu}}^{\lambda}=S_{i_{\nu+1}}$ always holds, which implies that $S_{i_{n+1}}/S_{j_{\nu}}$ has no $\pi$-sub-quotient. Further $S_{i_{1}}$ and $V/S_{j_{k}}$ both have no $\pi$-sub-quotient. Now one can ignore the $\nu$ with $S_{j_{\nu}}/S_{i_{\nu}}\ncong\pi$ and assume that all quotients are $\cong\pi$. From here on the classical proof of the Jordan-Hölder Theorem applies to show that $k=l$. After that, once we know that $N_{\Gamma,\omega}(\pi)$ equals the number of $\pi$-sub-quotients in the given $L$-filtration, part (c) also follows.

So it remains to show (d). Let $\lambda\in\operatorname{Spec}_{R}(\Delta)$. By Zorn’s lemma there exists a maximal $G$-stable subspace $V_{0}$ such that $V_{0}\cap V(\Delta,\lambda)=0$. Then its closure $\overline{V}_{0}$ is admissible and thus satisfies the same claim, i.e., $\overline{V}_{0}\cap V(\Delta,\lambda)=0$, so by maximality, $V_{0}$ is closed. Let $v\in V(\Delta,\lambda)$ and let $S(v)$ denote the closure of the span of $V_{0}+R(G)v$. Among all spaces $S(v)$ as $v$ varies in $V(\Delta,\lambda)\smallsetminus\{0\}$, there is a minimal one $V_{1}$. Then $V_{1}/V_{0}$ is irreducible. ∎

4 The spectral theorem

Let $G$ be a locally compact group and let $\Gamma\subset G$ be a cocompact lattice. This means that $\Gamma$ is a discrete subgroup such that the quotient $\Gamma\backslash G$ is compact. Let $\omega:\Gamma\to\operatorname{GL}(V)$ be a group homomorphism, where $V=V_{\omega}$ is a finite-dimensional complex vector space. Let $E=E_{\omega}=\Gamma\backslash(G\times V_{\omega})$, where $\Gamma$ acts diagonally. The projection onto the first factor makes $E$ a vector bundle over $\Gamma\backslash G$. The space $\Gamma(E)$ of continuous sections can be identified with the space $C(\Gamma\backslash G,\omega)$ of all continuous functions $f:G\to V_{\omega}$ such that $f(\gamma x)=\omega(\gamma)f(x)$ for all $\gamma\in\Gamma$. Choose a hermitian metric on $E$ to define the space $L^{2}(E)$ of $L^{2}$-sections. This space can be identified with the space $L^{2}(\Gamma\backslash G,\omega)$ of all measurable functions $f:G\to V_{\omega}$ with $f(\gamma x)=\omega(\gamma)f(x)$ and $\int_{F}\left\langle f(x),f(x)\right\rangle_{x}\,dx<\infty$, where $F\subset G$ is a compact fundamental domain for $\Gamma\backslash G$. The group $G$ acts by right translations on the Hilbert space $L^{2}(\Gamma\backslash G,\omega)$. This representation is continuous but in general not unitary. Let $R=R_{\Gamma,\omega}$ denote the right regular representation of $G$ on the Hilbert space $H=L^{2}(\Gamma\backslash G,\omega)$.

Theorem 4.1**.**

Let $G$ be a Lie group and $\Gamma\subset G$ a cocompact lattice. Fix a group-Laplacian $\Delta$. Then the representation $(R,V)$ with $V=L^{2}(\Gamma\backslash G,\omega)$ is $\Delta$-admissible. In particular, there exists a complete filtration for $(R,V)$, the multiplicities $m(\pi)$ of which are finite and given by the maximal lengths of $\pi$-filtrations.

Proof.

The element $\Delta\in U(\mathfrak{g})$ acts on $C^{\infty}(\Gamma\backslash G,\omega)$ as a differential operator of order two whose principal symbol equals the square of the norm given by the Riemannian metric, such operators are called generalized Laplacians in [BGV]. By [Shubin]*Theorems 8.4 and 9.3 and [Markus]*Theorem4.3 it follows that $\Delta$ has discrete spectrum in $L^{2}(\Gamma\backslash G,\omega)$, i.e., there exists a sequence $\lambda_{j}$ of complex numbers which do not accumulate in ${\mathbb{C}}$ such that the space $\bigoplus_{j=1}^{\infty}H(\Delta,\lambda_{j})$ is dense in $H=L^{2}(\Gamma\backslash G,\omega)$. Each $v\in H$ can uniquely be written as convergent $\sum_{j}u_{j}$ with $u_{j}\in H(\Delta,\lambda_{j})$.

One sets $\Lambda_{R}$ equal to ${\mathbb{C}}\smallsetminus\{\lambda_{j}:j\in{\mathbb{N}}\}$. Then for given $\lambda\in\Lambda_{R}$ the space $H(\Delta,\lambda)$ which lies in $C^{\infty}(\Gamma\backslash G,\omega)$, is finite-dimensional. The only tricky point is to show that for a given closed $G$-stable subspace $U\subset H$ one has $(\Delta-\xi)^{-1}U\subset U$ for $\xi\in\Lambda_{R}$. For this note that $(\Delta-\xi)^{-1}=f(\sqrt{\Delta})$ with $f(x)=(x^{2}-\xi)^{-1}$. The Fourier transform of $f$ is $\hat{f}(x)=\frac{e^{i|x|\sqrt{\xi}}}{2\sqrt{\xi}}$, where $\sqrt{\xi}$ denote the unique complex number $\alpha$ with $\operatorname{Im}(\alpha)>0$ and $\alpha^{2}=\xi$. Let $\chi_{0}$ be a smooth function on ${\mathbb{R}}$ with $0\leq\chi\leq 1$, $\chi(t)=1$ for $t\leq 0$ and $\chi_{1}(t)=0$ for $t\geq 1$. For $T>0$ set $\chi_{T}(t)=\chi_{0}(t-T)$ and let $f_{T}$ be defined by $\hat{f}_{T}(x)=\chi_{T}(|x|)\hat{f}(x)$. Then $\hat{f}_{T}(x)$ has compact support and by [CGT] it follows that the operator $f_{T}(\sqrt{\Delta})$ has finite propagation speed. We can view this operator of $G$ or on $\Gamma\backslash G$. The connection between the two is as follows: On $G$ this operator is invariant under left translation by elements of $G$, hence it is given by right convolution with a function, which, by finite propagation speed, has compact support. This function is continuous on $G$ and smooth on the set $G\smallsetminus\{1\}$. We denote it by $x\mapsto f_{T}(\sqrt{\Delta})(x)$. Then on $\Gamma\backslash G$ the operator $f_{T}(\sqrt{\Delta})$ has continuous kernel $k(x,y)=\sum_{\gamma\in\Gamma}f_{T}(\sqrt{\Delta})(x^{-1}\gamma y)$, the sum being locally finite. For $\phi\in L^{2}(E)$ one has $R(f_{T}(\sqrt{\Delta}))\phi(x)=\int_{G}f_{T}(\sqrt{\Delta})(y)\phi(xy)\,dy$ and approximating this integral by Riemann sums, one sees that $R(f_{T}(\sqrt{\Delta}))\phi$ lies in $U$ if $\phi\in U$. It therefore suffices to show that $R(f_{T}(\sqrt{\Delta}))\phi$ converges to $R(\Delta-\xi)^{-1}\phi$ as $T\to\infty$. On the compact manifold $\Gamma\backslash G$ this follows if we show that the kernel of the former converges uniformly to the kernel of the latter, which is a consequence of Theorem 1.4 of [CGT]. ∎

Theorem 4.2** (Trace formula).**

Let $G$ be a locally compact group and let $\Gamma\subset G$ be a cocompact lattice. Let $(\omega,V_{\omega})$ be a representation of the discrete group $\Gamma$ on a finite-dimensional complex vector space $V_{\omega}$ and define the Hilbert space $H=L^{2}(\Gamma\backslash G,\omega)$ as above. Then for each $f\in C^{\infty}_{c}(G)$ the operator $R(f)$ is trace class and its trace equals either side of the equation

[TABLE]

where $N_{\Gamma,\omega}(\pi)$ denotes the maximal length of a $\pi$-filtration in $H$, the sum on the right runs over all conjugacy classes $[\gamma]$ in $\Gamma$, the groups $G_{\gamma}$ and $\Gamma_{\gamma}$ are the centralizers of $\gamma$ in $G$ and $\Gamma$ and $\mathcal{O}_{\gamma}$ denotes the orbital integral

[TABLE]

The left hand side of the formula is also called the spectral side and the right hand side is the geometric side.

Proof.

First assume that $G$ is a Lie group. By the Theorem of Dixmier and Malliavin [DM], every $f\in C_{c}^{\infty}(G)$ is a finite sum of convolution products $g*h$ with $g,h\in C_{c}^{\infty}(G)$. If $f=g*h$ then $R(f)=R(g)R(h)$. Now the same calculus as in the unitary case [HA2]*Chapter 9 implies that $R(f)$ is an integral operator with smooth kernel $k(x,y)=\sum_{\gamma\in\Gamma}f(x^{-1}\gamma y)\omega(\gamma)$, so by [HA2]*Proposition 9.3.1 it is trace class and its trace equals $\int_{\Gamma\backslash G}\operatorname{tr}k(x,x)\,dx$, which with the same computation as in the proof of [HA2]*Theorem 9.3.2 is seen to be equal to

[TABLE]

We this get the geometric side of the trace formula. The spectral side is obtained from Theorem 3.4.

To finish the proof, we generalize the trace formula to arbitrary locally compact groups. So assume now that $G$ is the projective limit of its Lie quotients,

[TABLE]

A given $f\in C_{c}^{\infty}(G)$ will factorize over some Lie quotient $G/N$. We can assume the compact group $N$ chosen so small that $N\cap\Gamma=\{1\}$. Then $\Gamma$ induces a cocompact lattice in $G/N$ and the trace formula for this group implies the trace formula for the given $f$.

Finally, assume that trace formula holds for an open subgroup $H$ of $G$, then $\Gamma\cap H$ is a cocompact lattice in $H$ and the trace formula for $H$ implies the trace formula for $G$. ∎

5 Semisimple Lie groups

In the case of a semisimple group $G$ we here prove a slightly stronger spectral theorem which says that the right regular representation on $L^{2}(E)$ is a direct sum of representations of finite length.

Definition 5.1**.**

A representation $(R,V)$ of a locally compact group has finite length, if there exists a filtration

[TABLE]

of closed $G$-stable subspaces such that $F_{j}/F_{j-1}$ is irreducible for each $j$. The classical Jordan-Hölder Theorem says that then the irreducible quotients $F_{j}/F_{j-1}$ are uniquely determined by $V$ up to order.

Definition 5.2**.**

We say that a representation $(R,V)$ of a locally compact group $G$ is a Jordan-Hölder representation, if it is a direct sum of finite length representations. More precisely, we insist that there are closed $G$-stable subspaces $V_{i}$, $i\in I$ such that the direct sum $\bigoplus_{i\in I}V_{i}$ is dense in $V$.

Let $G$ be a semisimple Lie group with finite center and let $K$ be a maximal compact subgroup. Let $\Gamma\subset G$ be a cocompact lattice and let $(\chi,V_{\chi})$ be a finite dimensional complex representation of $\Gamma$. Then $\chi$ defines a vector bundle $E=E_{\chi}$ over $\Gamma\backslash G$. The smooth sections can be described as

[TABLE]

The choice of a hermitian metric on $E$ allows the definition of the Hilbert space $L^{2}(E)$ of square integrable sections. We equip $\Gamma^{\infty}(E)$ with the topology of $L^{2}(E)$.

Let $V_{\mathrm{fin}}$ be the space of all sections in $\Gamma^{\infty}(E)$ which are $K$-finite as well as $\mathfrak{z}$-finite, where $\mathfrak{z}$ is the center of the universal covering algebra $U(\mathfrak{g}_{\mathbb{C}})$ of the complexified Lie algebra $\mathfrak{g}_{\mathbb{C}}$ of $G$.

Theorem 5.3**.**

The $(\mathfrak{g},K)$-module $V_{\mathrm{fin}}$ is dense in $\Gamma^{\infty}(E)$ as well as in $L^{2}(E)$. The $G$-representations on $\Gamma^{\infty}(E)$ and on $L^{2}(E)$ are Jordan-Hölder representations.

Proof.

For every $(\tau,V_{\tau})\in\hat{K}$ the Casimir element $C\in\mathfrak{z}$ acts on the $\tau$-isotype

[TABLE]

as it acts on

[TABLE]

where $E_{\chi,\tau}$ is the vector bundle over $\Gamma\backslash G/K$ defined by $\chi\times\tau$. On $\Gamma^{\infty}(E_{\chi,\tau})$ the Casimir $C$ induces an operator which has the same principal symbol as the Laplacian for any given metric. Hence ([Shubin], Theorems 8.4 and 9.3) the operator $C$ has discrete spectrum on $L^{2}(E_{\chi,\tau})$ consisting of eigenvalues of finite multiplicity.

Let $\lambda\in{\mathbb{C}}$ be an eigenvalue and let $\Gamma^{\infty}(E_{\chi,\tau})(\lambda)$ be the corresponding finite dimensional generalized eigenspace. The image $V_{\tau,\lambda}$ of $\Gamma^{\infty}(E_{\chi,\tau})(\lambda)$ in $\Gamma^{\infty}(E)$ is $\mathfrak{z}$-stable and $K$-stable. Hence the generated $(\mathfrak{g},K)$-module $U(\mathfrak{g})V_{\tau,\lambda}$ is in $V_{\mathrm{fin}}$ and by Corollary 3.4.7 of [Wall] it is admissible and as it is finitely generated, it is a Harish-Chandra module, so by Corollary 10.42 of [Knapp] it has a finite composition series:

[TABLE]

with irreducible quotients $F_{j+1}/F_{j}$. We repeat this argument with a different $K$-type $\tau^{\prime}$ not occurring in $U(\mathfrak{g})V_{\tau,\lambda}$ if it exists. Otherwise, we repeat it with a different eigenvalue $\lambda$ to get the claim. ∎