Enhanced zeta distributions and its functional equations

TL;DR

This paper studies an enhanced symmetric space related to prehomogeneous vector spaces, establishing a meromorphic continuation of associated zeta integrals, and deriving explicit functional equations with gamma factors.

Contribution

It introduces a new framework for enhanced symmetric spaces, providing explicit formulas for zeta integrals' continuation and their functional equations.

Findings

Meromorphic continuation of zeta integrals with two complex variables.

Explicit formulas for the location of poles and residues.

Functional equations involving gamma factors.

Abstract

We consider an ``enhanced symmetric space'', which is a prehomogeneous vector space. This vector space is intimately related to a double flag variety studied in \cite{NO.2018}. On a distinguished open orbit called ``enhanced positive cone'', we consider a zeta integral with two complex variables, which is analytically continued to meromorphic family of tempered distributions. One of the main results of this paper is to establish a precise formula for the meromorphic continuation which clarifies the location of poles (and may be useful to obtain residues). We also compute the Fourier transform of the zeta distribution and obtain a functional equation with explicit gamma factors.

Equations

$$Z^{\mathrm{T}}(\phi ,s)={\int }_{\mathbb{R}}\phi (z)|z{|}^{s}dz(s\in \mathbb{C}).$$

$$Z^{\mathrm{GJ}}(\phi ,s)={\int }_{{\mathrm{M}}_n(\mathbb{R})}\phi (z)|\det z{|}^{s}dz,$$

$$V_{\mathbb{R}}\setminus \{z\in V_{\mathbb{R}}\mid P(z)=0\}={\bigcup }_{i=1}^{\ell }{\mathcal{O}}_i.$$

$$Z_i^{(G,V)}(\phi ,s)={\int }_{{\mathcal{O}}_i}\phi (z)|P(z){|}^{s}dz(\phi \in \mathcal{S}(V_{\mathbb{R}}),s\in \mathbb{C}),$$

$$(g,h)\cdot (z,y)=(gz{}^{t}g,gy{}^{t}h),\text{ where }(g,h)\in L\times H=G,(z,y)\in V\oplus E=W.$$

$$P_{1}(z,y)=\det z,\qquad P_{2}(z,y)=(-1)^{d}\det\Bigl{(}\begin{array}[]{@{\,}c@{\;\;}c@{\,}}{z}&{y}\\ {\!\!\,{}^{t}{y}}&{0}\end{array}\Bigr{)}\qquad((z,y)\in W_{\mathbb{C}}),$$

$$P_{2}(z,y)=\det z\cdot\det\bigl{(}\,{}^{t}{y}\,z^{-1}y\bigr{)},$$

$$P_i^{\ast }({\partial }_{z,y})e^{\mathrm{Tr}zw+\mathrm{Tr}{}^{t}yx}=P_i(w,x)e^{\mathrm{Tr}zw+\mathrm{Tr}{}^{t}yx}(z,w\in V,y,x\in E,i=1,2).$$

$$b_{1,0}(s)$$

$$b_{0,1}(s)$$

$$\displaystyle P_{1}^{\ast}(\partial_{z,y})\Bigl{(}P_{1}(z,y)^{s_{1}+1}P_{2}(z,y)^{s_{2}}\Bigr{)}$$

$$\displaystyle P_{2}^{\ast}(\partial_{z,y})\Bigl{(}P_{1}(z,y)^{s_{1}}P_{2}(z,y)^{s_{2}+1}\Bigr{)}$$

$$\tilde{\Omega }=\Omega \times {\mathrm{M}}_{n,d}^{\circ }(\mathbb{R}),$$

$$Z_{\tilde{\Omega }}(\phi ,s)$$

$$\displaystyle=\int_{\mathop{\mathrm{Sym}}\nolimits_{n}^{+}(\mathbb{R})}(\det z)^{s_{1}}\,dz\int_{\mathrm{M}_{n,d}(\mathbb{R})}\varphi(z,y)\Bigl{|}\det\Bigl{(}\begin{array}[]{@{\,}c@{\;\;}c@{\,}}{z}&{y}\\ {\,{}^{t}{y}}&{0}\end{array}\Bigr{)}\Bigr{|}^{s_{2}}\,dy$$

$${\Gamma }_k(\alpha )=\Gamma (\alpha )\Gamma (\alpha -\frac{1}{2})\cdots \Gamma (\alpha -\frac{k-1}{2})={\prod }_{j=1}^k\Gamma (\alpha -\frac{j-1}{2}),$$

$${\Gamma }_{\tilde{\Omega }}(s)={\Gamma }_d(s_1+\frac{d+1}{2}){\Gamma }_d(s_2+\frac{d+1}{2}){\Gamma }_d(s_2+\frac{n}{2}){\Gamma }_{n-d}(s_1+s_2+\frac{n+1}{2}).$$

$$\frac{1}{{\Gamma }_{\tilde{\Omega }}(s)}{Z}_{\tilde{\Omega }}(\phi ,s)$$

$$K^{+}_{s}(\widetilde{z})=\begin{cases}(\det z)^{s_{1}}\,\bigl{|}\det\Bigl{(}\begin{array}[]{@{\,}c@{\;\;}c@{\,}}{z}&{y}\\ {\,{}^{t}{y}}&{0}\end{array}\Bigr{)}\bigr{|}^{s_{2}}&\widetilde{z}\in\widetilde{\Omega}\\ 0&\text{otherwise.}\end{cases}$$

$$⟨\tilde{z},\tilde{w}⟩=\mathrm{Tr}zw+\mathrm{Tr}{}^{t}yx\text{ for }\tilde{z}=(z,y),\tilde{w}=(w,x)\in W,$$

$$\hat{\phi }(\tilde{w})={\int }_W\phi (\tilde{z})e^{-2\pi i⟨\tilde{z},\tilde{w}⟩}d\tilde{z}.$$

$$({\mathcal{F}}^{-1}\psi )(\tilde{z})=2^{-\frac{n(n-1)}{2}}{\int }_W\psi (\tilde{w})e^{2\pi i⟨\tilde{w},\tilde{z}⟩}d\tilde{w},$$

$${\Xi }_s(\tilde{w})$$

$$\displaystyle=\lim_{v\downarrow 0}\det(v+2\pi iw)^{s_{1}}\Bigl{(}(-1)^{d}\det\Bigl{(}\begin{array}[]{@{\,}c@{\;\;}c@{\,}}{v+2\pi iw}&{x}\\ {\,{}^{t}{x}}&{0}\end{array}\Bigr{)}\Bigr{)}^{s_{2}},$$

$${\int }_W{\Xi }_s(\tilde{w})\phi (\tilde{w})d\tilde{w}=\underset{v\downarrow 0}{\lim }{\int }_W{P}_1(v+2\pi iw,x{)}^{s_1}{P}_2(v+2\pi iw,x{)}^{s_2}\phi (\tilde{w})d\tilde{w}$$

$${\Xi }_{(0,0)}(\tilde{w})=\delta (\tilde{w})\text{(delta distribution supported in the origin)}.$$

$$\frac{1}{{\Gamma }_d(s_1+\frac{d+1}{2}){\Gamma }_d(s_2+\frac{n}{2}){\Gamma }_{n-d}(s_1+s_2+\frac{n+1}{2})}\widehat{K_s^{+}}=\frac{c(s)}{{\Gamma }_d(-s_2)}{\Xi }_{-(s_1+\frac{d+1}{2}),-(s_2+\frac{n}{2})},$$

$$c(s)=(2\pi {)}^{\frac{n(n-1)}{4}}{\pi }^{-2d(s_2+\frac{n}{4})},$$

$$\delta=\dfrac{1}{c(s)}\dfrac{\Gamma_{d}(s_{2}+\dfrac{d+1}{2})\,\Gamma_{d}(-s_{2})}{\Gamma_{\widetilde{\Omega}}(s)}\cdot\widehat{K^{+}_{s}}\Bigm{|}_{s=-\frac{1}{2}(d+1,n)}.$$

$$\frac{1}{{\Gamma }_{\tilde{\Omega }}(s)}$$

$$=\frac{c(s)(-2\pi i{)}^{-(n{s}_1+(n-d)s_2+\frac{n(n+1)}{2})}}{{\Gamma }_d(s_2+\frac{d+1}{2}){\Gamma }_d(-s_2)}{Z}_{\tilde{\Omega }}(\phi ;-(s_1+\frac{d+1}{2}),-(s_2+\frac{n}{2}))$$

$$=\frac{c(s)}{(-2\pi i{)}^{n{s}_1+(n-d)s_2+\frac{n(n+1)}{2}}(-\pi {)}^d}\prod _{j=1}^d\sin (s_2+\frac{d-j}{2})\times$$

$$Z_{\tilde{\Omega }}(\phi ;-(s_1+\frac{d+1}{2}),-(s_2+\frac{n}{2}))$$

$${\int }_W\widehat{K_s^{+}}(\tilde{w})\phi (\tilde{w})d\tilde{w}$$

Full text

1. Introduction

Zeta distributions or zeta integrals are studied extensively by many authors. Most classical one is the Tate’s zeta integral: for a Schwarz function $\varphi\in\mathscr{S}(\mathbb{R})$,

[TABLE]

The integral converges for $\mathop{Re}\nolimits s>-1$ and admits a meromorphic continuation to all the complex plane. A generalization is the Godement-Jacquet zeta integral:

[TABLE]

with obvious notations. Also we can consider zeta integrals over the space of symmetric matrices $\mathop{\mathrm{Sym}}\nolimits_{n}(\mathbb{R})$. In this case, we will consider the integral over the set of matrices with a fixed signature: $\Omega(p,q)=\{z\in\mathop{\mathrm{Sym}}\nolimits_{n}(\mathbb{R})\mid\mathop{sgn}\nolimits(z)=(p,q)\}\;(p+q=n)$. These zeta integrals are important in number theory since they produce zeta functions for lattices.

Around 1970’s, by T. Shintani and M. Sato ([15]), these zeta integrals are generalized to those for prehomogeneous vector spaces. Let $G_{\mathbb{C}}$ be a reductive algebraic group and assume that it linearly acts on a vector space $V_{\mathbb{C}}$ with an open orbit $\mathcal{O}$. In other words, the pair $(G_{\mathbb{C}},V_{\mathbb{C}})$ is a prehomogeneous vector space, which is abbreviated to PV. For the properties of PV, readers may refer to [6] or [15, 16]. To proceed further, we assume that $S=V_{\mathbb{C}}\setminus\mathcal{O}$ is an irreducible hypersurface which implies the existence of a unique fundamental relative invariant $P(z)\in\mathbb{C}[V_{\mathbb{C}}]$ and $S$ is defined by $P(v)=0$.

Let $V_{\mathbb{R}}$ be a real form of $V_{\mathbb{C}}$ on which $G_{\mathbb{R}}$ acts. Then $V_{\mathbb{R}}\setminus\{z\in V_{\mathbb{R}}\mid P(z)=0\}$ breaks up into several open $G_{\mathbb{R}}$ orbits:

[TABLE]

On each $\mathcal{O}_{i}$, we define a zeta integral

[TABLE]

regarded as a tempered distribution. For these zeta integrals, Sato and Shintani proved a fundamental theorem which states (1) existence of a meromorphic continuation of the distribution in the parameter $s\in\mathbb{C}$; (2) duality with respect to the Fourier transform, which is called functional equations. This theorem is very powerful and investigated by many authors, but in practice, it needs a serious work to determine the meromorphic continuation (e.g., to locate poles or to obtain residues) and to deduce explicit functional equations. It is out of the authors’ scope to list all the relevant works which study explicit forms of zeta integrals and their properties. However, among many of them, we would like to mention [17, 19, 18, 9, 7, 4, 1, 2, 13, 14, 3] and references therein.

Later, Fumihiro Sato generalized the fundamental theorem to several complex variables ([10, 12, 11]).

In this note, we will study the above two problems for a prehomogeneous vector space called an “enhanced symmetric space” associated with a double flag variety (see [8]). In this case, there are two fundamental relative invariants so that we consider a zeta integral in two complex variables $s=(s_{1},s_{2})$. We get an explicit meromorphic continuation (or gamma factors) indicating poles (Theorem 2.2) and also explicit functional equations in the case of “enhanced positive cone” (Theorems 3.4 and 5.2). To state the results more precisely, we need to prepare the notations and settings.

2. Enhanced zeta integral and its meromorphic continuation

Let $V=\mathop{\mathrm{Sym}}\nolimits_{n}(\mathbb{R})$ be the space of real symmetric matrices. Then $V$ inherits a structure of Euclidean Jordan algebra (see [5]). Let us denote $E=\mathrm{M}_{n,d}(\mathbb{R})$, a representation space of $V$, where $z\in V$ acts on $E$ by the left multiplication and as a consequence it admits a natural action of L=\mathop{\mathrm{missing}}{Str}\nolimits{}(V)=\mathrm{GL}_{n}(\mathbb{R}), the structure group of the Jordan algebra $V$. On the other hand, $E$ can be regarded as a direct sum of $d$-copies of the natural representation $\mathbb{R}^{n}$ of $V$. This allows an action of $H=\mathrm{GL}_{d}(\mathbb{R})$ on $E$ by the right multiplication.

Consider a direct sum $W=V\oplus E=\mathop{\mathrm{Sym}}\nolimits_{n}(\mathbb{R})\oplus\mathrm{M}_{n,d}(\mathbb{R})$ and let us denote $G=L\times H=\mathrm{GL}_{n}(\mathbb{R})\times\mathrm{GL}_{d}(\mathbb{R})$. There is a natural action of $G$ on $W$ via

[TABLE]

If we extend the base field $\mathbb{R}$ to $\mathbb{C}$, the space $W_{\mathbb{C}}=\mathop{\mathrm{Sym}}\nolimits_{n}(\mathbb{C})\oplus\mathrm{M}_{n,d}(\mathbb{C})$ is a prehomogeneous vector space with the action of $G_{\mathbb{C}}$ and it has only finitely many $G_{\mathbb{C}}$ orbits.

In the theory of prehomogeneous vector spaces, relative invariants and their $b$-functions play important roles. If $d\leq n$, the fundamental relative invariants in $\mathbb{C}[W_{\mathbb{C}}]$ (the space of regular functions, or polynomials over $W_{\mathbb{C}}$) are given by

[TABLE]

where $P_{1}$ (respectively $P_{2}$) is associated with the character $\chi_{P_{1}}(g,h)=(\det g)^{2}$ (respectively $\chi_{P_{2}}(g,h)=(\det g)^{2}(\det h)^{2}$) for $(g,h)\in G_{\mathbb{C}}$. Note that if $z\in\mathop{\mathrm{Sym}}\nolimits_{n}(\mathbb{C})$ is regular we can rewrite

[TABLE]

and this is the reason why we put $(-1)^{d}$ in the definition of $P_{2}$. If $z$ is a positive definite (real) symmetric matrix and $y$ is also a real matrix, $P_{i}(z,y)\;(i=1,2)$ are both positive. Any relative invariants are of the form $P_{1}^{m_{1}}P_{2}^{m_{2}}$ with $m_{1},m_{2}\in\mathbb{Z}_{\geq 0}$.

If $d>n$, the relative invariant $P_{2}(z,y)$ above vanishes identically, and only $P_{1}$ survives. In fact, in this case, any relative invariants are some powers of $P_{1}$. Thus, throughout this paper, we always assume the inequality $d\leq n$.

For the fundamental relative invariants $P_{1}$ and $P_{2}$, let us describe $b$-functions in two variables. We denote by $P_{i}^{\ast}(\partial_{z,y})$ a constant coefficient differential operator which is defined by

[TABLE]

Proposition 2.1** ([20, Propositions 6 & 7]).**

Let $s=(s_{1},s_{2})\in\mathbb{C}^{2}$ be a pair of complex variables. We put

[TABLE]

Then the following Bernstein-Sato identity holds:

[TABLE]

The polynomials $b_{1,0}(s)$ and $b_{0,1}(s)$ are called $b$-functions.

Let us return back to the base field $\mathbb{R}$. There is a unique open orbit over $\mathbb{C}$, but it breaks up into several open orbits over $\mathbb{R}$. It is not difficult to parametrize all of them (see Lemma 3.1 below), and among them, there is a distinguished open orbit denoted as

[TABLE]

where $\Omega=\mathop{\mathrm{Sym}}\nolimits_{n}^{+}(\mathbb{R})$ is the space of real positive symmetric matrices (a homogeneous symmetric cone contained in $V$ in the terminology of [5]) and $\mathrm{M}^{\circ}_{n,d}(\mathbb{R})$ is the set of full rank matrices in $E=\mathrm{M}_{n,d}(\mathbb{R})$. We call $\widetilde{\Omega}$ an enhanced positive symmetric cone.

We are interested in the following integral on $\widetilde{\Omega}$, which is called an enhanced zeta integral (or distribution).

[TABLE]

Here $s=(s_{1},s_{2})$ are complex parameters and $\varphi(z,y)\in\mathscr{S}(W)$ is a rapidly decreasing function (Schwarz function). The measures $dz$ and $dy$ are (restriction of) Lebesgue measures. The integral converges if $\mathop{Re}\nolimits s_{1},\mathop{Re}\nolimits s_{2}\gg 0$ are sufficiently large, and for the rest of $s$, we will consider analytic continuation as a tempered distribution. For this, we need some more notation. For a non-negative integer $k\geq 0$ and $\alpha\in\mathbb{C}$, we put

[TABLE]

and for our enhanced positive cone $\widetilde{\Omega}$, define

[TABLE]

Then the standard arguments of the meromorphic continuation using $b$-functions together with Proposition 2.1 prove the following theorem.

Theorem 2.2** (Meromorphic Continuation).**

The zeta integral normalized by the gamma factor

[TABLE]

is extended to an entire function in $s=(s_{1},s_{2})\in\mathbb{C}^{2}$ for any $\varphi\in\mathscr{S}(W)$. In other words, $Z_{\widetilde{\Omega}}{}(\varphi,s)$ can be extended to a meromorphic function with possible poles specified by the gamma factor $\Gamma_{\widetilde{\Omega}}(s)$.

3. Fourier transform and a functional equation

Let $\mathcal{O}_{i}\;(i=1,\dots,\ell)$ be open $G$ orbits in $W$. The enhanced positive cone $\widetilde{\Omega}$ is one of them.

Lemma 3.1** ([8]).**

A $G$-orbit through $(z,y)\in W$ is open if and only if $z\in\mathop{\mathrm{Sym}}\nolimits_{n}(\mathbb{R})$ and $\,{}^{t}{y}\,z^{-1}y\in\mathop{\mathrm{Sym}}\nolimits_{d}(\mathbb{R})$ are both regular. Open orbits are classified by the signature of $z$ and the signature of $\,{}^{t}{y}\,z^{-1}y$.

Remark 3.2*.*

Let us denote the image of the matrix $y$ by $[y]:=\mathop{Im}\nolimits y\subset\mathbb{R}^{n}$, which is a $d$-dimensional subspace of $\mathbb{R}^{n}$. Then the signature of $\,{}^{t}{y}\,z^{-1}y$ is equal to that of the quadratic form z^{-1}\big{|}_{[y]} restricted to $[y]$. Note that there is a natural condition on the signature which is induced from $z^{-1}$ (or $z$). See § 5.1 for details.

Our zeta integral $Z_{\widetilde{\Omega}}{}(\varphi,s)$ is considered to be a tempered distribution supported in the closure of $\widetilde{\Omega}$. Let us calculate the Fourier transform of the distribution.

Write $\widetilde{z}=(z,y)$ and we denote the integral kernel by

[TABLE]

We define a positive definite inner product on $W=\mathop{\mathrm{Sym}}\nolimits_{n}(\mathbb{R})\oplus\mathrm{M}_{n,d}(\mathbb{R})$ by

[TABLE]

and the Euclidean Fourier transform $\mathscr{F}\varphi=\widehat{\varphi}$ is defined as usual:

[TABLE]

Note that the Fourier inversion becomes

[TABLE]

because of the normalization of the inner product.

To state the result, we need a definition of the boundary value distribution of an analytic function, i.e., a hyperfunction. We do not need a general theory, so we just put

[TABLE]

where $v\in\Omega=\mathop{\mathrm{Sym}}\nolimits_{n}^{+}(\mathbb{R})$ moves to [math] in the positive cone. This is a well defined distribution supported in the whole $W$ interpreted as

[TABLE]

for a test function $\varphi\in\mathscr{S}(W)$ (at least for $\mathop{Re}\nolimits s_{1},\mathop{Re}\nolimits s_{2}\geq 0$, and then apply an analytic continuation). Notice that if $\mathop{Re}\nolimits s_{1}\geq 0$ and $\mathop{Re}\nolimits s_{2}\geq 0$, then we can take a limit $v\downarrow 0$ inside the integral, and we get $\Xi_{s}(\widetilde{w})=(2\pi i)^{ns_{1}+(n-d)s_{2}}P_{1}(\widetilde{w})^{s_{1}}P_{2}(\widetilde{w})^{s_{2}}$ with an appropriate choice of the branch of exponents. In particular, we get

[TABLE]

Remark 3.3*.*

We will rewrite $\Xi_{s}$ more explicitly as a linear sum of the distribution supported in the closure of each open orbits $\mathcal{O}_{i}$’s afterwards. See Lemma 5.1.

Theorem 3.4**.**

We get a formula for the Fourier transform of the integral kernel $K^{+}_{s}$ in the sense of a tempered distribution:

[TABLE]

where

[TABLE]

and $\Gamma_{k}(\alpha)$ is given in (2.1). In particular, $\widehat{K^{+}_{s}}$ has a pole at $s=-\frac{1}{2}(d+1,n)$ and there the first residue is a constant multiple of the delta distribution:

[TABLE]

Corollary 3.5**.**

If $\varphi\in\mathscr{S}(W)$ is supported in the closure of the enhanced positive cone $\widetilde{\Omega}$, we get a functional equation:

[TABLE]

Remark 3.6*.*

For $d=1$, Suzuki [19] proved the analytic continuation (Theorem 2.2) and the formula for Fourier transform (Theorem 3.4) above.

In a general theory of zeta integrals for several variables, in an abstract way, these formulas are already established by F. Sato ([10, 12, 11]) as we mentioned above, but determining explicit gamma factors/constants or residues with the location of poles are different issues. Recently the authors were informed that F. Sato has obtained explicit formulas in the present case, or rather, its “partial dual” (unpublished). We thank Professor F. Sato for sending us a draft on this subject.

4. Proof of the formula of the Fourier transform

Let us explain how to prove Theorem 3.4.

For that purpose, we suppose a test function is a product of functions in separate variables so that $\varphi(\widetilde{z})=\varphi_{1}(z)\varphi_{2}(y)$. By the definition of the Fourier transform of a distribution, we have

[TABLE]

4.1. Fourier transform of the determinant of quadratic maps

Let us denote the first integral in (4.1) over $y\in E$ by $I_{1}(z)$. Now write $z=\,{}^{t}{g}\,g$ for some $g\in L$. Since $z$ is a positive definite symmetric matrix, this is certainly possible and even $g$ can be taken from the lower triangular Borel subgroup. Then, after replacing $y$ by $\,{}^{t}{g}\,y$, we get

[TABLE]

It is easy to check that $\widehat{\varphi_{2}}(\,{}^{t}{g}\,y)=|\det g|^{-d}\widehat{L_{g}\varphi_{2}}(y)$, where $L_{g}\varphi_{2}(x)=\varphi_{2}(g^{-1}x)$ is the left translation. Thus, if we put $\psi(x)=L_{g}\varphi_{2}(x)$ temporarily,

[TABLE]

which is the Fourier transform of the complex power of the determinant of the quadratic map $Q(y)=\,{}^{t}{y}\,y$ associated to the representation of the Jordan algebra $V_{d}=\mathop{\mathrm{Sym}}\nolimits_{d}(\mathbb{R})$. Do not confuse this with our Jordan algebra $V=\mathop{\mathrm{Sym}}\nolimits_{n}(\mathbb{R})$. For this, we have the following

Lemma 4.1** ([4, Theorem 2]; see also [16]).**

Under the setting above and a complex variable $\alpha\in\mathbb{C}$, we have

[TABLE]

as tempered distributions.

Remark 4.2*.*

Note that our normalization of the Fourier transform is different from Clerc’s. Also we emphasize that $Q(y)=\,{}^{t}{y}\,y$ is the quadratic map associated to the Jordan algebra $V_{d}=\mathop{\mathrm{Sym}}\nolimits_{d}(\mathbb{R})$ of smaller size.

The formula in the above lemma is the zeta integral of PV $(\mathrm{GL}_{n}(\mathbb{C}),\mathrm{M}_{n,d}(\mathbb{C}))$, and in this sense, M. Sato gave it as an ‘exercise’ (or an example of such formulas) in his fundamental paper [16] on the theory of PV. On the other hand, Clerc [4] studied representations of Euclidean Jordan algebras in general, and he established a formula for Fourier transform of the determinant of quadratic maps associated to the representations in a broader context.

From this lemma, we continue

[TABLE]

Then the formula (4.1) becomes

[TABLE]

4.2. Laplace transform over the positive cone and the gamma constants

To compute the integral in (4.2) with respect to $z$, we put a convergence factor $e^{-\mathop{Tr}\nolimits zv^{2}}\;(v\in\Omega)$ and then take limit $v\downarrow 0$. Thus, we consider an integral below ($\alpha:=s_{1}+s_{2}+d/2,\;\;\beta:=-s_{2}-n/2$).

[TABLE]

Since the last integral over $z$ is analytic in the variable $v^{2}+2\pi iw\in\Omega+iV\subset V_{\mathbb{C}}$, we first calculate it when $w=0$, then make an analytic continuation. Thus we are to consider

[TABLE]

The following lemma is easy to derive.

Lemma 4.3**.**

For $x\in E$, put

[TABLE]

Then it satisfies (1) $\Phi(xa)=(\det a)^{2\beta}\Phi(x)\;\;(a\in\mathrm{GL}_{d}(\mathbb{R}))$; and (2) $\Phi(ux)=\Phi(x)\;\;(u\in\mathrm{O}_{n}(\mathbb{R}))$.

Note that any $x\in E=\mathrm{M}_{n,d}(\mathbb{R})$ can be decomposed into $x=u\,\mathbb{I}_{d}\,a\;\;(u\in\mathrm{O}_{n}(\mathbb{R}),a\in\mathrm{GL}_{d}(\mathbb{R}))$, where \mathbb{I}_{d}={\Bigl{(}\begin{array}[]{@{\,}c@{\,}}{1_{d}}\\ {0}\end{array}\Bigr{)}}\in E. Therefore from the lemma, we get

[TABLE]

We call $\gamma(\alpha,\beta):=\Phi(\mathbb{I}_{d})$ gamma constant. Since $\det(\,{}^{t}{\mathbb{I}_{d}}\,z\mathbb{I}_{d})=\Delta_{d}(z)$ ($d$-th principal minor), we get

[TABLE]

which can be expressed in terms of Gindikin’s gamma function.

Lemma 4.4** ([5, Theorem VII.1.1]).**

The gamma constant is given by

[TABLE]

Now, summing up all the relevant information, we get

[TABLE]

After substituting $\alpha=s_{1}+s_{2}+d/2,\;\beta=-s_{2}-n/2$ and comparing with (4.2), we get Theorem 3.4. The proof is completed.

5. Functional equation for the positive cone

Here we will rewrite Theorem 3.4 to express the Fourier transform as a linear sum of distributions supported in open orbits.

5.1. Open orbits and the restriction of the boundary value distribution

A classification of open $G$-orbits is already given in Lemma 3.1. Thus, an open orbit $\mathcal{O}\subset V\oplus E=W$ is characterized by the signature of $z$ and that of z^{-1}\big{|}{}_{[y]}, where $[y]=\mathop{Im}\nolimits y$. Namely, for quadruples of nonnegative integers $\rho=(p,q;p^{\prime},q^{\prime})$ which satisfies $p+q=n,\;p^{\prime}+q^{\prime}=d,\;0\leq p^{\prime}\leq p,\;0\leq q^{\prime}\leq q$, the set of all $\widetilde{z}=(z,y)\in W$ with $\mathop{sgn}\nolimits z=(p,q)$ and $\mathop{sgn}\nolimits(\,{}^{t}{y}\,z^{-1}y)=(p^{\prime},q^{\prime})$ make up an open orbit, which we will denote by $\mathcal{O}_{\rho}$. Note that $\mathop{sgn}\nolimits(\,{}^{t}{y}\,z^{-1}y)$ is the same as that of z^{-1}\big{|}{}_{[y]}.

In § 3 we introduced the boundary value distribution $\Xi_{s}(z,y)$ which is real analytic in $\widetilde{z}=(z,y)$ and meromorphic in $s=(s_{1},s_{2})$. On the other hand, we have a tempered distribution supported in each open orbit $\mathcal{O}_{\rho}$:

[TABLE]

Note that, since $\widetilde{\Omega}=\mathcal{O}_{(n,0;d,0)}$, $K^{+}_{s}=K^{\rho}_{s}$ for $\rho=(n,0;d,0)$.

Lemma 5.1**.**

For $\rho=(p,q;p^{\prime},q^{\prime})$, put u_{\rho}(s)=(2\pi)^{ns_{1}+(n-d)s_{2}}\exp\frac{\pi i}{2}\bigl{(}(p-q)(s_{1}+s_{2})-(p^{\prime}-q^{\prime})s_{2}\bigr{)}. If $\mathop{Re}\nolimits s_{1},\mathop{Re}\nolimits s_{2}>0$, we have \Xi_{s}\big{|}_{\mathcal{O}_{\rho}}=u_{\rho}(s)K^{\rho}_{s} so that it holds

[TABLE]

By analytic continuation, the equation is valid for any $s\in\mathbb{C}^{2}$ if both sides are interpreted as meromorphic functions.

Proof.

The proof is straightforward, but we give it shortly for the convenience of the readers. Note that each orbit $\mathcal{O}_{\rho}$ has a representative (z_{0},y_{0})=(\Bigl{(}\begin{array}[]{@{\,}c@{\;\;}c@{\,}}{1_{p}}&{0}\\ {0}&{-1_{q}}\end{array}\Bigr{)},\Biggl{(}\begin{array}[]{@{}c@{\,}c@{}}1_{p^{\prime}}&0\\[-0.43057pt] 0&0\\[-1.29167pt] 0&1_{q^{\prime}}\end{array}\Biggr{)}). Therefore a general element in the orbit can be written as $(z,y)=(g,h)\cdot(z_{0},y_{0})=(gz_{0}\,{}^{t}{g}\,,gy_{0}\,{}^{t}{h}\,)$. Thus for $v\in\Omega$ we compute

[TABLE]

where $v_{0}=g^{-1}v\,{}^{t}{g}\,^{-1}$. Notice that $\det g\,{}^{t}{g}\,=|{\det z}|$ and $\det h\,{}^{t}{h}\,=|{\det\,{}^{t}{y}\,z^{-1}y}|$.

Since by the assumption $\mathop{Re}\nolimits s_{1},\mathop{Re}\nolimits s_{2}>0$ holds and we can take the limit $v\downarrow 0$ in any way. So we choose $v_{0}=t1_{n}\;\;(t>0)$ as a scalar matrix and take the limit $t\downarrow 0$. Then the last formula in (5.3) becomes

[TABLE]

which gives $u_{\rho}(s)$. ∎

5.2. Functional equation for the enhanced positive cone

Summarizing Theorem 3.4 and Lemma 5.1, we get

Theorem 5.2**.**

The Fourier transform of the integral kernel $K^{+}_{s}$ satisfies the following functional equation:

[TABLE]

where $\rho=(p,q;p^{\prime},q^{\prime})$ moves over all the possible parameters of open orbits and $u_{\rho}(s)$ is given in Lemma 5.1. The rest of the notation is the same as those in Theorem 3.4.

Acknowledgements

The first author (K. N.) is supported by JSPS KAKENHI Grant Numbers #16K05070. He also thanks for warm hospitality during his visits to the Department of Mathematics, Aarhus University.