Bourbaki

§ 4. Representations of compact groups

In this paragraph, all topological vector spaces considered are complex, unless the contrary is explicitly stated.

Let G be a compact topological group, whose identity element is denoted by ee. The group G is unimodular (INT, VII, p. 20, § 1, no3^o3, cor. of prop. 3). Let μ\mu denote the normalized Haar measure on G (that is to say such that μ(G)=1\mu(G) = 1), and for 1⩽p⩽+∞1\leqslant p\leqslant +\infty, let Lp(G)\mathscr{L}^p(G) (resp. Lp(G)L^p(G)) denote the space LCp(G,μ)\mathscr{L}_{\mathbf{C}}^p(G, \mu) (resp. the space LCp(G,μ)L^p_{\mathbf{C}}(G, \mu)). Convolutions will always be considered relative to the measure μ\mu. Let p∈[1,+∞]p\in [1,+\infty ]. We identify C(G)\mathscr{C}(G) with a subspace of LpL^p(G), which is permissible since the support of μ\mu is equal to G.

For every irreducible unitary representation π∈G^\pi \in \widehat{G}, let EπE_{\pi} denote the space of π\pi.

1. Semisimplicity of finite-dimensional representations

We recall (INT, VII, p. 71, § 3, no1^o1, lemma 1) that for every continuous representation ϱ\varrho of G in a Hilbert space E, there exists a nondegenerate positive hermitian form qq on E such that the topological vector space structure of E defined by qq is identical with the initial structure of E, and such that ϱ\varrho is a unitary representation of G in the Hilbert space E endowed with the scalar product qq.

Proposition 1 03DN

Let ϱ\varrho be a finite-dimensional linear representation of G in a separated topological vector space E. There exists a scalar product qq on E such that ϱ\varrho is a unitary representation in the Hilbert space E endowed with qq. In particular, ϱ\varrho is semisimple.

Since E is finite-dimensional, there exists a Hilbert space structure on E. The result follows by applying the preceding remark and corollary 2 of V, p. 392.

Remark 03DO

We shall see later (cor. 4 of V, p. 466) that every unitary representation of G is semisimple.

Corollary 03DP

Let ϱ1\varrho_1 and ϱ2\varrho_2 be finite-dimensional representations of G. The representations ϱ1\varrho_1 and ϱ2\varrho_2 are isomorphic if and only if their characters are equal.

This follows from the proposition and corollary 3 of V, p. 392.

2. Irreducible representations

Lemma 1 03DQ

Every irreducible unitary representation of G is square-integrable.

In fact, the matrix coefficients of an irreducible unitary representation are continuous and bounded, and are therefore square-integrable on G, since G is compact.

Proposition 2 03DR

Let π\pi be an irreducible unitary representation of G in a Hilbert space E. The dimension of E is finite and equal to the formal degree of π\pi.

Since the representation π\pi is square-integrable (lemma 1), its formal degree cμ(π)c_\mu(\pi ) relative to μ\mu is defined (def. 4 of V, p. 423); it is a strictly positive real number. Let (ei)i∈I(e_i)_{i\in I} be a finite orthonormal family in E. Let xx be an element of E of norm 1 (there exists one since E is nonzero). For i∈Ii\in I, one has

cμ(π)∫G∣⟨ei∣π(g)x⟩∣2dμ(g)=1 c_\mu(\pi )\int_G|\langle e_i|\pi (g)x\rangle |^2d\mu(g) = 1

(prop. 8 of V, p. 424). Let us sum this formula over i∈Ii\in I. We obtain

Card(I) =cμ(π)∫G∑i∈I∣⟨ei∣π(g)x⟩∣2dμ(g)=c_\mu(\pi )\int_G\sum_{i\in I}|\langle e_i|\pi (g)x\rangle |^2d\mu(g)

⩽cμ(π)∫G∥π(g)x∥2dμ(g)=cμ(π) \leqslant c_\mu(\pi )\int_G\|\pi (g)x\|^2d\mu(g) =c_\mu(\pi )

by Bessel's inequality (EVT, V, p. 21, prop. 4). Hence the cardinal of I is bounded above by cμ(π)c_\mu(\pi ). This implies that the dimension of E is finite.

One may then apply the foregoing to an orthonormal basis (ei)i∈I(e_i)_{i\in I} of E. One obtains, by EVT, V, p. 22, prop. 5,

dim(E) =cμ(π)∫G∑i∈I∣⟨ei∣π(g)x⟩∣2dμ(g)=c_\mu(\pi )\int_G\sum_{i\in I}|\langle e_i|\pi (g)x\rangle |^2d\mu(g)

=cμ(π)∫G∥π(g)x∥2dμ(g)=cμ(π) =c_\mu(\pi )\int_G\|\pi (g)x\|^2d\mu(g) =c_\mu(\pi )

which completes the proof.

In particular, it is therefore permissible to speak of the character χπ\chi_{\pi} of an irreducible unitary representation π\pi of G. It is a continuous function on G.

Corollary 1 03DS

Let π\pi be an irreducible unitary representation of G in a Hilbert space E. The character of π\pi is a hermitian element of the involutive algebra L1(G)L^1(G).

Let x∈Gx\in G. Since G is unimodular, one has χπ∗(x)=\chi^*_{\pi}(x) = Tr(π(x−1)\pi (x^{-1})), whence χπ∗(x)=\chi^*_{\pi}(x) = Tr(π(x)\pi (x)) since π\pi is a unitary representation.

Corollary 2 03DT

a) Let π\pi be an irreducible unitary representation of G in a Hilbert space E. One has

∫′\int' 1 ′′''

⟨x∣π(g)x′⟩⟨y∣π(g)y⟩dμ(g)=⟨x∣y⟩⟨x∣y⟩ \langle x|\pi (g)x'\rangle \langle y|\pi (g)y\rangle d\mu(g) =\langle x|y\rangle \langle x|y\rangle

G_G dim(E)

for every (x,y,x′,y′)∈E4(x, y, x', y')\in E^4;

b) Let π1(\pi_1(resp. π2)\pi_2) be an irreducible unitary representation of G in a Hilbert space E1(E_1(resp. E2)E_2). If π1\pi_1 and π2\pi_2 are not isomorphic, one has

∫G⟨x∣π1(g)x′⟩⟨y∣π2(g)y′⟩dμ(g)=0 \int_G\langle x|\pi_1(g)x'\rangle \langle y|\pi_2(g)y'\rangle d\mu(g) = 0

for every (x,x′,y,y′)∈E12×E22(x, x', y, y')\in E^2_1\times E^2_2.

This follows at once from Prop. 8 of V, p. 424 and Prop. 9 of V, p. 424, taking into account Lemma 1 and the formula cμ(π)=c_\mu(\pi ) = dim(E) (Prop. 2).

Corollary 3 03DU

Let π1\pi_1 and π2\pi_2 be irreducible representations of G. In the Hilbert space L2(G)L^2(G), one has ⟨χπ1∣χπ2⟩=1\langle \chi_{\pi_1}|\chi_{\pi_2}\rangle = 1 if π1\pi_1 and π2\pi_2 are isomorphic and ⟨χπ1∣χπ2⟩=0\langle \chi_{\pi_1}|\chi_{\pi_2}\rangle = 0 otherwise. In other words, the family of the characters of the classes π∈G^\pi \in \widehat{G} is an orthonormal family in L2(G)L^2(G).

This results from Proposition 1 of V, p. 457 and Corollary 2, noting that for every orthonormal basis (ei)i∈I(e_i)_{i\in I} of the space of a finite-dimensional unitary representation π\pi of G, and for every g∈Gg\in G, one has the formula

χπ(g)=∑i∈I⟨ei∣π(g)ei⟩ \chi_{\pi}(g) =\sum_{i\in I}\langle e_i|\pi (g)e_i\rangle

Corollary 4 03DV

a) Let ϱ\varrho be a finite-dimensional continuous representation of G. One has

χϱ=∑π∈G^\chi_{\varrho}=\sum_{\pi\in\widehat{G}} dim(HomG(π,ϱ)_G(\pi , \varrho ))χπ=∑π∈G^\chi_{\pi}=\sum_{\pi\in\widehat{G}} dim(HomG(ϱ,π)_G(\varrho , \pi ))χπ\chi_{\pi}.

b) Let ϱ1\varrho_1 and ϱ2\varrho_2 be finite-dimensional continuous representations of G. One has

⟨χϱ1∣χϱ2⟩=\langle \chi_{\varrho_1}|\chi_{\varrho_2}\rangle = dim(HomG(ϱ1,ϱ2)_G(\varrho_1, \varrho_2)) == dim(HomG(ϱ2,ϱ1)_G(\varrho_2, \varrho_1)).

c) A finite-dimensional continuous representation ϱ\varrho of G is irreducible if and only if ∥χϱ∥2=1\|\chi_{\varrho}\|^2= 1.

Since ϱ\varrho is semisimple, it is isomorphic to the direct sum of its π\pi-isotypic components Mπ(ϱ)M_{\pi}(\varrho ) for π∈G^\pi \in \widehat{G}. Denoting by mπ(ϱ)m_{\pi}(\varrho ) the multiplicity of π\pi in ϱ\varrho (Def. 10 of V, p. 398), one then has

χϱ=∑π∈G^mπ(ϱ)χπ \chi_{\varrho}=\sum_{\pi\in\widehat{G}}m_{\pi}(\varrho )\chi_{\pi}

Assertion a) therefore results from the fact that

mπ(ϱ)=m_{\pi}(\varrho ) = dim HomG(π,ϱ)=_G(\pi , \varrho ) = dim HomG(ϱ,π)_G(\varrho , \pi )

(formula (1) of V, p. 377 and Corollary 2 of V, p. 387).

By bilinearity and orthonormality of the characters, assertion a) implies that

⟨χϱ1∣χϱ2⟩=∑π∈G^\langle \chi_{\varrho_1}|\chi_{\varrho_2}\rangle =\sum_{\pi\in\widehat{G}} dim(HomG(π,ϱ1)_G(\pi , \varrho_1)) dim(HomG(π,ϱ2)_G(\pi , \varrho_2)), and on the other hand one has canonical isomorphisms

HomG(ϱ1,ϱ2)→⨁(π1,π2)∈G^×G^_G(\varrho_1, \varrho_2)\rightarrow \bigoplus_{(\pi_1,\pi_2)\in\widehat{G}\times\widehat{G}} HomG(Mπ1(ϱ1),Mπ2(ϱ2))_G(M_{\pi_1}(\varrho_1),M_{\pi_2}(\varrho_2))

→⨁π∈G^\rightarrow \bigoplus_{\pi\in\widehat{G}} HomG(Mπ(ϱ1),Mπ(ϱ2))_G(M_{\pi}(\varrho_1),M_{\pi}(\varrho_2))

(formula (1) of V, p. 377), whence it follows that

dim(HomG(ϱ1,ϱ2)_G(\varrho_1, \varrho_2)) =∑π∈G^mπ(ϱ1)mπ(ϱ2)=\sum_{\pi\in\widehat{G}}m_{\pi}(\varrho_1)m_{\pi}(\varrho_2),

whence assertion b). Assertion c) also results from a) and Schur’s Lemma (prop. 6 of V, p. 386).

Corollary 5 03DW

Let π1\pi_1 and π2\pi_2 be irreducible unitary representations of G. One has π1(χ‾π2)=0\pi_1(\overline{\chi}_{\pi_2}) = 0 if π1\pi_1 is not isomorphic to π2\pi_2, and π1(χ‾π1)\pi_1(\overline{\chi}_{\pi_1}) is multiplication by 1/1/ dim(π1\pi_1).

The character of π2\pi_2 is a continuous central function on G, hence the linear mapping uu = π1(χ‾π2)\pi_1(\overline{\chi}_{\pi_2}) is defined and belongs to the space HomG(π1,π1)_G(\pi_1, \pi_1). It is a homothety by Schur’s Lemma (prop. 6 of V, p. 386), whose trace is

Tr(uu) == Tr(∫Gχπ2(g)‾π1(g)dμ(g))(\int_G\overline{\chi_{\pi_2}(g)}\pi_1(g)d\mu(g))

=∫Gχπ2(g)‾χπ1(g)dμ(g) =\int_G\overline{\chi_{\pi_2}(g)}\chi_{\pi_1}(g)d\mu(g)

By the orthogonality relations, the trace of uu is therefore zero if π1\pi_1 is not isomorphic to π2\pi_2, and equal to 1 otherwise. The assertion follows.

Remark 03DX

When G is finite, Schur’s orthogonality relation (resp. the orthogonality formula for characters) coincides with that of A, VIII, p. 399 (resp. that of A, VIII, p. 400, prop. 4). On the other hand, the “second orthogonality relation” for the characters of finite groups (A, VIII, p. 402, formula (32)) has no exact analogue when G is compact.

For finite G, the particular case of the second orthogonality formula corresponding to the conjugacy class of ee is the formula

Card G1 ∑π∈G^\sum_{\pi\in\widehat{G}} dim(π\pi )χπ(g)=\chi_{\pi}(g) = 10 sisinong=,eg=,e which is also interpreted as the calculus of the character of the regular representation γG\boldsymbol{\gamma }_G of G, and which is equivalent to the formula Tr(γG(f)\boldsymbol{\gamma }_G(f)) =f(e)=f(e) for every function ff from G into C\mathbf{C}.

Let G again be any compact group. For every function f∈Cf\in \mathscr{C}(G), the endomorphism γG(f)\boldsymbol{\gamma }_G(f) of L2(G)L^2(G) coincides with the endomorphism φ↦f∗φ\varphi \mapsto f*\varphi, and it is of finite trace (lemma 4 of V, p. 407 and corollary 2 of III, p. 33). By loc. cit. and Theorem 2 of IV, p. 177, one also has

Tr(γG(f)\boldsymbol{\gamma }_G(f)) =∫Gf(x−1x)dμ(x)=f(e)=\int_Gf(x^{-1}x)d\mu(x) =f(e).

Proposition 3 03DY

Suppose that G=G1×G2G = G_1\times G_2 where G1G_1 and G2G_2 are compact groups. The mapping bb from G^1×G^2\widehat{G}_1\times \widehat{G}_2 into G^\widehat{G} which to (π1,π2)(\pi_1, \pi_2) associates the class of π1⊠π2\pi_1\boxtimes \pi_2 is a bijection.

By cor. 6 of V, p. 389, the mapping bb is well defined.

Let π1\pi_1 and π2\pi_2 be elements of G^\widehat{G}. Let π∈G^\pi \in \widehat{G}. The π\pi-isotypical component of the restriction ϖ\varpi of π1⊠π2\pi_1\boxtimes \pi_2 to G1×{e}G_1\times \{e\} is equal to ϖ\varpi if π1=π\pi_1=\pi and is zero in the contrary case (lemma 8 of V, p. 384). Thus, the unitary representation π1⊠π2\pi_1\boxtimes \pi_2 determines π1\pi_1 up to isomorphism; analogously, it determines π2\pi_2, which proves that bb is injective.

Let us prove that bb is surjective. Let π\pi be an irreducible unitary representation of G1×G2G_1\times G_2 in a Hilbert space E. It is finite-dimensional (prop. 2). Let π1\pi_1 be an irreducible unitary representation of G1G_1 in a Hilbert space E1E_1 such that the restriction of π\pi to G1×{e}G_1\times \{e\} contains a subrepresentation isomorphic to π1\pi_1 (lemma 3 of V, p. 379). Let F = HomG1(π1,π)_{G_1}(\pi_1, \pi ). It is a nonzero finite-dimensional vector space. For h∈G2h\in G_2 and u∈Fu\in F, let ϱ(h)(u)\varrho (h)(u) be the linear mapping from E1E_1 into E defined by x↦π(e,h)(u(x))x\mapsto \pi (e, h)(u(x)). Since G1×{e}G_1\times \{e\} and {e}×G2\{e\} \times G_2 commute in G, the mapping ϱ(h)(u)\varrho (h)(u) belongs to the space F. The mapping ϱ\varrho is a linear representation of G2G_2 in the space F; it is continuous. Since F is not reduced to 0, there exists an irreducible subrepresentation π2\pi_2 of ϱ\varrho (lemma 3 of V, p. 379). Let E2E_2 be the space of π2\pi_2. The linear mapping v:E1⊗E2→Ev: E_1\otimes E_2\rightarrow E such that x⊗u↦u(x)x\otimes u\mapsto u(x) for x∈E1x\in E_1 and u∈E2u\in E_2 is then a nonzero G-morphism of π1⊠π2\pi_1\boxtimes \pi_2 into π\pi; since the representations π\pi and π1⊠π2\pi_1\boxtimes \pi_2 are irreducible and finite-dimensional, the morphism vv is an isomorphism (lemma 2 of V, p. 378).

This proposition is to be compared with theorem 1 of A, VIII, p. 208.

3. The Peter–Weyl theorem

Recall that Θ(G) denotes the space of matrix coefficients of the finite-dimensional unitary representations of G. The space Θ(G) is a unital subalgebra of C(G)\mathscr{C}(G) stable under complex conjugation (prop. 5 of V, p. 386).

For π∈G^\pi \in \widehat{G}, let ϱG(π)\boldsymbol{\varrho }_G(\pi ) denote the subspace of C(G)\mathscr{C}(G) generated by the matrix coefficients of π\pi. We identify it with a subspace of L2(G)L^2(G).

The space Θ(G) coincides with the sum of the spaces ϱG(π)\boldsymbol{\varrho }_G(\pi ) for π∈G^\pi \in \widehat{G} (in fact, every element of Θ(G) is a sum of matrix coefficients of irreducible representations of G since the finite-dimensional representations of G are semisimple by prop. 1 of V, p. 457). Moreover, this sum is direct since, by cor. 2 of V, p. 458, the spaces ϱG(π)\boldsymbol{\varrho }_G(\pi ) are pairwise orthogonal.

Proposition 4 03DZ

The space Θ(G) is dense in C(G)\mathscr{C}(G) and in L2(G)L^2(G).

The unital subalgebra Θ(G) of C(G)\mathscr{C}(G) is stable under conjugation. It coincides with the subalgebra Υ(G) by prop. 2 of V, p. 457, hence it separates the points of G (cor. of theorem 4 of V, p. 454). Consequently, it is dense in C(G)\mathscr{C}(G) by TG, X, p. 40, cor. 2, and, fortiori, it is dense in the space L2(G)(L^2(G) (cf. INT, IV, p. 155, §4, no7^o7, prop. 13).

Recall that the unitary biregular representation of G is the representation ϱG\boldsymbol{\varrho }_G of G×GG\times G in L2(G)L^2(G) such that (g1,g2)↦γG(g1)δG(g2)(g_1, g_2)\mapsto \boldsymbol{\gamma }_G(g_1)\boldsymbol{\delta }_G(g_2).

Proposition 5 03E0

Let π∈G^\pi \in \widehat{G}. The space ϱG(π)\boldsymbol{\varrho }_G(\pi ) is a subrepresentation of ϱG\boldsymbol{\varrho }_G which is isomorphic to π‾⊠π\overline{\pi}\boxtimes \pi. In particular, it is an irreducible representation of G×GG\times G.

By prop. 7 of V, p. 422 (applied with Z={e}Z =\{e\}), the space ϱG(π)\boldsymbol{\varrho }_G(\pi ) is a subrepresentation of ϱG\boldsymbol{\varrho }_G and the linear mapping from E‾π⊗Eπ\overline{E}_{\pi}\otimes E_{\pi} into L2(G)L^2(G) which associates to x⊗yx\otimes y the matrix coefficient g↦⟨x∣π(g)y⟩g\mapsto \langle x|\pi (g)y\rangle is a (G×(G\times G)-isomorphism of π‾⊠π\overline{\pi}\boxtimes \pi onto ϱG(π)\boldsymbol{\varrho }_G(\pi ). The subrepresentation ϱG(π)\boldsymbol{\varrho }_G(\pi ) is therefore irreducible (prop. 3 of V, p. 461).

Theorem 1 (Peter–Weyl) 03E1

Let G be a compact topological group. The biregular representation ϱG\boldsymbol{\varrho }_G of G is the Hilbert sum of the subrepresentations ϱG(π)\boldsymbol{\varrho }_G(\pi ) for π∈G^\pi \in \widehat{G}.

The spaces ϱG(π)\boldsymbol{\varrho }_G(\pi ) are pairwise orthogonal; they are irreducible subrepresentations of the biregular representation of G (prop. 5) which are pairwise non-isomorphic (prop. 3 of V, p. 461). The theorem then results from the fact that the sum Θ(G) of the spaces ϱG(π)\boldsymbol{\varrho }_G(\pi ) for π\pi ranging over G^\widehat{G} is dense in L2(G)L^2(G) (prop. 4).

Corollary 1 03E2

For every π∈G^\pi \in \widehat{G}, the subrepresentation ϱG(π)\boldsymbol{\varrho }_G(\pi ) is the (π‾⊠π)(\overline{\pi}\boxtimes \pi )-isotypical component of ϱG\boldsymbol{\varrho }_G.

Let F be the (π‾⊠π(\overline{\pi}\boxtimes \pi)-isotypical component of ϱG\boldsymbol{\varrho }_G. The space F contains ϱG(π)\boldsymbol{\varrho }_G(\pi ) (prop. 5). Moreover, for every τ∈G^\tau \in \widehat{G} different from π\pi, the intersection of F and ϱG(τ)\boldsymbol{\varrho }_G(\tau ) is zero (prop. 3 of V, p. 461). It follows that F=ϱG(π)F =\boldsymbol{\varrho }_G(\pi ) by applying the theorem.

Corollary 2 03E3

Let π1\pi_1 and π2\pi_2 be non-isomorphic irreducible representations of G. The (π‾1⊠π2)(\overline{\pi}_1\boxtimes \pi_2)-isotypical component of ϱG\boldsymbol{\varrho }_G is zero.

Corollary 3 03E4

The right (resp. left) regular representation of G is isomorphic to the Hilbert sum

π⨁∈G^π\pi \bigoplus\in \widehat{G}\pidim(π\pi ).

By the theorem and proposition 5, the restriction of the biregular representation of G to the subgroup H={e}×GH =\{e\} \times G is isomorphic to the Hilbert sum of the restrictions to H of the representations π‾⊠π\overline{\pi}\boxtimes \pi for π∈G^\pi \in \widehat{G}. These are isomorphic to the direct sum of dim(π\pi ) copies of π\pi (lemma 8 of V, p. 384). This implies the result for the right regular representation, and the case of the left regular representation is similar.

Corollary 4 03E5

Let π∈G^\pi \in \widehat{G}. The π\pi-isotypical component of δG\boldsymbol{\delta }_G (resp. the π‾\overline{\pi}-isotypical component of γG\boldsymbol{\gamma }_G) is equal to ϱG(π)\boldsymbol{\varrho }_G(\pi ).

The argument is similar to that of the preceding corollary, considering ϱG(π)\boldsymbol{\varrho }_G(\pi ) as a subrepresentation of δG\boldsymbol{\delta }_G (resp. of γG\boldsymbol{\gamma }_G).

Remark 1 03E6

If G is finite, these statements correspond to the results of A, VIII, p. 398, remark.

Remark 2 03E7

Suppose that G is commutative. The set G^\widehat{G} is identified with the dual group of G (V, p. 393, remark). For every χ∈G^\chi \in \widehat{G}, the space of ϱG(χ)\boldsymbol{\varrho }_G(\chi ) is the 1-dimensional subspace of L2(G)L^2(G) generated by χ\chi. Theorem 1 for G is therefore then equivalent to the corollary of theorem 1 of II, p. 215.

Remark 3 03E8

If there exists an integer n⩾0n\geqslant 0 such that G is a compact subgroup of GL(Cn)\mathbf{G}\mathbf{L}(\mathbf{C}^n), the identical representation of G in GL(Cn)\mathbf{G}\mathbf{L}(\mathbf{C}^n) suffices to separate the points of G, and one can therefore then prove proposition 4 directly, and then the Peter–Weyl theorem, without appealing to the Gelfand–Raikov theorem.

Remark 4 03E9

The Peter–Weyl theorem implies in particular that the natural continuous homomorphism of G into ∏π∈G^U(Eπ)\prod_{\pi\in\widehat{G}}\mathbf{U}(E_{\pi}) is injective.

4. Matrix coefficients and G-finite functions

Proposition 6 03EA

The following subspaces of L2(G)L^2(G) are equal:

a) The space Θ(G) ;

b) The algebraic direct sum of the subspaces ϱG(π)\boldsymbol{\varrho }_G(\pi ) of L2(G)L^2(G);

c) The space of G-finite vectors (cf. V, p. 376) of γG\boldsymbol{\gamma }_G;

d) The space of G-finite vectors of δG\boldsymbol{\delta }_G;

e) The space of (G×G)(G\times G)-finite vectors of ϱG\boldsymbol{\varrho }_G.

In particular, every G-finite vector of γG,δG\boldsymbol{\gamma }_G,\boldsymbol{\delta }_G or ϱG\boldsymbol{\varrho }_G belongs to C(G)\mathscr{C}(G).

Let us denote by FaF_a (resp. Fb,Fc,Fd,FeF_b, F_c, F_d, F_e) the space defined by condition a) (resp. b), c), d), e)). We have already observed that Fa=FbF_a= F_b.

We have Fb⊂FcF_b\subset F_c because ϱG(π)\boldsymbol{\varrho }_G(\pi ) is a finite-dimensional subrepresentation of γG\boldsymbol{\gamma }_G for every π∈G^\pi \in \widehat{G}. Conversely, let ff be a G-finite vector of γG\boldsymbol{\gamma }_G. The subspace EfE_f generated by the functions γG(g)f\boldsymbol{\gamma }_G(g)f for g∈Gg\in G is a finite-dimensional subrepresentation of γG\boldsymbol{\gamma }_G. It is equal to the direct sum of its π\pi-isotypic components for π∈G^\pi \in \widehat{G} (Prop. 1 of V, p. 457). By Cor. 4 of V, p. 463, this implies that EfE_f is contained in the sum of the spaces ϱG(π)\boldsymbol{\varrho }_G(\pi ), hence Fc⊂FbF_c\subset F_b.

By analogous reasoning, one obtains Fb=FdF_b= F_d, and since ϱG(π)\boldsymbol{\varrho }_G(\pi ) is a subrepresentation of ϱG\boldsymbol{\varrho }_G, one establishes in the same way that Fb=FeF_b= F_e.

Corollary 03EB

Let E⊂L2(G)E\subset L^2(G) be a finite-dimensional vector subspace defining a subrepresentation of γG(\boldsymbol{\gamma }_G(resp. of δG\boldsymbol{\delta }_G, of ϱG)\boldsymbol{\varrho }_G). Then E is contained in C(G)\mathscr{C}(G).

In fact, every element of E is a G-finite vector of the representation γG\boldsymbol{\gamma }_G.

5. Representations in a separated quasi-complete space

Proposition 7 03EC

Let ϱ\varrho be a continuous representation of G in a separated locally convex quasi-complete space E. The sum of the finite-dimensional subrepresentations of E is dense in E.

Let x∈Ex\in E and let U be an open neighbourhood of xx. The set of measures ν∈M(G)\nu \in \mathscr{M}(G) such that ϱ(ν)x∈U\varrho (\nu )x\in U is open in M(G)\mathscr{M}(G) for the topology of compact convergence (cf. no2^o2 of V, p. 400). It contains εe\varepsilon_e, hence it contains a measure of the form ν=f1⋅μ\nu =f_1\cdot \mu where f1∈C(G)f_1\in \mathscr{C}(G) (INT, VIII, p. 171, § 4, no7^o7, Prop. 19) and, consequently, it contains a measure of the form f2⋅μf_2\cdot \mu where f2f_2 is a G-finite function (Prop. 6 of V, p. 464 and Prop. 4 of V, p. 462).

The subrepresentation F of γG\boldsymbol{\gamma }_G generated by f2f_2 is finite-dimensional. Let F~\widetilde{F} be the image of the linear mapping f↦ϱ(f)xf\mapsto \varrho (f)x of F into E. The space F~\widetilde{F} is finite-dimensional, and it contains the element ϱ(f2)x\varrho (f_2)x of U. Since ϱ(g)ϱ(f)=ϱ(γG(g)f)\varrho (g)\varrho (f) =\varrho (\boldsymbol{\gamma }_G(g)f) for every g∈Gg\in G and every f∈Ff\in F (formula (1) of V, p. 406), the space F~\widetilde{F} is a subrepresentation of ϱ\varrho which meets U. The proposition is proved.

Corollary 1 03ED

Let π\pi be an irreducible continuous representation of G in a separated quasi-complete locally convex space E. The space E is finite-dimensional.

Corollary 2 03EE

Let ϱ\varrho be a continuous representation of G in a separated quasi-complete locally convex space E and let π\pi be an irreducible continuous representation of G.

a) The G-morphism pπ=p_{\pi}= dim(π\pi )ϱ(χ‾π)\varrho (\overline{\chi}_{\pi}) of E into E is a continuous projector of E whose image is the π\pi-isotypical component of ϱ\varrho;

b) If ϱ\varrho is a unitary representation, then the projector pπp_{\pi} is the orthoprojector of E with image Mπ(ϱ)M_{\pi}(\varrho ).

Let us prove a). The linear mapping pπ=p_{\pi}= dim(π\pi )ϱ(χ‾π)\varrho (\overline{\chi}_{\pi}) is well-defined, since E is quasi-complete and G is compact (V, p. 401). It is an element of HomG(ϱ,ϱ)_G(\varrho , \varrho ) since the character of π\pi is a continuous central function on G.

Let ϖ\varpi be a finite-dimensional subrepresentation of E and F its space. The mapping pπp_{\pi} induces, by passing to subspaces, the endomorphism (dim π\pi )ϖ(χ‾π)\varpi (\overline{\chi}_{\pi}) of F. This endomorphism is therefore zero if ϖ\varpi is not isomorphic to π\pi, and is the identity mapping of F otherwise (in fact, this is the case if ϖ\varpi is irreducible by Cor. 5 of V, p. 460, and the general case follows from this since ϖ\varpi is semisimple by Prop. 1 of V, p. 457).

Finally, since the sum of the finite-dimensional subrepresentations of E is dense in E (Prop. 7) and pπp_{\pi} is continuous, it follows that pπp_{\pi} is a projector whose image is the π\pi-isotypical component of ϱ\varrho. If E is Hilbert space, the projector pπp_{\pi} is hermitian by Cor. 1 of V, p. 458, hence it is an orthoprojector (Lemma 3, (ii) of I, p. 133).

Corollary 3 03EF

Let ϱ\varrho be a continuous representation of G in a separated quasi-complete locally convex space E. The endomorphism

x↦∫Gϱ(g)xdμ(g) x\mapsto \int_G\varrho (g)x d\mu(g)

of E is a projector of E whose image is the space EGE^G of invariant vectors in E. In particular, if E is finite-dimensional, then

dim(EG^G) =∫Gχϱ(g)dμ(g)=\int_G\chi_{\varrho}(g)d\mu(g).

The first assertion is the particular case of the preceding corollary when π\pi is the trivial representation of dimension 1 of G. The second follows from it since the dimension of EGE^G is the trace of the orthoprojector onto EGE^G, that is,

dim(EG^G) == Tr(∫Gϱ(g)dμ(g))=∫Gχϱ(g)dμ(g)(\int_G\varrho (g)d\mu(g))=\int_G\chi_{\varrho}(g)d\mu(g).

In particular, when E is finite-dimensional, there exists a vector x≠0x\not = 0 in E such that ϱ(g)x=x\varrho (g)x=x for every g∈Gg\in G if and only if

∫Gχϱ(g)dμ(g)≠0 \int_G\chi_{\varrho}(g)d\mu(g)\not = 0

Corollary 4 03EG

Let ϱ\varrho be a unitary representation of G in a Hilbert space E. The space E is the Hilbert sum of the π\pi-isotypical components Mπ(ϱ)M_{\pi}(\varrho ) for π∈G^\pi \in \widehat{G}. In particular, the representation ϱ\varrho is semisimple.

The isotypical components of ϱ\varrho corresponding to non-isomorphic irreducible representations of G are orthogonal (prop. 8 of V, p. 394). Since every finite-dimensional unitary representation of G is semisimple (prop. 1 of V, p. 457), the sum of the isotypical components Mπ(ϱ)M_{\pi}(\varrho ) for π∈G^\pi \in \widehat{G} is dense in E (prop. 7). The corollary follows.

6. Characters and Conjugacy Classes

Let ϱ\varrho be a unitary representation of G in a finite-dimensional Hilbert space E. The character of ϱ\varrho satisfies ∣χϱ∣⩽|\chi_{\varrho}|\leqslant dim(E). Let (ei)i∈I(e_i)_{i\in I} be an orthonormal basis of E; the character χϱ\chi_{\varrho} is the sum of the diagonal matrix coefficients g↦⟨ei∣ϱ(g)ei⟩g\mapsto \langle e_i|\varrho (g)e_i\rangle of ϱ\varrho. In particular, the character of ϱ\varrho is a G-finite function, and if ϱ\varrho is irreducible, then χϱ∈ϱG(ϱ)\chi_{\varrho}\in \boldsymbol{\varrho }_G(\varrho ).

We have the following formulas

χϱ1⊕ϱ2=χϱ1+χϱ2,χϱ1⊗ϱ2=χϱ1χϱ2,χϱ˘=χϱ‾=χ‾ϱ \chi_{\varrho_1\oplus\varrho_2}=\chi_{\varrho_1}+\chi_{\varrho_2},\chi_{\varrho_1\otimes\varrho_2}=\chi_{\varrho_1}\chi_{\varrho_2},\chi_{\breve{\varrho}}=\chi_{\overline{\varrho}}=\overline{\chi}_{\varrho}

(cf. A, VIII, p. 388–389).

Proposition 8 03EH

One has

(1) χπ∗χσ=0\chi_{\pi}*\chi_{\sigma}= 0 for all π,σ\pi , \sigma belonging to G^,π≠σ\widehat{G}, \pi \not =\sigma,

(2) χπ∗χπ=\chi_{\pi}*\chi_{\pi}= dim(1π1\pi )χπ\chi_{\pi} for every π\pi belonging to G^\widehat{G}.

Let π\pi and σ\sigma be irreducible representations of G. One has

dim(π\pi )(χπ∗χσ)=(\chi_{\pi}*\chi_{\sigma}) = dim(π\pi )γG(χπ)χσ\boldsymbol{\gamma }_G(\chi_{\pi})\chi_{\sigma}

(lemma 4 of V, p. 407). The function dim(π\pi )γG(χπ)χσ\boldsymbol{\gamma }_G(\chi_{\pi})\chi_{\sigma} is the orthogonal projection of χσ\chi_{\sigma} on the π‾\overline{\pi}-isotypical component of γG\boldsymbol{\gamma }_G (cor. 2 of V, p. 465), that is, on ϱG(π)\boldsymbol{\varrho }_G(\pi ) (cor. 4 of V, p. 463). Since χσ\chi_{\sigma} belongs to ϱG(σ)\boldsymbol{\varrho }_G(\sigma ), the result follows from theorem 1 of V, p. 462.

Lemma 2 03EI

The graph of the equivalence relation “ x∈Gx\in G and y∈Gy\in G and xx is conjugate to yy ” in G is closed.

In fact, this graph is the image of the continuous mapping of the compact space G×GG\times G into itself defined by (x,y)↦(x,yxy−1)(x, y)\mapsto (x, yxy^{-1}).

We denote by G♯G^{\sharp} the space of conjugacy classes of G endowed with the quotient topology; it is a compact space by lemma 2 and TG, I, p. 78, prop. 8. Let ϖ:G→G♯\varpi : G\rightarrow G^{\sharp} be the canonical projection. The mapping of C(G♯)\mathscr{C}(G^{\sharp}) into C(G)\mathscr{C}(G) defined by f↦f∘ϖf\mapsto f\circ \varpi identifies the involutive algebra C(G♯)\mathscr{C}(G^{\sharp}) with the involutive subalgebra of C(G)\mathscr{C}(G) formed by the continuous central functions.

The measures on G♯G^{\sharp} are identified with the central measures on G (V, p. 402, def. 1).

The linear form on C(G♯)\mathscr{C}(G^{\sharp}) defined by f↦∫Gff\mapsto \int_Gf is then a positive measure of mass 1 on G♯G^{\sharp}, denoted by μ♯\mu^{\sharp}. For every p∈[1,+∞]p\in [1,+\infty ], we denote by Lp(G♯)\mathscr{L}^p(G^{\sharp}) (resp. Lp(G♯)L^p(G^{\sharp})) the space LCp(G♯,μ♯)\mathscr{L}_{\mathbf{C}}^p(G^{\sharp}, \mu^{\sharp}) (resp. LCp(G♯,μ♯)L^p_{\mathbf{C}}(G^{\sharp}, \mu^{\sharp})). We identify Lp(G♯)L^p(G^{\sharp}) with the closure of C(G♯)\mathscr{C}(G^{\sharp}) in Lp(G)L^p(G); in particular, it is a closed subspace of Lp(G)L^p(G).

We also denote by Θ(G♯)=Θ(G)∩C(G♯)\Theta (G^{\sharp}) = \Theta (G)\cap \mathscr{C}(G^{\sharp}) the space of central matrix coefficients of G. It is a unital involutive subalgebra of C(G♯)\mathscr{C}(G^{\sharp}).

When G is a Lie group, the space Θ(G♯)\Theta (G^{\sharp}) is also denoted by ZΘ(G) in LIE, IX, p. 71.

Lemma 3 03EJ

Let π\pi be an irreducible unitary representation of G. The vector space L2(G♯)∩ϱG(π)L^2(G^{\sharp})\cap \boldsymbol{\varrho }_G(\pi ) has dimension one and is generated by the character of π\pi.

Consider the unitary representation σ\sigma of G in the space ϱG(π)\boldsymbol{\varrho }_G(\pi ) defined by σ(g)=ϱG(g,g)\sigma (g) =\boldsymbol{\varrho }_G(g, g). Since the unitary representation ϱG(π)\boldsymbol{\varrho }_G(\pi ) of G×GG\times G is isomorphic to π‾⊠π\overline{\pi}\boxtimes \pi (prop. 5 of V, p. 462), its character is given by

χσ(g)=χπ‾⊠π(g,g)=∣χπ(g)∣2 \chi_{\sigma}(g) =\chi_{\overline{\pi}\boxtimes\pi}(g, g) =|\chi_{\pi}(g)|^2

for every g∈Gg\in G. The space L2(G♯)∩ϱG(π)L^2(G^{\sharp})\cap \boldsymbol{\varrho }_G(\pi ) is the subspace of the elements invariant under this representation, and its dimension is equal to

∫Gχσ(g)dμ(g)=∫G∣χπ(g)∣2dμ(g)=1 \int_G\chi_{\sigma}(g)d\mu(g) =\int_G|\chi_{\pi}(g)|^2d\mu(g) = 1

(cor. 3 of V, p. 466 and cor. 3 of V, p. 459). Since the character of π\pi is a nonzero element of L2(G♯)∩ϱG(π)L^2(G^{\sharp})\cap \boldsymbol{\varrho }_G(\pi ), it is a basis of this space.

Proposition 9 03EK

The family (χπ)π∈G^(\chi_{\pi})_{\pi\in\widehat{G}} of the characters of the irreducible unitary representations of G is an orthonormal basis of L2(G♯)L^2(G^{\sharp}).

By the Peter-Weyl theorem (th. 1 of V, p. 462), the closed subspace L2(G♯)L^2(G^{\sharp}) of L2(G)L^2(G) formed by the elements invariant under the unitary representation g↦ϱG(g,g)g\mapsto \boldsymbol{\varrho }_G(g, g) of G is the Hilbert sum of the subspaces L2(G♯)∩ϱG(π)L^2(G^{\sharp})\cap \boldsymbol{\varrho }_G(\pi ) for π∈G^\pi \in \widehat{G}. The proposition therefore follows from the preceding lemma and cor. 3 of V, p. 459.

Corollary 1 03EL

The vector space Θ(G♯)\Theta (G^{\sharp}) is dense in C(G♯)\mathscr{C}(G^{\sharp}) and in L2(G♯)L^2(G^{\sharp}).

The second assertion follows from prop. 9. As to the first assertion, consider the linear representation ϱ\varrho of G on the Banach space C(G)\mathscr{C}(G) defined by

ϱ(g)f(x)=f(g−1xg) \varrho (g)f(x) =f(g^{-1}xg)

for every f∈C(G)f\in \mathscr{C}(G) and every x∈Gx\in G. It is a continuous isometric representation, and C(G♯)\mathscr{C}(G^{\sharp}) is the space of the elements invariant under this representation. The continuous projector p=ϱ(1)p=\varrho (1) of C(G)\mathscr{C}(G) therefore has image C(G♯)\mathscr{C}(G^{\sharp}) (cor. 3 of V, p. 466); it has norm ⩽1\leqslant 1.

Let f∈C(G♯)f\in \mathscr{C}(G^{\sharp}) and let ε>0\varepsilon >0. There exists f~∈Θ(G)\widetilde{f}\in \Theta (G) such that ∥f−f~∥⩽ε\|f-\widetilde{f}\|\leqslant \varepsilon (prop. 4 of V, p. 462), and then ∥f−p(f~)∥=∥p(f−f~)∥⩽ε\|f-p(\widetilde{f})\|=\|p(f-\widetilde{f})\|\leqslant \varepsilon; since p(f~)∈C(G♯)p(\widetilde{f})\in \mathscr{C}(G^{\sharp}), we conclude that Θ(G♯)\Theta (G^{\sharp}) is dense in C(G♯)\mathscr{C}(G^{\sharp}).

Let R(G)R(G) denote the Grothendieck ring of continuous representations of GG in finite-dimensional separated complex vector spaces (cf. LIE, IX, p. 70 and A, VIII, p. 182, applied to the additive class of continuous representations of GG in finite-dimensional separated complex vector spaces, viewed as C\mathbf{C}[G]-modules). Since every continuous linear representation of GG in a finite-dimensional separated topological vector space is semisimple (prop. 1 of V, p. 457), the abelian group R(G)R(G) is free and the classes of irreducible unitary representations π∈G^\pi \in \widehat{G} form a basis of it (A, VIII, p. 186, prop. 7).

Corollary 2 03EM

a) The family of characters of the irreducible representations of GG is a basis of Θ(G♯)\Theta (G^{\sharp});

b) The mapping u:R(G)⊗ZC→Θ(G♯)u: R(G)\otimes_{\mathbf{Z}}\mathbf{C}\rightarrow \Theta (G^{\sharp}) such that u(π⊗1)=χπu(\pi \otimes 1) =\chi_{\pi} for every π∈G^\pi \in \widehat{G} is an isomorphism of algebras over C\mathbf{C}.

The first assertion results from Proposition 9 and Corollary 1.

Since the classes of the representations π∈G^\pi \in \widehat{G} form a basis of the free Z\mathbf{Z}-module R(G)R(G), the mapping uu is well-defined. It is a morphism of C\mathbf{C}-algebras, and it is an isomorphism by a).

Corollary 3 03EN

Let HH be a locally compact topological group such that GG is a compact subgroup of HH. Let ϱ\varrho be a unitary representation of HH in a Hilbert space EE. If the π\pi-isotypical component of the restriction of ϱ\varrho to GG is finite-dimensional for every π∈G^\pi \in \widehat{G}, then the representation ϱ\varrho of HH is semisimple and every irreducible unitary representation of HH occurs with finite multiplicity in ϱ\varrho.

Let us denote by σ\sigma the restriction of ϱ\varrho to the compact subgroup GG of HH. Let π\pi be an irreducible representation of GG. The endomorphism σ(χπ)∈L(E)\sigma (\chi_{\pi})\in \mathscr{L}(E) is of finite rank, since its image is the π‾\overline{\pi}-isotypical component of σ\sigma (cor. 2 of V, p. 465) and this is finite-dimensional by hypothesis. Consequently, the endomorphism σ(f)\sigma (f) is of finite rank for every f∈Θ(G♯)f\in \Theta (G^{\sharp}) and is compact for every f∈C(G♯)f\in \mathscr{C}(G^{\sharp}) (cor. 1 of V, p. 468 and cor. of prop. 2 of III, p. 4).

Let jj denote the canonical injection of GG into HH. It is a continuous mapping which is ν\nu-proper for every measure ν\nu on GG (INT, V, p. 69, § 6, No.o1^o1, Remark 1). Let B\mathfrak{B} be the filter of compact neighbourhoods of the element ee in GG. For every V∈BV\in \mathfrak{B}, there exists a positive continuous central function fVf_V on GG with support contained in VV whose integral over GG is equal to 11. Let βV\beta_V denote the image measure j(fV⋅μ)j(f_V\cdot \mu); it is a positive measure with compact support on HH such that ϱ(βV)=σ(fV)\varrho (\beta_V) =\sigma (f_V) (INT, V, p. 71, § 6, No.o2^o2, Theorem 1). From the foregoing, the filter basis on Mc(H)\mathscr{M}_c(H) which is the image of B\mathfrak{B} under the mapping V↦βVV\mapsto \beta_V satisfies the conditions of Proposition 2 of V, p. 402, and the assertion follows.

Corollary 4 (Weyl's criterion for equidistribution)

Let MM be a set of positive central measures on GG such that ν(G)=1\nu (G) = 1 for every ν∈M\nu \in M. Let F\mathfrak{F} be a filter on MM. In order that the filter F\mathfrak{F} converge vaguely to the measure μ♯\mu^{\sharp} on G♯G^{\sharp}, it is necessary and sufficient that, for every non-trivial irreducible unitary representation π\pi, one have

limν,F∫Gχπ(x)dν(x)=0_{\nu ,\mathfrak{F}}\int_G\chi_{\pi}(x)d\nu (x) = 0.

Since ν(G♯)=1=μ♯(G♯)\nu (G^{\sharp}) = 1 =\mu^{\sharp}(G^{\sharp}) for ν∈M\nu \in M, the hypothesis means that

limν,F∫G♯χπ(x)dν(x)=∫G♯χπ(x)dμ♯(x)_{\nu ,\mathfrak{F}}\int_{G^{\sharp}}\chi_{\pi}(x)d\nu (x) =\int_{G^{\sharp}}\chi_{\pi}(x)d\mu^{\sharp}(x)

for every representation π∈G^\pi \in \widehat{G}, hence it is equivalent to the condition

limν,F∫G♯f(x)dν(x)=∫G♯f(x)dμ♯(x)_{\nu ,\mathfrak{F}}\int_{G^{\sharp}}f(x)d\nu (x) =\int_{G^{\sharp}}f(x)d\mu^{\sharp}(x)

for every function f∈Θ(G♯)f\in \Theta (G^{\sharp}) by linearity. Since the space Θ(G♯)\Theta (G^{\sharp}) is dense in C(G♯)\mathscr{C}(G^{\sharp}) (Corollary 1), the hypothesis is therefore equivalent to the convergence of the filter F\mathfrak{F} to μ♯\mu^{\sharp} in M(G♯)\mathscr{M}(G^{\sharp}) endowed with the topology of simple convergence in C\mathscr{C} (G), which coincides with the topology of vague convergence since G is compact (cf. INT, III, p. 59, § 1, No.o9^o9).

7. The Fourier cotransform

The set G^\widehat{G} is endowed with the discrete topology. We denote by F(G^)F(\widehat{G}) the product algebra of the End(Eπ_{\pi}) for π\pi belonging to G^\widehat{G} and by Fb(G^)F_b(\widehat{G}) the ∗*-product algebra of the End(Eπ_{\pi}) (Example 5 of I, p. 103); it is the set of families (uπ)π∈G^(u_{\pi})_{\pi\in\widehat{G}} such that supπ∈G^∥uπ∥<+∞_{\pi\in\widehat{G}}\|u_{\pi}\|<+\infty.

We denote by F0(G^)F_0(\widehat{G}) the closed ∗*-subalgebra of Fb(G^)F_b(\widehat{G}) formed by the families (uπ)π∈G^(u_{\pi})_{\pi\in\widehat{G}} such that ∥uπ∥\|u_{\pi}\| tends to 0 at infinity.

Let ν∈M1(G)\nu \in \mathscr{M}^1(G). For every π∈G^\pi \in \widehat{G}, we have ∥π(ν)∥⩽∥ν∥\|\pi (\nu )\|\leqslant \|\nu \|, hence the family (π(ν))π∈G^(\pi (\nu ))_{\pi\in\widehat{G}} belongs to Fb(G^)F_b(\widehat{G}).

Definition 1 03EO

For every measure ν∈M1(G)\nu \in \mathscr{M}^1(G), the element (π(ν))π∈G^(\pi (\nu ))_{\pi\in\widehat{G}} of Fb(G^)F_b(\widehat{G}) is denoted by FG(ν)\mathscr{F}_G(\nu ). The map of M1(G)\mathscr{M}^1(G) into Fb(G^)F_b(\widehat{G}) thus defined is called the Fourier cotransform of G, and F‾G(ν)\overline{\mathscr{F}}_G(\nu ) is called the Fourier cotransform of the measure ν\nu.

For f∈L1f\in L^1(G), we shall write F‾G(f)=F‾G(f⋅μ)\overline{\mathscr{F}}_G(f) =\overline{\mathscr{F}}_G(f\cdot \mu).

For every representation π∈G^\pi \in \widehat{G}, the map ν↦π(ν)\nu \mapsto \pi (\nu ) is a unital morphism of involutive Banach algebras of M1(G)\mathscr{M}^1(G) into End(Eπ_{\pi}) (Lemma 1 of V, p. 401). The Fourier cotransform is therefore a unital morphism of involutive Banach algebras of M1(G)\mathscr{M}^1(G) into Fb(G^)F_b(\widehat{G}).

8. The Fourier transform

We retain the notation of the preceding number.

Let π∈G^\pi \in \widehat{G}. We endow the vector space End(Eπ_{\pi}) with the structure of a Hilbert space whose scalar product is given by

⟨u1∣u2⟩=\langle u_1|u_2\rangle = dim(π\pi)Tr(u1∗u2u^*_1u_2) == dim(π\pi)Tr(u2u1∗u_2u^*_1)

for u1,u2u_1,u_2 in End(Eπ_{\pi}) ((cf. EVT, V, p. 52, Theorem 1).

We denote by ∥u∥2\|u\|_2 = √⟨u∣u⟩‾\surd\overline{\langle u|u\rangle} the norm of an element uu of End(Eπ_{\pi}) for π∈G^\pi \in \widehat{G}. For every g∈Gg\in G, one has ∥π(g)∥2=\|\pi (g)\|_2= dim(π\pi ) since π(g)\pi (g) is unitary.

The norm here denoted by ∥u∥2\|u\|_2 differs by a factor dim(π\pi ) from the norm defined in EVT, V, p. 52 on the space of Hilbert-Schmidt mappings of EπE_{\pi}.

Lemma 4 03EP

Let π\pi be an irreducible representation of G. The mapping f↦π(f)f\mapsto \pi (f) defines by passing to the subspaces an isometric isomorphism of ϱG(π‾)\boldsymbol{\varrho }_G(\overline{\pi}) into End(Eπ_{\pi}).

Put επ(g,h)u=π(g)∘u∘π(h−1)\varepsilon_{\pi}(g, h)u=\pi (g)\circ u\circ \pi (h^{-1}) for every (g,h)∈G×G(g, h)\in G\times G and for every u∈u\in End(Eπ_{\pi}). The mapping επ\varepsilon_{\pi} is a continuous representation of G×GG\times G in End(Eπ_{\pi}). It is unitary. In fact, since π\pi itself is unitary, one obtains

∥επ(g,h)u∥22=\|\varepsilon_{\pi}(g, h)u\|^2_2= dim(π\pi ) Tr((π(g)uπ(h−1))∗π(g)uπ(h−1))((\pi (g)u\pi (h^{-1}))^*\pi (g)u\pi (h^{-1}))

= dim(π\pi ) Tr(π(h)u∗uπ(h)−1\pi (h)u^*u\pi (h)^{-1}) == dim(π\pi ) Tr(u∗uu^*u) =∥u∥22=\|u\|^2_2 for every (g,h)∈G×G(g, h)\in G\times G and every u∈u\in End(Eπ_{\pi}).

The mapping Ψ defined by f↦π(f)f\mapsto \pi (f) is a (G×(G\times G)-morphism of ϱG(π‾)\boldsymbol{\varrho }_G(\overline{\pi}) into End(Eπ_{\pi}), since

π(ϱG(g,h)f)=∫Gf(g−1xh)π(x)dμ(x)=π(g)π(f)π(h−1) \pi (\boldsymbol{\varrho }_G(g, h)f) =\int_Gf(g^{-1}xh)\pi (x)d\mu(x) =\pi (g)\pi (f)\pi (h^{-1})

for every (g,h)∈G×G(g, h)\in G\times G and every f∈ϱG(π‾)f\in \boldsymbol{\varrho }_G(\overline{\pi}). Since ϱG(π‾)\boldsymbol{\varrho }_G(\overline{\pi}) is an irreducible representation (th. 1, a) of V, p. 462), there exists λ∈C∗\lambda \in \mathbf{C}^* such that the mapping λΨ\lambda \Psi is zero or isometric (cor. 5, a) of V, p. 388).

Let f=f= dim(π\pi )χ‾π∈ϱG(π‾)\overline{\chi}_{\pi}\in \boldsymbol{\varrho }_G(\overline{\pi}). Then π(f)=1Eπ\pi (f) = 1_{E_{\pi}} (cor. 2 of V, p. 465), whence ∥π(f)∥2=\|\pi (f)\|_2= dim(π\pi ) =∥f∥=\|f\| (cor. 3 of V, p. 459). Consequently, the mapping Ψ is isometric. Since ϱG(π‾)\boldsymbol{\varrho }_G(\overline{\pi}) and End(Eπ_{\pi}) are of the same dimension, Ψ is an isometric isomorphism.

We denote by F2(G^)F^2(\widehat{G}) the Hilbert sum of the Hilbert spaces End(Eπ_{\pi}). The norm of an element x∈F2(G^)x\in F^2(\widehat{G}) is again denoted by ∥x∥2\|x\|_2.

In LIE, IX, p. 79, this space is denoted L2(G^)L^2(\widehat{G}), a notation which we prefer to avoid here so as not to create confusion with the space ℓ2(G^)\ell^2(\widehat{G}).

Let (uπ)(u_{\pi}) be an element of F2(G^)F^2(\widehat{G}). Since

π∈G^∑π∈G^∥uπ∥22=∑π∈G^^{\pi\in\widehat{G}}\sum_{\pi\in\widehat{G}}\|u_{\pi}\|^2_2=\sum_{\pi\in\widehat{G}} dim(π\pi ) Tr(uπ∗uπu^*_{\pi}u_{\pi})<+∞<+\infty

and ∥uπ∥2⩽\|u_{\pi}\|^2\leqslant Tr(uπ∗uπu^*_{\pi}u_{\pi}) ((cf. EVT, V, p. 52, formula (33)), the norm in L(Eπ)\mathscr{L}(E_{\pi}) of the endomorphism uπu_{\pi} tends to 0 at infinity. One can therefore identify F2(G^)F^2(\widehat{G}) with a subspace of F0(G^)F_0(\widehat{G}).

Let π∈G^\pi \in \widehat{G} and u∈u\in End(Eπ_{\pi}). We denote by Fπ(u)\mathscr{F}_{\pi}(u) the function on G defined by Fπ(u)(g)=⟨π(g)∣u⟩=\mathscr{F}_{\pi}(u)(g) =\langle \pi (g)|u\rangle = dim(π\pi)Tr(π(g)∗u\pi (g)^*u) for every g∈Gg\in G. This is a continuous function on G. If G is a compact real Lie group, then the function Fπ(u)\mathscr{F}_{\pi}(u) is analytic on G (LIE, III, § 8, no1^o1, th. 1).

Since G is compact, one can identify L2(G)L^2(G) with a subspace of L1(G)L^1(G).

Theorem 2 03EQ

The Fourier cotransformation of G induces by passing to the subspaces an isometric isomorphism of L2(G)L^2(G) onto F2(G^)F^2(\widehat{G}). Its inverse FG\mathscr{F}_G associates with an element (uπ)π∈G^(u_{\pi})_{\pi\in\widehat{G}} of F2(G^)F^2(\widehat{G}) the sum of the series

∑π∈G^Fπ(uπ) \sum_{\pi\in\widehat{G}}\mathscr{F}_{\pi}(u_{\pi})

which converges in L2(G)L^2(G).

By the Peter-Weyl theorem (th. 1 of V, p. 462), the Hilbert space L2(G)L^2(G) is the Hilbert sum of the spaces ϱG(π‾)\boldsymbol{\varrho }_G(\overline{\pi}) for π∈G^\pi \in \widehat{G}. For every π∈G^\pi \in \widehat{G}, the linear mapping f↦π(f)f\mapsto \pi (f) of ϱG(π‾)\boldsymbol{\varrho }_G(\overline{\pi}) into End(Eπ_{\pi}) is an isometric isomorphism (lemma 4). Consequently, the restriction to L2(G)L^2(G) of the Fourier cotransformation defines an isometric isomorphism of L2(G)L^2(G) onto F2(G^)F^2(\widehat{G}).

Let f∈L2(G)f\in L^2(G). Let π∈G^\pi \in \widehat{G}, and let fπ‾∈C(G)f_{\overline{\pi}}\in \mathscr{C}(G) be the orthogonal projection of ff onto ϱG(π‾)\boldsymbol{\varrho }_G(\overline{\pi}) (th. 1 of V, p. 462). Since this space is the π‾\overline{\pi}-isotypical component of δG\boldsymbol{\delta }_G (cor. 4 of V, p. 463), one has fπ‾=f_{\overline{\pi}}= dim(π\pi )δG(χπ)(f)\boldsymbol{\delta }_G(\chi_{\pi})(f) by cor. 2 of V, p. 465. For every x∈Gx\in G, this gives

fπ‾(x)=f_{\overline{\pi}}(x) = dim(π\pi )∫Gχπ(g)(δG(g)f)(x)dμ(g)\int_G\chi_{\pi}(g)(\boldsymbol{\delta }_G(g)f)(x)d\mu(g)

= dim(π\pi )∫Gχπ(g)f(xg)dμ(g)\int_G\chi_{\pi}(g)f(xg)d\mu(g)

= dim(π\pi )∫Gχπ(x−1y)f(y)dμ(y)\int_G\chi_{\pi}(x^{-1}y)f(y)d\mu(y)

= dim(π\pi)Tr(π(x−1)π(f)\pi (x^{-1})\pi (f)) =⟨π(x)∣π(f)⟩=\langle \pi (x)|\pi (f)\rangle,

that is to say fπ‾=Fπ(π(f))f_{\overline{\pi}}=\mathscr{F}_{\pi}(\pi (f)).

Since ff is the sum in L2(G)L^2(G) of the family (fπ‾)(f_{\overline{\pi}}) by the Peter–Weyl theorem, we obtain

f=∑π∈G^Fπ(π(f)) f=\sum_{\pi\in\widehat{G}}\mathscr{F}_{\pi}(\pi (f))

where the series converges in L2(G)L^2(G). This proves the theorem.

Definition 2 03ER

The isometric isomorphism FG\mathscr{F}_G of F2(G^)F^2(\widehat{G}) onto L2(G)L^2(G) inverse to the isomorphism induced by the Fourier cotransform is called the Fourier transform of G. The image of an element of F2(G^)F^2(\widehat{G}) is called its Fourier transform.

Remark 1 03ES

The Fourier transform of (uπ)π∈G^∈F2(G^)(u_{\pi})_{\pi\in\widehat{G}}\in F^2(\widehat{G}) is therefore the class in L2(G)L^2(G) of the series

g↦∑π∈G^⟨π(g)∣uπ⟩=∑π∈G^g\mapsto \sum_{\pi\in\widehat{G}}\langle \pi (g)|u_{\pi}\rangle =\sum_{\pi\in\widehat{G}} dim(π\pi)Tr(uπ∘π(g)−1u_{\pi}\circ \pi (g)^{-1}).

Remark 2 03ET

Let f∈L2(G)f\in L^2(G). One then has the Plancherel formula

∥f∥2=∥F‾G(f)∥2=∑π∈G^∥π(f)∥2 \|f\|^2=\|\overline{\mathscr{F}}_G(f)\|^2=\sum_{\pi\in\widehat{G}}\|\pi (f)\|^2

Moreover, by Theorem 2,f,f is the sum in L2(G)L^2(G) of the family (fπ)π∈G^(f_{\pi})_{\pi\in\widehat{G}} where

fπ(x)=Fπ(π(f))(x) f_{\pi}(x) =\mathscr{F}_{\pi}(\pi (f))(x)

=⟨π(x)∣π(f)⟩==\langle \pi (x)|\pi (f)\rangle = dim(π\pi )∫Gf(g)χπ(x−1g)dμ(g)\int_Gf(g)\chi_{\pi}(x^{-1}g)d\mu(g)

for every x∈Gx\in G and every π∈G^\pi \in \widehat{G}.

Suppose that G is commutative. Since the irreducible representations of G are of dimension 1 (cor. 7 of V, p. 390) and the dual group G^\widehat{G} is discrete (prop. 18 of II, p. 233), the algebras Fb(G^)F_b(\widehat{G}) and F0(G^)F_0(\widehat{G}) are identified, respectively, with the algebra Cb(G^)\mathscr{C}_b(\widehat{G}) of bounded continuous functions on G^\widehat{G} and the algebra C0(G^)\mathscr{C}_0(\widehat{G}) of continuous functions tending to 0 at infinity on G^\widehat{G}. Since G is compact, the Haar measure on G^\widehat{G} dual to μ\mu is the counting measure μ^\widehat{\mu} (prop. 18 of II, p. 233). The Hilbert sum F2(G^)F^2(\widehat{G}) of the spaces End(Eπ_{\pi}) for π∈G^\pi \in \widehat{G} is identified with the Hilbert space L2(G^,μ^)L^2(\widehat{G},\widehat{\mu}).

Let ν∈M1(G)\nu \in \mathscr{M}^1(G). Then, for every unitary character χ∈G^\chi \in \widehat{G}, one has

χ(ν)=∫Gχν \chi (\nu ) =\int_G\chi \nu

which proves that the Fourier cotransform defined above coincides with the Fourier cotransform of G defined in II, p. 206.

Every f∈L2(G)f\in \mathscr{L}^2(G) is integrable and the formula ∥F‾G(f)∥2=∥f∥\|\overline{\mathscr{F}}_G(f)\|_2=\|f\| is the Plancherel formula (II, p. 215, Theorem 1).

Proposition 10 03EU

The Fourier cotransform is an injective morphism of involutive Banach algebras of M1(G)\mathscr{M}^1(G) into Fb(G^)F_b(\widehat{G}) which maps L1(G)L^1(G) into F0(G^)F_0(\widehat{G}).

Since G is compact, the space C(G)\mathscr{C}(G) is included in and dense in L1(G)\mathscr{L}^1(G) and L2(G)\mathscr{L}^2(G).

Let ν∈M1(G)\nu \in \mathscr{M}^1(G) be such that F‾G(ν)=0\overline{\mathscr{F}}_G(\nu ) = 0. Then for every continuous function ff on G, one has F‾G(ν∗f)=0\overline{\mathscr{F}}_G(\nu *f) = 0. Now ν∗f\nu *f belongs to C(G)\mathscr{C}(G) (INT, VIII, p. 152, § 4, No.o2^o2, prop. 3). Since, by Theorem 2, the Fourier cotransform is injective on L2L^2(G), and a fortiori on the space C\mathscr{C} (G), one has ν∗f=0\nu *f= 0 for f∈C(G)f\in \mathscr{C}(G). In particular, it follows that

∫Gf(x)dν(x)=(ν∗fˇ)(e)=0 \int_Gf(x)d\nu (x) = (\nu *\check{f})(e) = 0

for every function f∈C(G)f\in \mathscr{C}(G) (INT, VIII, loc. cit.), hence the measure ν\nu is zero.

The image of C(G)\mathscr{C}(G) under the Fourier cotransform is contained in F2(G^)F^2(\widehat{G}), and a fortiori in F0(G^)F_0(\widehat{G}). Since F0(G^)F_0(\widehat{G}) is closed in Fb(G^)F_b(\widehat{G}) and C(G)\mathscr{C}(G) is dense in L1L^1(G), the image of L1(G)L^1(G) under the Fourier cotransform is likewise contained in F0(G^)F_0(\widehat{G}).

Proposition 11 03EV

a) Let u=(uπ)π∈G^u= (u_{\pi})_{\pi\in\widehat{G}} be an element of F(G^)F(\widehat{G}). If the family (Fπ(uπ))π∈G^(\mathscr{F}_{\pi}(u_{\pi}))_{\pi\in\widehat{G}} is uniformly summable in C(G)\mathscr{C}(G), then its sum ff is a continuous function on G whose Fourier cotransform is uu;

b) Let f∈L1(G)f\in L^1(G). If the family (Fπ(π(f)))π∈G(\mathscr{F}_{\pi}(\pi (f)))_{\pi\in G} is uniformly summable in C(G)\mathscr{C}(G), then its sum is a continuous func^\widehat{c}tion on G whose class in L1(G)L^1(G) is equal to ff.

Let us prove assertion a). Since G is compact, the sum ff of the family (Fπ(uπ))(\mathscr{F}_{\pi}(u_{\pi})) belongs to L2(G)L^2(G) and the series

∑π∈G^Fπ(uπ) \sum_{\pi\in\widehat{G}}\mathscr{F}_{\pi}(u_{\pi})

converges in L2(G)L^2(G) to ff (INT, IV, p. 127, § 3, No.o3^o3, prop. 4). Hence one has FG((uπ)π)=f\mathscr{F}_G((u_{\pi})_{\pi}) =f in L2L^2(G), whence (uπ)π∈G^=F‾G(f)(u_{\pi})_{\pi\in\widehat{G}}=\overline{\mathscr{F}}_G(f) (Theorem 2).

Let us prove assertion b). Assertion a) may be applied to the family (π(f))π∈G(\pi (f))_{\pi\in G}. The sum gg of the family (Fπ(π(f)))π∈G(\mathscr{F}_{\pi}(\pi (f)))_{\pi\in G} is a continuous function, hence belonging to L2L^2(G), such that FG(g)=(π(f))π∈G\mathscr{F}_G(g) = (\pi (f))_{\pi\in G}. Since FG\mathscr{F}_G is injective on L2L^2(G), one has f=gf=g in L2L^2(G), hence in L1(G)L^1(G).

The algebra C(G^)\mathscr{C}(\widehat{G}) of complex-valued functions on G^\widehat{G} is identified with the center of the algebra F(G^)F(\widehat{G}) by the mapping which to f:G^→Cf:\widehat{G}\rightarrow \mathbf{C} associates the central element (f(π)1Eπ)π∈G^(f(\pi )1_{E_{\pi}})_{\pi\in\widehat{G}} of F(G^)F(\widehat{G}). Let us denote by M1(G♯)\mathscr{M}^1(G^{\sharp}) the space of bounded central measures on G. It is a closed subspace of the Banach algebra M1(G)\mathscr{M}^1(G). For every measure ν∈M1(G♯)\nu \in \mathscr{M}^1(G^{\sharp}) and every representation π∈G^\pi \in \widehat{G}, one has π(ν)∈\pi (\nu )\in EndG(Eπ)_G(E_{\pi}) hence π(ν)\pi (\nu ) is a multiple of the identity mapping of EπE_{\pi} by Schur’s Lemma (Prop. 6 of V, p. 386). The restriction of the Fourier cotransform to M1(G♯)\mathscr{M}^1(G^{\sharp}) is therefore identified with a unital morphism of involutive Banach algebras from M1(G♯)\mathscr{M}^1(G^{\sharp}) into C(G^)\mathscr{C}(\widehat{G}). This morphism is injective; the image of L1(G♯)L^1(G^{\sharp}) is contained in Cb(G^)\mathscr{C}_b(\widehat{G}) and its restriction to L2(G♯)L^2(G^{\sharp}) is an isometric isomorphism onto L2(G^,μ^)L^2(\widehat{G},\widehat{\mu}).

9. Frobenius-Schur Indicator and Larsen’s Alternative

Recall (LIE, VIII, §7, No. o6^o6, Def. 2) that an irreducible representation of G in a complex vector space E of finite dimension is said to be of orthogonal type (resp. of symplectic type) if there exists a non-zero symmetric bilinear form on E invariant under G (resp. if there exists a non-zero alternating bilinear form on E invariant under G). It is said to be of complex type if there exists no non-zero bilinear form on E invariant under G.

When an irreducible representation in E is of orthogonal type (resp. symplectic), the space of symmetric (resp. alternating) G-invariant bilinear forms on E is of dimension 1 and every non-zero G-invariant bilinear form on E is non-degenerate.

A representation of orthogonal type is sometimes said to be of real type, and a representation of symplectic type is sometimes said to be of quaternionic type (cf. LIE, IX, App. II).

Let π\pi be an irreducible representation of G in a complex vector space E. The three possibilities above are distinguished by the value of the quantity

FS(π\pi ) =∫Gχπ(g2)dμ(g)=\int_G\chi_{\pi}(g^2)d\mu(g),

which is called the Frobenius-Schur indicator of π\pi. One has in fact

1 if π\pi is of orthogonal type,

FS(π\pi ) =−1=-1 if π\pi is of symplectic type,

0 if π\pi is of complex type.

(LIE, IX, App. 2, p. 103, Prop. 3 and p. 105, Prop. 4).

When G is a Lie group, one can calculate FS(π\pi ) with the aid of Prop. 1 of LIE, IX, p. 69.

Definition 3 03EW

Let ϱ\varrho be a finite-dimensional unitary representation of G in a Hilbert space E. Let kk be a positive integer. The absolute moment of order 2k2k of ϱ\varrho is defined to be, and is denoted by M2k(ϱ)M_{2k}(\varrho ), the dimension of the subspace of G-invariant elements in the representation ϱ‾⊗k⊗ϱ⊗k\overline{\varrho}^{\otimes k}\otimes \varrho^{\otimes k}.

Theorem 3 (Larsen’s Alternative) 03EX

Assume that G is infinite. Let π\pi be a faithful unitary representation of G in a Hilbert space E of finite dimension ⩾2\geqslant 2.

a) Assume that the derived group of G is infinite. Then M4(π)⩾2M_4(\pi )\geqslant 2 with equality if and only if G contains SU(E)\mathbf{S}\mathbf{U}(E);

b) Assume that dim(E) ⩾3\geqslant 3, that π\pi is of orthogonal or symplectic type, and that dim(E) ≠4\not = 4 if π\pi is of orthogonal type. Then M4(π)⩾3M_4(\pi )\geqslant 3 with equality if and only if the complexified Lie algebra of G is equal to the Lie algebra of a non-degenerate G-invariant bilinear form on E. This is the case if and only if the identity component G0G_0 of G is a maximal compact subgroup of the automorphism group of such a bilinear form.

We shall use the following lemmas in the proof.

Lemma 5 03EY

Let π\pi be a unitary representation of G in a Hilbert space E of finite dimension. Let (ϱi)i∈I(\varrho_i)_{i\in I} be a family of non-zero unitary representations of G and (ni)i∈I(n_i)_{i\in I} a family of integers ⩾1\geqslant 1 such that the representation π‾⊗π\overline{\pi}\otimes \pi of G (resp. the representation π⊗π\pi \otimes \pi ) is isomorphic to the direct sum ⨁i∈Iϱini\bigoplus_{i\in I}\varrho^{n_i}_i. Then one has

M4(π)⩾∑i∈Ini2 M_4(\pi )\geqslant \sum_{i\in I}n^2_i

with equality if and only if the representations ϱi\varrho_i are irreducible and pairwise non-isomorphic.

Let us denote by χi\chi_i the character of ϱi\varrho_i for i∈Ii\in I. One has

∣χπ∣2=χπ‾⊗π=∑i∈Iniχi|\chi_{\pi}|^2=\chi_{\overline{\pi}\otimes\pi}=\sum_{i\in I}n_i\chi_i, (resp. χπ2=χπ⊗π=∑i∈Iniχi\chi^2_{\pi}=\chi_{\pi\otimes\pi}=\sum_{i\in I}n_i\chi_i).

From the definition and Cor. 3 of V, p. 466, there follows

M4(π)=∫G∣χπ∣4dμ M_4(\pi ) =\int_G|\chi_{\pi}|^4d\mu

whence, in both cases, the formula

M4(π)=∑i,jninj∫Gχiχ‾jdμ=∑i,jninjM_4(\pi ) =\sum_{i,j}n_in_j\int_G\chi_i\overline{\chi}_jd\mu=\sum_{i,j}n_in_j dim HomG(ϱi,ϱj)_G(\varrho_i, \varrho_j)

(Cor. 4, b) of V, p. 459). Let i∈Ii\in I. Since the representation ϱi\varrho_i is not zero, the space HomG(ϱi,ϱi)_G(\varrho_i, \varrho_i) is non-zero, and one deduces the lower bound

M4(π)⩾∑i∈Ini2 M_4(\pi )\geqslant \sum_{i\in I}n^2_i

Moreover, since ni⩾n_i\geqslant 1, there is equality if and only if one has dim HomG(ϱi,ϱj)=0_G(\varrho_i, \varrho_j) = 0 if i≠ji\not =j and dim HomG(ϱi,ϱi)=1_G(\varrho_i, \varrho_i) = 1 for all ii. The second condition holds if and only if the representations ϱi\varrho_i are irreducible (loc. cit.). The first is then satisfied if and only if the representations ϱi\varrho_i are pairwise non-isomorphic, by Schur’s Lemma (V, p. 387, Cor. 2).

If E is a module over a commutative ring A, the a-module S2(E)\mathsf{S}^2(E) (resp. ∧2E\wedge^2E) will be called the symmetric square (resp. exterior square) of E.

Lemma 6 03EZ

Let qq be a non-degenerate bilinear form on a finite-dimensional complex vector space E of dimension ⩾3\geqslant 3.

a) If qq is symmetric, then the adjoint representation of the orthogonal Lie algebra so(q)\mathfrak{s}\mathfrak{o}(q) is isomorphic to the exterior square ∧2E\wedge^2E of the natural representation so(q)→gl(E)\mathfrak{s}\mathfrak{o}(q)\rightarrow \mathfrak{g}\mathfrak{l}(E);

b) If qq is alternating, then the adjoint representation of the symplectic Lie algebra sp(q)\mathfrak{s}\mathfrak{p}(q) is isomorphic to the symmetric square S2(E)\mathsf{S}^2(E) of the natural representation sp(q)→gl(E)\mathfrak{s}\mathfrak{p}(q)\rightarrow \mathfrak{g}\mathfrak{l}(E).

Let C\mathbf{C} be provided with the identical involutive automorphism. The bilinear form qq is then ε\varepsilon-Hermitian (A, IX, § 3, no1^o1, Def. 1) with ε=1\varepsilon = 1 in case a) and ε=−1\varepsilon =-1 in case b). For every u∈u\in End(E), let tu^tu denote the adjoint of uu with respect to qq. Thus one has q(x,u(y))=q(tu(x),y)q(x, u(y)) =q(^tu(x), y) for every (x,y)∈E×E(x, y)\in E\times E. Let vv denote the unique automorphism of E⊗EE\otimes E such that v(x⊗y)=y⊗xv(x\otimes y) =y\otimes x for every (x,y)∈E2(x, y)\in E^2.

There exists a unique linear mapping ww of E⊗EE\otimes E into End(E) such that w(a⊗b)w(a\otimes b) is the linear mapping defined by x↦q(a,x)bx\mapsto q(a, x)b for every (a,b,x)∈E3(a, b, x)\in E^3. The mapping ww is an isomorphism. For every s∈E⊗Es\in E\otimes E, one has

t(w(s))=εw(v(s))(3) t(w(s)) =\varepsilon w(v(s)) \tag{3}

In fact, for all (x,y)(x, y) and (a,b)(a, b) in E×EE\times E, there follows

q(x,w(a⊗b)y)=q(a,y)q(x,b)=εq(b,x)q(a,y)=q(εw(b⊗a)x,y) q(x, w(a\otimes b)y) =q(a, y)q(x, b) =\varepsilon q(b, x)q(a, y) =q(\varepsilon w(b\otimes a)x, y)

Finally, denote by x↦x∗x\mapsto x^* the isomorphism of E onto E′E' deduced from qq, that is to say such that ⟨x∗,y⟩=q(x,y)\langle x^*, y\rangle =q(x, y) for every (x,y)∈E2(x, y)\in E^2.

With these notations established, let H be the orthogonal group (resp. symplectic) of qq. It is a Lie subgroup of GL(E)\mathbf{G}\mathbf{L}(E) whose Lie algebra h\mathfrak{h} is the subspace of End(E) consisting of the u∈u\in End(E) such that tu=−u^tu=-u (LIE, III, p. 146, cor. 1). Since the form qq is H-invariant, we have

⟨(ga)∗,b⟩=q(ga,b)=⟨a,g−1b⟩ \langle (ga)^*, b\rangle =q(ga, b) =\langle a, g^{-1}b\rangle

for every g∈Hg\in H and every (a,b)∈E×E(a, b)\in E\times E.

Equip End(E) with the adjoint representation Ad of H. The linear mapping ww is a morphism of representations of H: in fact, for every g∈Hg\in H and every (a,b,x)∈E3(a, b, x)\in E^3, we have

w(g(a⊗b))(x)=⟨(ga)∗,x⟩gb=⟨a,g−1x⟩gb w(g(a\otimes b))(x) =\langle (ga)^*, x\rangle gb=\langle a, g^{-1}x\rangle gb

=g(⟨a,g−1x⟩b)==g(\langle a, g^{-1}x\rangle b) = Ad(gg)(w(a⊗b))x(w(a\otimes b))x,

whence the conclusion by linearity.

Since vv is likewise a morphism of representations of H, the same is true of the linear mapping θ=w−εw∘v\theta =w-\varepsilon w\circ v; the latter is therefore a morphism of h\mathfrak{h}-modules.

Let F ⊂E⊗E\subset E\otimes E be the subspace of elements s∈E⊗Es\in E\otimes E such that v(s)=−εsv(s) =-\varepsilon s. If ε=−1\varepsilon =-1, the restriction to F of the canonical mapping of E⊗EE\otimes E into the symmetric square of E is an isomorphism of C\mathbf{C}-vector spaces (A, III, p. 72); if ε=1\varepsilon = 1, the restriction to F of the canonical mapping of E⊗EE\otimes E into the exterior square of E is an isomorphism of C\mathbf{C}-vector spaces (loc. cit., p. 82).

The image of F under θ\theta is contained in h:\mathfrak{h}: in fact, for every s∈Fs\in F, formula (3) implies

tθ(s)=tw(s)−εtw(v(s))=εw(v(s))−w(s)=−θ(s) ^t\theta (s) =^tw(s)-\varepsilon^tw(v(s)) =\varepsilon w(v(s))-w(s) =-\theta (s)

By definition of F, the restriction of the mapping θ\theta to F coincides with that of 2w2w and the linear mapping θ\theta is therefore injective on F. Since dim(F) = dim(h\mathfrak{h}) (A, III, p. 75, th. 1 and p. 87, cor. 1 and LIE, VIII, p. 192 and p. 201), one concludes that θ\theta induces by passing to the subspaces an isomorphism of h\mathfrak{h}-modules of F onto h\mathfrak{h}, whence the lemma.

Lemma 7 03F0

Let H2H_2 be a real Lie group and let H1H_1 be a closed subgroup of H2H_2. The complexified Lie algebra of H1H_1 identifies with a subrepresentation of the adjoint representation of H1H_1 on the complexified Lie algebra of H2H_2.

In fact, the restriction to H1H_1 of the adjoint representation of H2H_2 on its Lie algebra identifies with the adjoint representation of H1H_1.

Lemma 8 03F1

Let E be a finite-dimensional complex Hilbert space of dimension nn.

a) The groups SU(E)\mathbf{S}\mathbf{U}(E) and U(E)\mathbf{U}(E) are connected;

b) The derived group of SU(E)\mathbf{S}\mathbf{U}(E) is equal to SU(E)\mathbf{S}\mathbf{U}(E);

c) The derived group of U(E)\mathbf{U}(E) is equal to SU(E)\mathbf{S}\mathbf{U}(E).

We may suppose that n⩾1n\geqslant 1 and that E=CnE =\mathbf{C}^n.

Let A be the subgroup of U(E)\mathbf{U}(E) formed by the diagonal matrices; it is homeomorphic to Un\mathbf{U}^n, and therefore connected (TG, VI, p. 11, cor. 2 and I, p. 83, prop. 8). Its intersection with SU(E)\mathbf{S}\mathbf{U}(E) is homeomorphic to Un−1\mathbf{U}^{n-1}, and is therefore likewise connected.

By Theorem 1 of IV, p. 149, the group U(E)\mathbf{U}(E) (resp. SU(E)\mathbf{S}\mathbf{U}(E)) is the union of the connected subsets gAg−1gAg^{-1} for g∈Gg\in G (resp. of the connected subsets g(A∩SU(E))g−1g(A\cap \mathbf{S}\mathbf{U}(E))g^{-1}); since the identity element belongs to each of these sets, the space U(E)\mathbf{U}(E) (resp. SU(E)\mathbf{S}\mathbf{U}(E)) is connected (TG, I, p. 81, prop. 2).

Let us prove b). The assertion is true when n=1n= 1, since SU(E)\mathbf{S}\mathbf{U}(E) is then reduced to the identity element. Suppose therefore that n⩾2n\geqslant 2. The Lie algebra of SU(E)\mathbf{S}\mathbf{U}(E) is then simple (LIE, IX, p. 20, § 3, No.o4^o4) and hence the derived group of SU(E)\mathbf{S}\mathbf{U}(E) has finite index in SU(E)\mathbf{S}\mathbf{U}(E). The result follows, since SU(E)\mathbf{S}\mathbf{U}(E) is connected by a).

Assertion c) follows from b), since the derived group of U(E)\mathbf{U}(E) is contained in SU(E)\mathbf{S}\mathbf{U}(E).

Let us now prove Theorem 3. Let nn denote the dimension of the Hilbert space E. Since the representation π\pi is faithful, we may suppose that G is a compact subgroup of U(E)\mathbf{U}(E). The group G is closed in the real Lie group U\mathbf{U}(E), and is therefore a compact real Lie group (LIE, III, §8, No.o2^o2, Theorem 2). By hypothesis, G is infinite, and therefore of dimension ⩾1\geqslant 1. Let g\mathfrak{g} be its Lie algebra; it is non-zero.

Let us prove a). Suppose that the derived group D(G) is infinite. We have the G-invariant decomposition End(E) =C1E⊕sl(E)=\mathbf{C}1_E\oplus \mathfrak{s}\mathfrak{l}(E). The representations of G on C1E\mathbf{C}1_E and on sl(E)\mathfrak{s}\mathfrak{l}(E) are not isomorphic, since dim sl(E)⩾2\mathfrak{s}\mathfrak{l}(E)\geqslant 2. By Lemma 5, we therefore have M4(π)⩾2M_4(\pi )\geqslant 2 with equality if and only if the representation of G in sl(E)\mathfrak{s}\mathfrak{l}(E) is irreducible. Now, the derived group of G is contained in SL\mathbf{S}\mathbf{L}(E), and the complexification of the Lie algebra of D(G) may therefore be identified with a subspace of sl\mathfrak{s}\mathfrak{l}(E), which is a subrepresentation of the representation of G in sl(E)\mathfrak{s}\mathfrak{l}(E) (Lemma 7). This subrepresentation is non-zero, since D(G) is infinite. We therefore have M4(π)=2M_4(\pi ) = 2 if and only if (Dg)(C)=sl(E)(\mathscr{D}\mathfrak{g})_{(\mathbf{C})}=\mathfrak{s}\mathfrak{l}(E). Since D(G) is contained in SU\mathbf{S}\mathbf{U}(E), and since SU(E)=D(SU(E))\mathbf{S}\mathbf{U}(E) = D(\mathbf{S}\mathbf{U}(E)) (Lemma 8), this condition is equivalent to SU(E)⊂G\mathbf{S}\mathbf{U}(E)\subset G.

For the proof of assertion b), we resume the notation of LIE, VIII, §13, No.o^o 2–4.

Suppose that π\pi is of symplectic type and that dim(E) ⩾3\geqslant 3. Let qq be a non-zero G-invariant alternating bilinear form on E; it is nondegenerate (LIE, IX, p. 103, App. 2, prop. 3) and G is a compact subgroup of the symplectic group Sp(q)\mathbf{S}\mathbf{p}(q). Let us denote by q∗∈∧2Eq^*\in \wedge^2E the element with which the alternating form qq is identified. The complexified Lie algebra g(C)\mathfrak{g}_{(\mathbf{C})} of G is contained in sp(q)\mathfrak{s}\mathfrak{p}(q). Now, since dim(E) ⩾3\geqslant 3, the representation of sp(q)\mathfrak{s}\mathfrak{p}(q) in E⊗EE\otimes E admits the direct-sum decomposition

E⊗E=S2(E)⊕∧2E=S2(E)⊕E2⊕Cq∗ E\otimes E =\mathsf{S}^2(E)\oplus \wedge^2E =\mathsf{S}^2(E)\oplus E_2\oplus \mathbf{C}q^*

where E2E_2 is the second fundamental representation of sp(q)\mathfrak{s}\mathfrak{p}(q) (LIE, VIII, p. 202, §13, No.o3^o3, (IV)). The representations E2E_2 and Cq∗\mathbf{C}q^* of sp(q)\mathfrak{s}\mathfrak{p}(q) are irreducible (loc. cit.) and the representation S2(E)\mathsf{S}^2(E) is non-zero.

After lemma 5, one therefore has M4(π)⩾3M_4(\pi )\geqslant 3 with equality if and only if the representations of G in S2\mathsf{S}^2(E), E2E_2 and Cq∗\mathbf{C}q^* are irreducible and pairwise non-isomorphic.

Suppose that M4(π)=3M_4(\pi ) = 3. Since the representation S2(E)\mathsf{S}^2(E) is identified with the adjoint representation of sp(q)\mathfrak{s}\mathfrak{p}(q) (lemma 6), it contains as subrepresentation the complexification of the adjoint representation of G (lemma 7). One therefore has g(C)=sp(q)\mathfrak{g}_{(\mathbf{C})}=\mathfrak{s}\mathfrak{p}(q).

Conversely, suppose that g(C)=sp(q)\mathfrak{g}_{(\mathbf{C})}=\mathfrak{s}\mathfrak{p}(q). The adjoint representation of sp(q)\mathfrak{s}\mathfrak{p}(q) and the representation E2E_2 of sp(q)\mathfrak{s}\mathfrak{p}(q) are irreducible, of dimensions n(n+1)/2n(n+ 1)/2 and n(n−1)/2−1n(n-1)/2-1 respectively (LIE, VIII, p. 202, §13, No.o3^o3, (IV)), which are different and ⩾2\geqslant 2 since n⩾3n\geqslant 3, whence M4(π)=3M_4(\pi ) = 3.

Finally, suppose that π\pi is of real type and that dim(E) ⩾3\geqslant 3 and dim(E) ≠4\not = 4. Let qq be a non-zero symmetric bilinear form, G-invariant on E; it is separating (LIE, IX, p. 103, App. 2, prop. 3) and G is a compact subgroup of the orthogonal group O(q)\mathbf{O}(q). Let us denote by q∗∈S2(E)q^*\in \mathsf{S}^2(E) the element with which qq is identified.

The complexified Lie algebra g(C)\mathfrak{g}_{(\mathbf{C})} of G is contained in so(q)\mathfrak{s}\mathfrak{o}(q). The representation of so(q)\mathfrak{s}\mathfrak{o}(q) in E⊗EE\otimes E admits the decomposition into a direct sum

E⊗E=∧2E⊕S2(E)=∧2E⊕S02(E)⊕Cq∗ E\otimes E =\wedge^2E\oplus \mathsf{S}^2(E) =\wedge^2E\oplus \mathsf{S}^2_0(E)\oplus \mathbf{C}q^*

where S02(E)\mathsf{S}^2_0(E) is the orthogonal of q∗q^* in S2(E)\mathsf{S}^2(E). These representations are of dimension at least 2 since dim(E) ⩾3\geqslant 3. After lemma 5, one has M4(π)⩾3M_4(\pi )\geqslant 3 with equality if and only if the representations of G in ∧2E\wedge^2E and S02(E)\mathsf{S}^2_0(E) are irreducible and non-isomorphic.

Suppose that M4(π)=3M_4(\pi ) = 3. Since the representation ∧2E\wedge^2E is identified with the adjoint representation of so(q)\mathfrak{s}\mathfrak{o}(q) (lemma 6), it contains as subrepresentation the complexification of the adjoint representation of G (lemma 7). The condition M4(π)=3M_4(\pi ) = 3 therefore implies that g(C)=so(q)\mathfrak{g}_{(\mathbf{C})}=\mathfrak{s}\mathfrak{o}(q).

Conversely, suppose that g(C)=so(q)\mathfrak{g}_{(\mathbf{C})}=\mathfrak{s}\mathfrak{o}(q). The adjoint representation of so(q)\mathfrak{s}\mathfrak{o}(q) is irreducible, since dim(E) ≠4\not = 4 (LIE, VIII, § 13, p. 193, (I) and p. 206, (I)), and of dimension n(n−1)/2n(n-1)/2. The representation S02(E)\mathsf{S}^2_0(E) of so(q)\mathfrak{s}\mathfrak{o}(q) has highest weight 2ϖ12\varpi_1. By comparing its dimension with that of the irreducible representation of so(q)\mathfrak{s}\mathfrak{o}(q) of highest weight 2ϖ12\varpi_1, calculated by Weyl's formula (cf. LIE, VIII, §9, no2^o2, th. 2 and LIE, VI, plates II and IV), one verifies that S02(E)\mathsf{S}^2_0(E) is irreducible. It follows therefore that M4(π)=3M_4(\pi ) = 3 if g(C)=so(q)\mathfrak{g}_{(\mathbf{C})}=\mathfrak{s}\mathfrak{o}(q).

Remark 1 03F2

The condition "G is infinite" is necessary in Theorem 3 (exercise 9 of V, p. 507).

Remark 2 03F3

Let n∈Nn\in \mathbf{N}. For every unitary representation ϱ\varrho of a compact group in a Hilbert space of dimension nn, one has M4(ϱ)⩽n2M_4(\varrho )\leqslant n^2; equality is possible (exercise 15 of V, p. 508).

Remark 3 03F4

One can prove (cf. R. Guralnick and P.H. Tiep, Decomposition of small tensor powers and Larsen’s conjecture, Representation Theory 9 (2005), 138–208) that the condition that G be infinite can be omitted if one assumes that E is of dimension ⩾7\geqslant 7 and if one replaces the hypothesis M4(π)=2M_4(\pi ) = 2 (resp. M4(π)=3M_4(\pi ) = 3) by M8(π)=24M_8(\pi ) = 24 (resp. M8(π)=105M_8(\pi ) = 105).

Exercises

See the exercises for § 4.

Machine translation of the English, by gpt-5.4, against glossary version 34. Not checked by a person.