The mapping $\varpi$ is a linear representation of $G\times H$ in E; let us verify that it is continuous. Since $\|\varpi (g, h)\|\leqslant \|\varrho (g)\|\|\pi (h)\|$ for every $(g, h)\in G\times H$, the representation $\varpi$ is bounded, and therefore its image is equicontinuous in $\mathscr{L}(E)$. It then suffices to prove that, for every $x\in E$, the mapping $(g, h)\mapsto \varpi (g, h)x$ is continuous (Remark 2 of INT, VIII, p. 129, § 2, n$^o1$). Let $(g_0, h_0)\in G\times H$. Put $y=\pi (h_0)x$. For every $(g, h)\in G\times H$, one has

Since $\varrho$ is bounded and since $\varrho$ and $\pi$ are continuous, this implies the assertion.

In particular, if $\pi$ and $\varrho$ are irreducible and finite-dimensional, then every nonzero G-morphism from $\pi$ into $\varrho$ is an isomorphism.

Since the space of $\varrho$ is separated, the kernel of a G-morphism $u$ from $\pi$ into $\varrho$ is closed, and gives rise to a subrepresentation of $\pi$. If the morphism $u$ is nonzero, its kernel must therefore be reduced to 0, since $\pi$ is an irreducible representation in a separated topological vector space. Similarly, the closure of the image of a nonzero G-morphism from $\varrho$ into $\pi$ is a nonzero subrepresentation of $\pi$, and is therefore equal to the space of $\pi$.

The last assertion follows from the preceding.

Since E is finite-dimensional and nonzero, there exists a G-invariant subspace F of E which is nonzero and of minimal dimension. This subspace is closed in E and defines a subrepresentation of E; every subrepresentation of F is also a subrepresentation of E, and the representation F is therefore irreducible by minimality.

The condition is obviously necessary. It is sufficient by Remark 2 of INT, VIII, p. 129, § 2, n$^o1$, since the image of $\varrho$ is equicontinuous in $\mathscr{L}(E)$ and since the continuity of the mapping $g\mapsto \varrho (g)x$ at $e$ implies its continuity on G.

Let G be a topological group. If $\varrho_1$ and $\varrho_2$ are unitary representations of G and if $u$ belong to Hom$_G(\varrho_1, \varrho_2)$, then $u^*$ belong to Hom$_G(\varrho_2, \varrho_1)$. In fact, since $\varrho_1$ and $\varrho_2$ are unitary representations, one has for every $g\in G$

What precedes shows that Hom$_G(\varrho , \varrho )$ is a unital self-adjoint subalgebra of $\mathscr{L}(E)$. Since it is closed in $\mathscr{L}$ (E), it is an involutive subalgebra of $\mathscr{L}(E)$.

Let $g\in G$ and $x\in$ dom($u^*$). For every $y\in$ dom($u$), one has

since $\varrho (g)^*=\varrho (g)^{-1}=\varrho (g^{-1})$. This proves that $\varrho (g)x\in$ dom($u^*$) and that $u^*(\varrho (g)x) =\pi (g)(u^*(x))$. In particular, the domain of $u^*\circ \varrho (g)$ contains the domain of $u^*$, and $\pi (g)\circ u^*\subset u^*\circ \varrho (g)$.

But moreover, if $x\in$ dom($u^*\circ \varrho (g)$), then $x=\varrho (g^{-1})(\varrho (g)x)$ belongs to dom($u^*$) by what precedes applied to $g^{-1}$. It follows that $u^*\circ \varrho (g) =\pi (g)\circ u^*$.

If $u$ is a G-morphism, then one has

for all $g\in G$ and all $(x, y)\in E_2\times E_1$, hence the indicated mapping is a linear mapping from Hom$_G(\varrho_1, \varrho_2)$ into Sesq$_G(\varrho_2, \varrho_1)$. By EVT, V, p. 16, Corollary 2, it is injective.

Conversely, let $q$ be a G-invariant sesquilinear form and $u$ the unique linear mapping from $E_1$ into $E_2$ such that $q(x, y) =\langle x|u(y)\rangle$ for all $(x, y)\in E_2\times E_1($loc. cit.). For every $g$ in G and every $(x, y)$ in $E_2\times E_1$, one has

hence $u=\varrho_2(g)\circ u\circ \varrho_1(g^{-1})$. Consequently, $u\in$ Hom$_G(\varrho_2, \varrho_1)$. The proposition follows.

Let $\varrho$ be a unitary representation of a topological group G in a Hilbert space E. If $K =\mathbf{C}$, let $\overline{E}$ denote the conjugate space of E (EVT, V, p. 6). If $K =\mathbf{R}$, put $\overline{E}= E$. The conjugate representation $\overline{\varrho}$ is the representation of G in $\overline{E}$ defined by $\overline{\varrho}(g) =\varrho (g)$ for all $g\in G$. It is a unitary representation of G; by definition, a subspace of $\overline{E}$ is a subrepresentation of $\overline{E}$ if and only if it is a subrepresentation of E.

a) The mapping $u$ is an isomorphism of the conjugate representation $\overline{\varrho}$ onto the contragredient representation $\breve{\varrho}$;

b) Endow $E'$ with the structure of a Hilbert space obtained by transport of structure by means of $u$; the contragredient representation $\breve{\varrho}$ is a unitary representation of G in $E'$.

By EVT, V, p. 15, Theorem 3 and the following remark, the mapping $u$ is an isometric isomorphism.

For every $g$ in G, every $x$ in $\overline{E}$ and every $y$ in E, one has

hence $\breve{\varrho}(g)\circ u=u\circ \overline{\varrho}(g)$, which proves a); assertion b) follows immediately.

(i) The representation $\varrho$ is irreducible;

(ii) The contragredient representation $\breve{\varrho}$ is irreducible;

(iii) The conjugate representation $\overline{\varrho}$ is irreducible.

This results from Proposition 2, and from the remark preceding it concerning the subrepresentations of E.

The space $F^{\circ}$ is closed. For every $x\in F^{\circ}$, every $g\in G$ and every $y\in F$, one has $\langle \varrho (g)x|y\rangle =\langle x|\varrho (g^{-1})y\rangle = 0$ since $\varrho$ is unitary and F is a subrepresentation. Hence $\varrho (g)x\in F^{\circ}$.

The canonical projection of E onto $E/F$ is a G-morphism, hence the mapping of $F^{\circ}$ into $E/F$ which is deduced from it by passing to the subspace is a G-morphism of $F^{\circ}$ into $E/F$; by EVT, V, p. 13, it is an isometric isomorphism.

We shall also denote by $\pi^{\circ}$ the representation of G in $F^{\circ}$.

Suppose that F is a subrepresentation of $\varrho$. Let $x\in E$ and write $x=p(x) +y$ where $y\in F^{\circ}$. For every $g$ in G, one has

and as $\varrho (g)(p(x))$ belongs to F and $\varrho (g)y$ to $F^{\circ}$ (Prop. 3), one has $p(\varrho (g)x) =\varrho (g)(p(x))$. Hence $p$ belongs to Hom$_G(\varrho , \varrho )$.

Conversely, if $p\in$ Hom$_G(\varrho , \varrho )$ then $1_E-p$ is a G-morphism, hence F = Ker(1$_E-p$) is a subrepresentation of $\varrho$.

By Lemma 4 of V, p. 380, it is enough to prove that for every $i$ in I and every $x$ in $E_i$, the mapping $g\mapsto \varrho (g)x$ is continuous at the identity element $e$ of G. This mapping is the composition of the continuous mapping $g\mapsto \varrho_i(g)x_i$ and the canonical injection of $E_i$ into E. It is therefore continuous.

The representation $\varrho$ is called the Hilbert sum of the unitary representations $(\varrho_i)_{i\in I}$; it is denoted by $\varrho =\bigoplus_{i\in I}\varrho_i$.

Let G and H be topological groups. Let $\varrho_1$ (resp. $\varrho_2$) be a unitary representation of G (resp. H) in a Hilbert space $E_1$ (resp. $E_2$). Let $E = E_1\widehat{\otimes}_2E_2$ be the Hilbert tensor product space of $E_1$ and $E_2$ (EVT, V, p. 28, Def. 1). For $(g, h)\in G\times H$, let $\varrho (g, h)$ be the continuous endomorphism $\varrho_1(g)\widehat{\otimes}_2\varrho_2(h)$ of E (EVT, V, p. 28) ; it will be denoted simply by $\varrho_1(g)\otimes \varrho_2(h)$ when there is no danger of ambiguity.

b) For every orthonormal basis $(e_i)_{i\in I}$ of $E_1$, the mapping from the Hilbert sum $\bigoplus_{i\in I}E_2$ into E defined by $(y_i)_{i\in I}\mapsto \sum_{i\in I}e_i\otimes y_i$ is an isometric H-isomorphism of the Hilbert sum $\bigoplus_{i\in I}\varrho_2$ onto Res$^{G\times H}_{\{e\}\times H}(\varrho )$;

c) For every orthonormal basis $(f_j)_{j\in J}$ of $E_2$, the mapping from the Hilbert sum $\bigoplus_{j\in J}E_1$ into E defined by $(x_j)_{j\in J}\mapsto \sum_{j\in J}x_j\otimes f_j$ is an isometric G-isomorphism of the Hilbert sum $\bigoplus_{j\in J}\varrho_1$ onto Res$^{G\times H}_{G\times \{e\}}(\varrho )$.

The mapping $(g, h)\mapsto \varrho (g, h)$ is a homomorphism of G into $\mathbf{G}\mathbf{L}(E) ($cf. EVT, V, p. 28, n$^o2)$. Let $(e_i)_{i\in I}$ be an orthonormal basis of $E_1$. By EVT, V, p. 29, prop. 3 and cor. 2, the mapping

is an isometry of the Hilbert direct sum $F_2=\bigoplus_{i\in I}E_2$ onto E. By means of this isometry, the representation $\varrho_H:h\mapsto \varrho (e, h)$ of H in E is identified with the direct-sum representation of the representations $(\varrho_2)_{i\in I}$ in $F_2$. In particular, it is a unitary representation of H in E (lemma 7). Analogously, the homomorphism $\varrho_G:g\mapsto \varrho (g, e)$ is a unitary representation of G in E.

Let $(g, h)\in G\times H$. We have $\varrho (g, h) =\varrho_G(g)\circ \varrho_H(h)$; therefore $\varrho (g, h)$ is unitary; moreover, $\varrho_G(g)$ and $\varrho_H(h)$ commute, hence lemma 1 of V, p. 377 implies that $\varrho$ is a unitary representation of $G\times H$.

Finally, the assertions concerning the restriction of $\varrho$ to the subgroups $\{e\} \times H$ and $G\times \{e\}$ were obtained in the course of the preceding argument.

The representation $(g, h)\mapsto \varrho_1(g)\otimes \varrho_2(h)$ of $G\times H$ is called the external tensor product of the unitary representations $\varrho_1$ and $\varrho_2$, and is denoted by $\varrho_1\boxtimes \varrho_2$.

Let $n\in \mathbf{N}$ and let $(G_i)_{1\leqslant i\leqslant n}$ be a finite family of topological groups. Let $\varrho_i$ be a unitary representation of $G_i$ in a Hilbert space $E_i$ for $1\leqslant i\leqslant n$. One defines analogously a representation

of $G_1\times \cdots \times G_n$ in the Hilbert space $E = E_1\widehat{\otimes}_2\cdots \widehat{\otimes}_2E_n$ (EVT, V, p. 27).

Suppose that $G_i= G$ for $1\leqslant i\leqslant n$. Let $\Delta_n: G\rightarrow G^n$ be the homomorphism defined by $g\mapsto (g, . . . , g)$ for every $g\in G$. One denotes by $\varrho_1\otimes \cdots \otimes \varrho_n$ the unitary representation $(\varrho_1\boxtimes \cdots \boxtimes \varrho_n)\circ \Delta_n$ of G. One says that it is the tensor product of the unitary representations $\varrho_i$.

For every permutation $\sigma$ of $\{1, . . . , n\}$, the canonical isomorphism

(EVT, V, p. 28) is an isometric isomorphism

of representations of G.

If $\varrho_i=\varrho$ for $1\leqslant i\leqslant n$, where $\varrho$ is a unitary representation of G, one also denotes by $\varrho^{\otimes n}$ the tensor-product representation of the $\varrho_i$, and one says that it is the $n^e$ tensor power of $\varrho$.

The constant function 1 is a matrix coefficient of the trivial representation of G on $\mathbf{C}$. Let $\varrho$ be a unitary representation of G and let $x$ and $y$ be vectors of the space of $\varrho$. For every $\lambda \in \mathbf{C}$ and every $g\in G$, we have $\lambda \langle x|\varrho (g)y\rangle =\langle x|\varrho (g)(\lambda y)\rangle$. Moreover, we have

for every $g\in G$. Consequently, the sets Θ(G) and Υ(G) are stable under multiplication by scalars and under conjugation.

Let $\varrho_1$ and $\varrho_2$ be unitary representations of G; let $(x_1, y_1)$ be vectors of the space of $\varrho_1$ and $(x_2, y_2)$ vectors of the space of $\varrho_2$. For every $g\in G$, we then have

which proves that Θ(G) and Υ(G) are stable under addition and multiplication. The proposition follows.

The space Hom$_G(\varrho , \varrho )$ is a unital stellar subalgebra of $\mathscr{L}(E)$ (lemma 5 of V, p. 380). For every subrepresentation F of E, the orthogonal projector $p$ with image F is an idempotent element of the stellar algebra Hom$_G(\varrho , \varrho )$ (prop. 4 of V, p. 383). If the latter is equal to $\mathbf{C}\cdot 1_E$, this means that $p= 0$ or $p= 1_E$, which means that F = 0 or F = E. Hence $\varrho$ is irreducible.

Conversely, suppose $\varrho$ irreducible. Let $u$ and $v$ be permutable elements of the stellar algebra Hom$_G(\varrho , \varrho )$ such that $uv= 0$. Suppose $u$ nonzero. Then the kernel F of $u$ defines a subrepresentation of $\varrho$ different from E, hence F is reduced to 0; since F contains the image of $v$, it follows that $v= 0$. By proposition 10 of I, p. 113, we therefore have Hom$_G(\varrho , \varrho ) =\mathbf{C}\cdot 1_E$.

The partial operator $u^*\circ u$ is a positive self-adjoint partial operator on E (Prop. 12 of IV, p. 241); the same is true of $v= 1_E+u^*\circ u$, and the latter is injective since $-1\notin$ Sp($u^*\circ u$) (Prop. 17 of IV, p. 248). We have $u^*\circ \pi (g) =\pi (g)\circ u^*$ for all $g\in G$ (Lemma 6 of V, p. 381), whence $v\circ \pi (g) =\pi (g)\circ v$ for all $g\in G$. Since $v$ is injective, it follows that $v^{-1}\circ \pi (g) =\pi (g)\circ v^{-1}$ for all $g\in G$. But $v^{-1}$ belongs to $\mathscr{L}$ (E), hence by Prop. 6 there exists $\lambda \in \mathbf{C}$ such that $v^{-1}=\lambda 1_E$. Necessarily $\lambda \not = 0$, which implies that E = Im($v^{-1}$) $=$ dom($v$)$\subset$ dom($u$), whence dom($u$) $= E$. Since $u$ is closed, we have $u\in \mathscr{L}(E)$ (TVS, I, p. 19, Cor. 5), hence $u\in$ Hom$_G(\pi , \pi )$ and $u$ is a homothety by Prop. 6.