Definition 5

Let G be a topological group and let $\varrho$ be a unitary representation of G in a Hilbert space E. Let $x$ and $y$ be elements of E. The function from G into K given by $g\mapsto \langle x|\varrho (g)y\rangle$ is called a matrix coefficient of $\varrho$, or a representative function. If $x=y$, it is said to be a diagonal matrix coefficient. If $\varrho$ is finite-dimensional, it is said to be a finite-dimensional matrix coefficient.

The matrix coefficients of $\varrho$ are continuous bounded functions on G. We denote by Υ(G) (resp. Θ(G)) the set of matrix coefficients of complex unitary representations (resp. complex finite-dimensional unitary representations) of G.