Proposition 5

Let $n\in I$.

a) For every subspace $F\in \mathscr{F}_{n+1}$ adapted to $u$, one has $\lambda_n(u) =r_F(u)$;

b) For every subspace $F\in \mathscr{F}_n$ adapted to $u$, one has $\lambda_n(u) = R_F(u)$.

Let $F\in \mathscr{F}_{n+1}$ be a subspace adapted to $u$, and $(f_i)_{0\leqslant i\leqslant n}$ an orthonormal basis of F such that $u(f_i) =\lambda_i(u)f_i$ for every $i$. For every $x$ in F, one has

with equality if $x=f_n$. This implies that $r_F(u) =\lambda_n(u)$, whence assertion a).

Let $F\in \mathscr{F}_n$ be a subspace adapted to $u$. The closed subspace $F^{\circ}$ is nonzero (for $n=$ dim(F) $<$ dim(E)) and stable under $u$; the endomorphism $\widetilde{u}$ of $F^{\circ}$ deduced from $u$ by passing to subspaces is compact and positive.

One has Sp($\widetilde{u}$)$\subset$ Sp($u$). Moreover, one has Sp($\widetilde{u}$)$\subset [0, \lambda_n(u)]$. In fact, it is enough to verify that $\lambda \leqslant \lambda_n(u)$ for every $\lambda \in$ Sp$_s(\widetilde{u})$. The number $\lambda$ is then an eigenvalue of $u$, hence there exists $j\in I$ such that $\lambda =\lambda_j(u)$. The eigenspace of $u$ relative to $\lambda_j(u)$ is therefore not contained in F, which implies that $\lambda_j(u)\leqslant \lambda_n(u)$.

Suppose that $\lambda_n(u)>0$. Then the eigenspace of $u$ relative to $\lambda_n(u)$ is not contained in F, and therefore $\lambda_n(u)$ belongs to the point spectrum of $\widetilde{u}$. It follows that sup(Sp($\widetilde{u}$)) $=\lambda_n(u)$. If $\lambda_n(u) = 0$, one has the same result since the spectrum of $\widetilde{u}$ is then reduced to $\{0\}$.

Moreover, by definition one has $R_F(u) = R_{\{0\}}(\widetilde{u})$, and finally $R_{\{0\}}(\widetilde{u}) =$ sup(Sp($\widetilde{u}$)) by Prop. 9 of I, p. 139, a). Assertion b) is proved.