For an operator $A$ on a complex Banach space $X$ and a closed subspace $M$ of $X$, the local commutant of $A$ at $M$ is the set $C(A;M)$ of all operators $T$ such that $TAx = AT x$, for all vectors $x$ in $M$. It is clear that $C(A;M)$ is a closed space of operators, however, it is not an algebra, in general. We will show that $C(A;M)$ is an algebra if and only if the subspace $M_{A}$ of all those vectors $x$ such that $TAx = AT x$, for all $T$ in $C(A;M)$, is ultrainvariant, that is, invariant for every operator in $C(A;M)$.