Nope, it’s enough to unfocus with ESC, although, as you’ve already pointed out, the cursor position is thereby lost.