Combat on Horseback in real life your horse does not slow down from you engaging your enemy. The Combatant is trained to maintain how to do it on horseback and the horses themselves are trained for this kind of Combat as well. so real life does not even apply to this negative effect on mounts in Graal Classic.

Maybe only the horses shouldn't slow down from swinging your sword and only other mounts should