As Rezon suggested, that issue is known as rubber banding. In the 9,480 spars I've completed, I recall my opponent rubber banding in no more than 10 spars (of which I lost 8 or 9). It really is quite a rare occurrence. I predict that you will respond to this with "it happens all the time in towers blah blah":

The title of this thread is "Best iDevice Sparrer".

Anyways, if this is the issue that you've been trying to push through this entire thread, it doesn't exactly make you look too bright, since minor latency (under 500) and rubber banding cause two completely different scenarios.