Because while you're dragging with mouse, it does not actually move anything, it just pretends. The change to the document happens only as you release the mouse, and then the delay is the same as when moving by an arrow key. It cannot possibly be different as it's the same code at work.
Well, it is different... but your explanation tells me why. It's only updating the document once with mouse, as opposed to 25 times when I use shift+arrow a bit. =)
Ah, of course. When you press arrow once, Inkscape has no way of knowing you're gonna press it 10 more times after that, so it does the entire document updating routine for all objects on each keypress. Use Shift+arrows to move objects 10 times as far in one go.
Still, document updating should be optimized if possible. I remember Mental saying something about hordes of listeners attached to repr nodes and the time needed to process them all. Mental, can you comment - is there any chance this can be sped up noticeably?