This does sound like an interesting feature to play around with.
I had a thought while reading the description. Since you can switch which HSL component you're modifying during the drag, it might be nice to have a modifier key that means "no change", so you can reposition the mouse cursor without affecting anything.
Here's my line of thought: Suppose you're modifying saturation, and you want the minimum. You Shift-drag to the bottom of the window. Now you want to decrease the lightness. But you can't, because the mouse cursor can't move down any farther. Or if you weren't at the absolute bottom, you'll have much less precision for the lightness because you only have a very small area in the downward direction. And if you try to reposition, you'll change something you don't want. You could release the mouse, and Ctrl-drag from the swatch again, sure. But what about making Shift-Ctrl-drag into a no change mode, so you can reposition the mouse, and then release either Shift or Ctrl or both, and go back to modifying another component.
Kurt