That is a good idea in itself, but we just don't have any more
modifiers left (I plan to use Alt for "no change" drag). And in any
case it would be more logical to adjust opacity by dragging the
opacity control, not color swatch. Though, unfortunately, it looks
like this trick won't work with a spinbutton anyway.
Do modifier keys have to be either SHIFT, ALT or CTRL?