Another possibility is to use SIOX for this mode. We already have it
in our codebase - it's used as one of the tracing modes. Its inputs
are exactly this: a bitmap and a path covering some areas of it, and
it selects all areas that "look similar" (see http://www.siox.org/).
It would be perfect to try both SIOX and a simple fill-from-each-point
and compare the results - maybe one of these will be less useful than
the other, but maybe they will be both useful so we'll keep both :)