Darkweasel, flagsarenotlanguages.com (and others) would agree with you.
A nice compromise would be flags just for languages; many conlangs (like Esperanto) have them. The problem would be recognition, but you could use something intuitive. Like a standard sample of Hangul for Korean (of course, your suggestion to use names could accommodate this too).
Anyway, there is already misuse of terms: for instance, "English" suggests the language of the Angles, but many other peoples use it now, as you said.
-Arlo James Barnes