You are indeed interpreting it incorrectly and although I could explain it, I am feeling terribly lazy. I think I'll just let some one else do it. Shameful, I know.
I just wanted to add my two cents.
I used to think about this, when I was much younger I had a long think about this, but I stopped thinking about it so I didn't reach a conclusion.
But looking back, I've found that I'm thinking three things.
1 - It's possible to some extent. If you replace the colour green with blue, blue with red, and red with green, and let all the other colours change respectively, there would be no way of telling, as long as the tint, saturation etc are all the same. The difference between this and colour blindness is that with colour blindless there is evidence because it makes a difference. This wouldn't make any difference that would be visible to anyone else, so there's a distinct division between the two.
2 - It doesn't really matter. Because if this does indeed happen, it doesn't really affect anything. (Still, interesting to think about)
3 - While I find it entirely plausible (at least when not thinking about it biologically), I don't believe it actually happens, for the reason that there is no reason for it to happen. (As far as we know)
That's kind of like the giant spaghetti monster argument, in a way. But with a different purpose, of course.
