It's not the meat—it's the miles.
(My understanding is that even relatively green meat is "dirtier" than relatively unsustainable plants, locavores notwithstanding.)
I once had some tell me that eating soy was bad because of the environmental impacts. It was in a vegan vs meat eater debate, and it took me a bit to realize he had no idea what was fed to cows and other farm animals. I hear the "growing x is so unsustainable" to a degree that surprises me as an anti-veg argument, because part of me assumes that most people are familiar with the sheer quantity that is needed to sustain meat production. I guess I expect too much from people.