Downtown is tiny and so cute! I had to go to pubs because that is my business (I had the pad thai at Gahan that I've had at the location in Halifax, and I don't recommend it--so overpriced but I was tired of just watching my boss eat nachos), and didn't eat anything delicious. However, I saw a bunch of falafel places and sushi. They have a Formosa Tea House (I've never been to the one in Dartmouth so I can't comment).
For fun, I loved Marc's Lounge. There was live music and it was very intimate, people were nice and there was a great vibe--I felt like I wanted to hang out there forever!
From here: http://www.vegpei.com/restaurants.html
it looks like the vegan options are mostly of the inexpensive and sushi/Lebanese/Chinese persuasion. Which is good because it's an expensive little city! Have fun and report back--I can't wait to go back to PEI.