Having been to Madison recently, I think the stretch of State St. that the Canterbury is off of is not one of the louder stretches (and it is set a half-block or so off
re the Farmer's market - look for one of the many cheese vendors that sell 10 year old chedder - amazing, if not cheap, cheese.
Also there is a Thai restaurant a block off of State Street (in what looks like an old autobody shop) which is extremely good.
For one, they will make any dish on the menu very spicy if you ask.
But more interestingly (which I have to test on a future visit), the owner offered the last time I was there to cook nearly anything if I asked him ahead of time - i.e. Madison too may have a Thai restaurant willing to make "authentic" dishes.
Anyway one of my favorites of Madison.
Shannon