Sean,
I can remember when it was common to have three feet of ice in the harbor and CG Buoy Tenders breaking it constantly, but those days seem to be gone. Portland is our favorite getaway in the winter. I can't remember the last time that the harbor froze over. As for the "grittyness" of the city, while the folks on some of the streets (Congress Street, especially) may look less than desirable, they are harmless. My wife and I walk the streets throughtout the downtown area and have never even been approached by a panhandler. I would consider it to be a safe and congenial city.
Jim