Have done 2 seasons in Bahamas and embarking on 3rd in a boat that draws 7'6" no problems at all just read the charts
and go where you can. There are at least a 1000 anchorages
that you can which is enough for most peoples life times!
and the US coast is the issue, we have 66' mast and our standard route
Hampton - Cape Lookout Bight - Wrightsville Beach - Charleston - St Marys - Lake Worth
. This means anchoring and no marinas/bridges.
You would also have an option at Port Royal, Canaveral (marina) and St Augustine we just haven't bothered. I may have missed others but that route
means no passage
more than 1 night at sea and gets you south quickly.