Southern Ca VS North Carolina

[deluxe68]

We are going to be moving aboard our boat later this year or next year. We have still not decided east coast vs west coast. We bought the boat at its current home (Rhode Island) and after a year the only two tax free (warm) options seem to be North Carolina or Southern California. We have very little blue water experience so we want to do shorter cruising at first. The marina prices in NC are half of what they would be in San Diego but I am kinda put off by of the stories from sailors concerning the frying pan et al shoals close to the east coast. We are not really interested in using the ICW (mast too tall) and would rather stay out away from the shallow stuff. Am I just being paranoid? Is the North Carolina coastal area that nasty for cruising?