If you guys like to snorkel/scuba, I remember Blue Heron bridge (just north of Peanut Island) being an interesting place during flood tide.
Just to the north of you on the west side, you'll find Riviera Beach... The marina (many dive boats operate from here - good/entertaining restaurant as well) is ok and secure, however, I do not
recommend walking around town in this area - definitely a shady area - I used to spend a fair amount of time in this marina when I taught diving
Ditto on the West Palm Beach advise - good area but you will definitely need transportation to get to/from there.
Once in a while, there used to be a floating bar by Peanut Island (years ago) - I don't know if they are still around...