I was in Colombier four nights ago and prefer it over Gustavia. You can anchor
there, but the wardens prefer it if you take a mooring
ball. You can anchor
between the two mooring fields in 15 feet or so of water
in sand. The mooring balls have no pennants, which can make it difficult to pick them up; if you do go there take the northern mooring field as the southern one can get backwinded and I ended up getting tangled mooring lines there and had to cut them away!
There is lots of room in the Gustavia anchorages
and the holding in 20-25 feet is excellent with sand and clear water
(lots of turtles) but there is always a swell that can get uncomfortable. The inside med-mooring places tend to be reserved for the megayachts but you can get a slot there if you ask at the capitanerie, the prices are similar to alongside dockspace elsewhere.
for some pictures of St. Barths and Colombier
(Zanshin is the blue boat close to the big rocky island, I was there just to clear in before heading to Colombier)