If you intend on visiting the USA via sailboat, then the visa waiver program does not apply and you will need to get a US Visa while still in the U.K. (this applies to all Visa-Waiver countries). This will take several weeks and, depending upon what the consular officials decide, you might need to supply things such as titles/deeds to property or rental contracts so that you can prove that you have ties to the U.K. and do not intend on overstaying in the USA when you visit. The final step is an interview, which I think can currently only be done in London, so you'd have to travel there for a day.
None of the commonly visited Caribbean
countries require you to have a visa to visit, but keep in mind that the BVI, while part of the commonwealth, will only grant you the normal 2-3 week visa stamp on arrival.
If you intend to cruise
around the USA by boat you will need to apply for a cruising permit
and remember that each time you lift
up and drop anchor
you will have to call in to Homeland Security
or risk paying a hefty fine. Recently a thread was open here about a boat which changed their dock
in the same marina without notification and were subsequently fined!
(Sorry, I just re-read your post and see that you were referring to the USA visa. I believe that you have to apply for the visa from your place/country of residence, but I have heard that there might be exceptions made, I believe that the Bahamas
were mentioned, but it would be much easier to do this from home. I had to make 2 visits to the US consulate to finish my visa application, luckily the consulate at the time was only 1 hour away.).