There's always some moisture in a balsa core. Dealing with that is not the big deal but finding the cause is really more important because that could indicate a far larger problem than just moisture in the hull. So the real question is WHY there's moisture there.
Moisture in the core around fittings, penetrations and places where the boat took a bump is pretty normal and a logical place for moisture to be. It isn't great but "to be expected". It is just a result of a badly done job when the fitting was installed, or over time the caulking around the fitting has given up, or a crack in the
fiberglass has allowed
water intrusion. This is why some people avoid boats with balsa core hulls
Moisture in places on the hull that are not close to fittings and penetrations can be a bigger deal. For example, apart from pox, if the high moisture levels on the hull is behind where the ice box is located, could mean that
water has found its way into the
insulation *from inside the icebox* and so you'll need to tear out the whole thing and redo it.
The bottom line is that eventually moisture will cause
delamination if it is not stopped. It is only a matter of time, like a bad tooth. Maybe a long time. Maybe not.
Fixing
rot in core is really not that big of a deal: cut the fiberglass, dig out the core and grind around the hole, stick in some foam and filler, glass over it. Can be done by non-experts. Plenty of Youtube videos. The real pain is then trying to make the patch match the rest of the boat. This may require repainting the whole thing. Which means stripping off the old
paint, filling and fairing,
sanding some more, priming, at two coats of
paint expertly applied so as to not show brush strokes etc etc. A big job. Of course under the waterline any patch would be covered by antifouling.
Whether you should walk away or not is a personal question and depends on your finances, willingness to
DIY, cost of labor around you etc. There are always other boats! BUT any old boat will need
work. That's just life. Some of us like it!

It is just a question of first figuring out what the problem *really* is before
buying in.