That exact problem has happened to us three times with our Autohelm
Type 100 course computer and ST60 control head
Both times the problem was intermittent but when it occurred - it was exactly as you described. The 1st & 2nd time was way up north in the Sea of Cortez
with no possibility of professional help. The third time was between Tacoma and Seattle
as my brother and I headed out for a non-stop sail to San Diego
The first time the problem occurred it was an intermittent connection in the fluxgate
compass. When it made and broke contact it sent incorrect heading measurements to the course computer (e.g actual heading 90, intended heading 90, and Fluxgate
was sending 20, 160, 55...etc). The poor course computer just tried to steer the boat back to the requested heading.
I resoldered the fine little wire connection in the Fluxgate and that solved
the problem for a year or so.
The 2nd time was an intermittent power wire breakage from the course computer to the Fluxgate compass. I could see the break where the 12V power wire had rubbed on the edge of a compartment it passed thru. I spliced in a new section of wire and that solved
the problem for almost eight years.
The 3rd time was also an intermittent 12V fluxgate connection but was impossible to find. We measured the resistance between the course computer and fluxgate and found an intermittent high resistance but were unable to find the source of the resistance. We wired in an entirely new 12V power lead to the fluxgate and that solved the problem.
In May 2013 I replaced the 19 year old fluxgate compass because it was starting to provide very slowly changing values and the Type 100 would try to follow those changing values.
Hope this helps.