I worked on a Cape Dory
that had the same problem, there are two ways to fix this.
First you can have the prop milled down to reset the frist nut then put the second backing nut on with the cotter pin cuts in it.
Second option a the one I used, I ordered two bronze nuts from Bearing Headqorters, one was a hafe nut and the other had the slots for the cotter pin, I still had to cut the backing nut down 1/8 to be sure I eas past the taper on the shaft.
Been over a year not and the guy tells me it is holding fine.