This theme has been hashed and rehashed and I'm not sure there's much new to say about it.
Once a boat is too big to manhandle the last foot or so into position (more than say 45 feet), there are few disadvantages to size, in terms of short-handed handling, so the bigger the better, from that point of view (not
from the point of view of cost, berthing availability/cost, etc., etc.). Bigger boats are certainly easier to handle at sea as they provide a more stable platform. They also tend to be blown around less during docking
Single-handing any boat is scary for me; but single-handing my 54 footer was not harder than single
handing other boats of my experience, and in some ways it was easier. The biggest disadvantage for single-handing was the considerable distance from the helm
to the rail which needed to be covered in order to get dock lines on.
winches and bow thrusters on larger boats are very useful, and not just for short-handed sailing.
I absolutely never find myself thinking "wow, I wish I had a smaller boat," except maybe when I'm buying
antifoul. If that's any hint. And I very often find myself thinking "Thank God I didn't buy that Oyster 485; it would have been a little tight." There's a very good reason why 99% of sailors go at least a little bigger, when changing boats, and exceedingly rarely, do they go smaller.
And my boat, although she is intelligently designed by one of the real masters in the field, does not have enough lazarette or forepeak or other technical space, and has too small an engine
space (not quite walk-in). I would sacrifice some of the accommodation to have more technical space, or, wouldn't mind too much a slightly larger boat to solve those problems. 60 feet would be fine, I think.
That's my point of view; YMMV.