I bought a 54' HR and as said above, all boats feel small at sea so I'm happy to have the extra safety
and speed of a bigger boat. In marinas
however handling can be difficult because most marinas
seem to have been designed for shorter boats. I would not have bought the boat if I had not been able to install a stern thruster. (it had a bow thruster). The old hands poo poo the idea of thrusters etc for boat handling but for someone without years of experience it can be a big help in tight spaces. Draft
is also a matter to consider and a deep keel
does restrict options somewhat so it depends where one plans to go.
The costs and maintenance
on a boat seem to go up exponentially with size so that is a personal factor that has to be considered. The upgrade costs of a big old boat can be quite scary.
Physical strength for operating the boat is less of a problem than it used to be and a few electric
winches can make life easier which is one reason why boats are getting longer every year.