Smaller boats are sailed more. Bigger boats take more time to maintain and of course everything costs more.
Find a sea kindly hull
as small as you can with the most accommdation and stowage for the type of use you intend. And don't forget that you want to be self sufficient and not require crew to sail or move your boat
If you are weekending you don't need tankage and stowage for example.
Shiva is a 36' Contest which meets my criteria. Very large cockpit
, main salon
, even an aft cabin
, fullsize chart table, U shaped galley
and a fair amount of tankage and lockers a plenty. I've lived aboard, done plenty of offshore
passages singlehanded and with crew and now cruise
her coastally. More than 4 on board for passage
Most important she was easy to set up for single
and short handed sailing and this was very important. I see big yachts and there's always several people on board. I prefer the privacy and I think nothing of taking off in any conditions - alone. This type of freedom is what sailing is to me. And I have all the creature comforts except AC which is hardly needed.
I would look long and hard at lots of plans and make sure they have what you want, and they can be easily sailed and handled.