Just a random thought: my very first boat to own was a 42'
classic plastic boat. It was only the second boat I had ever steered let alone cruised in. And the other was a little rental in the Bay of Biscayne which I promptly grounded on a sandbar across the bay (with my girlfriend) and had to have a tow at sundown from the rental place. I was suitably mortified and convinced I had no understanding let alone skill in sailing any kind of boat.
Fast forward to when I was 50, in Colorado, and I got a bug up my you-know-where to go cruising which required a suitable boat. I researched and researched and researched and decided a 37' boat with a wide following and reputation would be suitable. I then proceeded to buy an off brand 42' Whitby, which turned out to be a very reputable boat but I really didn't know that at the time.
So the Admiral and I were fully petrified the first year of this monster heavy boat. We quivered in fear every time we pushed off from the
dock for the first year, wondering how many other
boats we would crash in to before we even got out of the marina. And then wondering if we could come back unsunk and untowed. And then
dock with knees quivering and words being shouted from the Admiral that I was going too fast and needed to slow down (every
single time).
We just took baby steps and ended up selling the boat in
New Zealand 8 years later after many wonderous adventures and smiles on our faces. We came back and 10 years later just bought another lesser (almost unknown) known boat that that was once 42' before the PO added a couple more.
More fun to follow. The moral of the story: like the other poster said (or implied), you don't need to start with a
small boat. Go with one that will suit you for your cruising dreams and then learn as you go. Do your
research, make a decision, and make it
work. Just understand how different it will be, and how ridiculously expensive it will be to fix and upgrade any new or
used boat you will get. And then be prepared to smile a lot.