Check out this link:
Free GIS and CAD Software
You are heading into a confusing realm (at least to me). I tried to navigate that world back when I was first trying to come up with a way to create navigation
Just so you know, not all vector charts are created equal. There are many, many formats, most of which are incompatible with each other.
BUT, in the GIS world, likely you can find someone who has already done what you are trying to do. It just may take some digging.
You might post this question in the OpenCPN
sub-forum. I think there is someone in there that makes a living doing that.