So far you've gotten some good
advice.
I don't call
solar panels "12v" or "24v" etc, your panels are 36v panels and the most important number is your 45Voc. This will determine the max. number of panels in series based on max. input voltage of your controller.
Your controller is rated at 150v input max. So with those panels, you would be able to connect them in 2 parallel strings of 2 ea. If you had 6, you could do 2 strings of 3 ea with a max voltage of 135v.
As far as max. wattage goes, *theoretically* you could put 1250w on it for a 12V
battery bank, or if you have a 24v battery bank, you could install 2500w.
However, since MPPT controllers self regulate to their max. output, there isn't anything wrong with going big if you have the room for it. The 1250watts = 80 amps equation would be under perfect conditions - midsummer, noon, no shading, etc. The reality is, the panels are mounted flat, the sun is rarely directly overhead, the
winter sun is lower in the sky and there is always shading on a
boat. So if you have the room for it, you could install 8 panels for 1440 watts and you'd get more power, more often, for longer periods without blowing anything up.
As far as wire size goes, the higher the voltage, the lower the current, which allows you to get away with a smaller gauge wire. The correct answer would be based on total length from panels to controller (both ways, not one way) and total current expected when you've finished adding panels. I'd strongly recommend a 2 panels per string setup unless you decide to install panels in sets of 3, in which case you could install 3 panels strings for a max of 6 or 9. If you plan on doing 8 panels then figure 4 strings of 2 panels in series. The 90Voc and 20 amp current max. at the junction box would dictate your correct wire size from there to the controller. The short branch connections to the junction box with 10awg and MC4 connectors should be fine, if all 8 panels are next to each other. If you have them spread out all over the
boat, then you'll have to calculate branch wire size based on 5A per branch.
Hope this helps!