Since you are always seeing voltage on your meter when switching banks, sounds like you have a 'make before break' switch. In other words, the current
battery bank is not switched off before the new battery bank is on line. If you have that type, switch you can switch at will, just don't turn it off with the engine
One way to be sure that you don't damage the alternator is only switch between batteries with the engine
at IDLE. If you switch at higher rpm
, the alternator can produce enough juice to fry itself.
The change in voltage probably results because of different loads from the batteries because of different size, charge and/or condition of the different banks.