I would suspect that the charger
is connected to the batteries "downstream" of the ammeter (or the shunt for the ammeter, whichever you have) and that if that connection was moved "upstream" of the ammeter/shunt, then the ammeter would not know anything about the charger
. It wouldn't be deflected backwards, it would have no way to know what the charger was doing, and the problem would be solved
just by moving one connection point, nothing extra needed.
Should be no issues affecting performance, etc. this way, either.