Advise on removing pin from door handle

[hlev00]

The picture shows a door handle with a pin holding it place. I have a punch set but am not confident in which way to proceed. The pin is recessed slightly on the top of the handle and protrudes slightly on the bottom. This is consistent on all doors on the boat. I think the pins are tapered and so were inserted from bellow (where the pin is still protruding). When I try to punch out from the top there is no movement.

My guess is that there is corrosion making this process more difficult but I wanted to check on advice before I make things worse. I will try a drop of Blaster or WD40 before trying again.

Thanks,
Harry