We have a Yanmar
4JH3E. The rated max RPM
is 3,800 but like Our Doug we don't usually run over 3,000. I have seen a "loss of RPM" before but it was the throttle cable system slipping. It seems that this engine has a tendency to "push back" on the throttle when revved above 3,000. So in our case it was a mechanical thing.
If the prop is over pitched it should cause some dark smoke. Nothing like that was reported. I doubt the exhaust elbow can intermittently clog. It is possible the pickup tube was getting temporarily clogged but usually the symptom is the engine revs up and then dies.
Next time you have a chance do a long straight run. Push the revs up to 3,500 for a couple minutes and see if they hold. If revs fall off carefully look at the throttle linkage to be sure the cable is not slipping slightly.