Neons are actually better in this application. Firstly driving LEDs from mains can require quite high wattage resistors , secondly faults in LEDs result in shorts with quite a nasty destruction of the PN junction.
A neon has two advantages first , under the strike voltage no current
flows, and b , the current is very low , resulting in low power
No circuit should have a hot to protective earth led ( or any other circuit) , this injects current continuously into the earth wire , which is bad practice.
The standard circuit is a neon between neutral and earth . Its has the advantage that in normal operation the neon is off and no current is injected into protective earth.
If you want to test the safety ground , but the hot to earth indicator on a test button