I own the Zoom H2, the Sony PCM-D50 and have a trial LS-10.
The Zoom H2 does very good recordings with the internal mics and is capable of surround sound, which the others aren't. But its cheap build quality disappoints. Significant handling noise.
The small LS-10 is the most compact, has the best menu system but its internal mics lean a bit on the "high end." That takes about 10 seconds to adjust in Audacity or some other audio editor by applying a Bass Boost effect.
The Sony PCM-D50 is solidly built but not pocket sized. The Sony dead cat windscreen is ridiculously priced at $50. But it gives pleasing recordings on both the internal mics and any externals I attach.
I haven't tried the Edirol recorders.
If you are going to be doing voicetracks, meaning have someone tell the story, you will probably want to use an external mic, like an ElectroVoice E50. That gives too much hiss on the H2 but works on the Olympus and Sony.
The only mic I've found that works suitably on the H2 is a Sennheiser M66/K6 combo...it really pumps audio power. That's about $600 worth of microphone with the grip and dead cat.
Of these three, the Sony is the best bet. The Olympus a close second (unless size is of serious importance,) and the H2 is fine for recording ambient sounds.