I entirely agree with what Mitchellm posted: I put forward what I thought would be the best you can do for the least amount of money. The next step up would be one of the newer compact PCM recorders, which record at higher sampling rates, use uncompressed file formats, record to removable flash media, and have better internal bits that do some cool stuff I don't understand but seems to make a difference to those who do.
The current choices include the Marantz 620; Sony PCM-D50; Olympus LS-10 ...etc., which run $300-$500. Check transom.org for reviews. Similarly for better mics: I'd look at the Audio Technica AT-875 and AT-897, both are shotgun mics at around $200-$300.
There are undoubtedly other good buys in that same price range. I've just listed what I'd buy when the day comes that there's $500 burning a hole in my pocket. Oddly, it hasn't happened yet.