Sure, it would be a good idea to check out caves, because you can earn quite a lot of EP (and money, and items...) by killing opponents, and not only by finishing sidequests. Didn't you go in there before, not even out of curiosity? Exploring everything is half the fun of the game, in my opinion, but to each his own.
As to (*) sidequests, the bounty quests from the noticeboard in Riverville should be designed for low level characters. Most (**) quests aren't terribly difficult either, sometimes it just indicates that you would have to walk a bit further to find your destination than that you would encounter stronger enemies.