Some of the orbs are just there for decoration; others are used to solve puzzles in certain areas; others still are traps that go off when you get near.
You'll find the decorations and traps most commonly in dungeons populated with magic users, and I think that the puzzle-related orbs are pretty much limited to areas you visit in the course of solving certain quests.