When you kill enemies or complete quests, you receive experience points. Once you have enough, you advance to the next level. When that happens (1) you choose to increase health, mana or stamina, (2) you receive learning points for increasing combat skills, handicraft points for increasing non-combat skills (enchant,lockpick,etc), and a memory point to select a talent. You can see how many points you have on your character page (H key).
To use the memory point, go to your magic (P key) and select Meditate, then hit the Y key to go to your memory. You can select one talent on the memory trees for each memory point you have. Then hit Y again to go back to the real world.
To increase your skills, you need to find or buy learning books. Reading a learning book uses up 1 learning point or 1 handicraft point, depending on the type of skill, and increases that skill by 1 point. (See your character page H key). Apprentice books are used to increase skills up to level 25, Adept books 26-50, expert books level 51-75, and master books level 76-100.