From IvanWiki

Higher Agility allows you to dodge attacks and move around more quickly.

Carrying a light pack trains your agility. Drinking Ommel Sweat also increases Agility and Dexterity.

You can wear boots of agility to increase Agility, with each +1 in AV also resulting in a +1 bonus to agility.