Well metal armour has a negative magic defence bonus, so magic will hit you harder whilst you are wearing it. Try wearing dragon hide or leather, as these types of armour have a good magic defence bonus. Try raising your magic level aswell; the higher your magic level, the higher your resistance to magical attacks against you will be.