It's good to keep it on 2-3 stacks because using E takes some of your health so if you spam 4 stacks E without possibility to heal up you'll be 50% health before fight starts. You must predict when someone might be close to you and then keep E with 4 stacks

(Also remember that only 25% of your original spell vamp works with AOE so if you have 20% spell vamp your E, R and W will have only 5% spell vamp) Remember that your W deals the more dmg the more BONUS health you have so rushing haunting guise or later going rylai is sometimes a good thing.