To defend yourself, or defend others.
To defend yourself; self-defense. If someones attacking you, I don't know many people that would not fight back at all. Self-defense is in your best interest, be it running away or otherwise, you are still defending yourself and hence using violence as a means is acceptable.
To defend others. If someone is being mugged infront of you, and the person being mugged pleas for your help, violence is acceptable. Though it may not be in your best interest to help, perhaps there's a gang of muggers, it doesn't negate the fact that the use of violence is acceptable. Someone has had force initiated on them, and they are begging for you to help, violence is acceptable as not only would the person being mugged act on self-defense, you are defending the rights of that person.
These two would be the only accepted forms of violence. There aren't m/any moral pitfalls when extrapolating these two principles, they cover virtually everything.