Sounds like a pretty tough situation that you are going through right now. First I would like to just put out that just because your mom didn't actually raise you/take care of you as a child doesn't mean that she doesn't love you. I'm sure that she loves you very much, but having her father raise you was the best option at the time for both you and your mother.
As for you going to see you mother, I don't see why not. Sit down with your current adoption parents and tell them how you are felling, and that you want to at least try going and reuniting with your birth mother. Chances are that your birth mother is a very loving person, that has probably wanted to be with you for a long time. In my opinion I just don't think that any child should be separated from there own birth mother when they have the chance to reunite.
As for you leaving your current adopted family, It's really your choice. One thing to remember is that just because you go to live with your mom/visit your mom doesn't mean that you and your adoption parents will be separated forever. There shouldn't be a problem with you going in visiting them every now and again, as I assume you have created a pretty good bond between yourself and your adoption parents.
Anyways I wish you the best of luck in whatever you do, and don't be persuaded completely by my assistance. If you want to stay with your adopted parents then do it, and if you want to live with your real mother that's fine as well. There is really no right or wrong answer here, and my assistance is just to try and help you along the way. Again I hope this helps, and good luck!