That happened to me all the time with my old computer. I got a new computer and it doesn't do that as much.
I also don't use my router when I'm just on the computer because it seems to lower my bandwidth and make me lag. I have tried multiple routers from Netgear, D-Link, and Linksys but I'd have to say as far as lag and overheating goes, I prefer my Belkin.
if your router is overheating, Try to not put stuff on top of it. Also, try setting it out in the open. I know my router is always stuffed behind my monitor, but I know it doesn't get proper ventilation there. Try leaving it set out on you're desk and just trying the obvious way's to keep it from overheating