The problem with prostitution, at present, is not the idea of paying for sex. That, in itself, is at an absolute maximum, an issue of morals, and I am of the opinion that the government should not be the bureau of morality. Morality is something ingrained into children during their socialisation process, and to interfere with that (when parents, for the most part, are perfectly capable of doing so) serves only to cause conflict between people of opposing views on an issue that barely affects their lives.
The issue with prostitution, much like drugs, is the black market surrounding it due to its legality. By forcing such a huge industry (with legitimate applications, such as caring for physical/mental invalids) into the shadows like it has been in much of Western society, we are forcing it into the realm of drug abuse, slavery, violence and other crime that is often associated with prostitution (especially in third world countries).
The solution, frankly, is to regulate. One can make their own moral judgement on the industry, and they are free to do so. One, however, should not be able to force their principles onto the entirety of a country purely because they have a certain view towards the issue.
People who feel that the sex industry has no legitimate purpose should refer to
this video. It's a stand up video from an Australian guy (prostitution is legal in brothels here in most of Australia, if not all), and goes into pretty fair detail.