Asking for book recommendations is pretty useless in your case. You barely read, so you can't give examples of what you enjoy.
And then someone will suggest something, you'll try to read it, it won't be for you, you'll think "gosh, if this was supposed to be good, then reading must just suck", and there ends your resolution to read more.
To accomplish your specific goal of surrounding yourself with intellectual people, start reading classic novels. To Kill a Mockingbird, Animal Farm, 1984, Great Gatsby, Pride and Prejudiced, The Catcher in the Rye, Of Mice and Men, etc.
Reading ANYTHING will help you intellectually because it will broaden your vocabulary. Even bad authors tend to use words you're not familiar with, which helps your vocabulary, and ultimately you're ability to articulate. The better you articulate, the better you can express ideas. This is an important skill in communicating with intellectual crowds, even if you yourself aren't yet super smart yourself... if that makes sense at all.
With that in mind, ignore the age old axiom "don't just a book by its cover", find a book that looks interesting to you, pick it up, read it, and see if you like it. If you like dragons, great. Find a cover with dragons all over it. If you like medieval themed, look for something covered with swords or castles. If you like sports----- rinse and repeat.
