"Records, aka LPs, have been around since the 1950s, so there are lots of them out there. I've bought great records for a buck or two at thrift shops and yard sales, and found them on the street for free, but records aren't yesterday's news; lots of young bands are releasing LPs. The way things are going, the LP will probably outlast the CD as a mainstream format.
Speaking of yard sales and thrift shops, you can probably find dirt cheap turntables in those places, but the chances of finding a turntable in good working condition there are small. Turntables are fairly delicate devices, and there are lots of ways they can go bad that might not be immediately obvious. That's why I don't recommend buying used turntables, unless you know the owner and he or she can verify its condition."