MIKE AZEKA:
Subwoofers are hard to blend with the main speakers, and require careful placement, adjustment, and room treatment to smoothly blend with the main speakers. Subwoofers are known to excite standing waves in rooms that otherwise sound good at higher frequencies, the best subwoofer location to avoid this might be too far into the room for appearance and tastes. If you play vinyl it can cause muddy sound if subwoofer vibration feeds back slightly through the turntable.
Both the main speakers and the subwoofer overlap in frequency near the crossover point, they can cancel each other and create noticeable dips in the frequency response, or sound different from each other near the crossover frequency. Non-audiophiles aren't concerned with seamless blending of the subwoofer, audiophiles often obsess on things like dips and bulges in frequency response, obvious differences in perceived “speed” of the main speakers versus the subwoofer, and avoiding room standing waves.
Lastly, high performance subwoofers are expensive, over $1,000 is normal. Some audiophiles have already spent a lot on their hobby, they have a hard time spending up to $3000 more for a good subwoofer.