The current setup was a Dell DJ MP3 Player attacked to a $50 boombox.
I pulled out some Bose bookshelf speakers that had been sitting idle on a shelf. I pulled apart the sound system for the computer. This was a Creative system from 8 years ago that ran about $100. I figure, the subwoofer/amp was about $70 and the speakers were $30. I dumped the speakers and kept the subwoofer.
I piled the speakers on top of my bookcase, wired them to the subwoofer/amp, and wired that to the Dell DJ. The result worked very well. $0. New and much better sound.
The computer will get wired to my downstairs stereo, which is the best sound system in the house. I would move that upstairs, but I don't have the space on my bookshelves for a system that size. One day I may have a room to listen to music again. By then, all my equipment will be osbolete.
I hope to pick up a reasonably good $50 set of over-ear headphones for those time when I want headphones. They won't get used too much, but they also won't cost that much. Little hands find things.