Is it possible to "somewhat damage" a speaker?

I'm having what I think is a bit of a speaker problem. But I'm not sure. I just received a sansui 4000 receiver today, and connected it to my turntable, an Elac Miracord 750, and then connected two speakers (Fisher, "bookshelf" style, looks like they're from the 80s--came with the MC-712 audio component system) into the whole thing. One of the speakers works fine, but the other speaker is barely making a sound. When I first turned on the receiver, I really had the volume blasting, so much that it shut down the whole speaker system and cause the Sansui to go into "protection" mode. I fixed that problem and kept the volume reasonable, but as a result that second speaker was barely making any noise. I plugged the same speaker into the MC-712 and it was doing the same. The other day, both speakers were just fine in the MC-712. Do you think I damaged that second speaker? It still technically makes the faintest sound, but in comparison to the first speaker, it's shot. Any suggestions? Just bite the bullet and get some new, decent speakers?