why does henry hazlitt say that "public works means more taxes" is a bad thing??

In Henry Hazlitt's book "economics in one lesson", the 4th chapter talks about low income housing, and how it is bad for the economy in the long run. He says that the money that is put into government subsidized housing (meaning too much of unnecessary low income housing) is tax money that the tax payers could have used for things they needed. I understand that there is money being spent on one thing rather than the other. But if it is just a diversion of funds, then why does it hurt the economy? I would think that it would make no difference. Instead of someone giving money to Maytag for a new fridge, the money is now given to the construction worker to build the low income housing, the maintenance workers when its done, and everyone else that works there. Why does it make a difference when the government gives the tax money to the construction worker to spend, rather than the money being given to the refrigerator company to spend. I would think that they would both have the exact same long term effects on the economy. I don't understand why a diversion of funds from buying something useless over something useful hurts the economy. I get that buying something useless over something useful is upsetting for the individual taxpayer. but either way, workers are getting money to spend. It's just one person getting the money instead of another. So in the long run, what is he trying to point out that is harmful to the economy. If you could please help me understand I thank you in advance. Also, I could do w/o any rude answers/remarks. I am just an uneducated person trying to understand the basics of economics.Thank you.