Why are people complaining about "tax evasion"?

Countries are allowed to set whichever tax rate they want. If Cayman Islands or Monaco don't want to tax their citizens and businesses what is problem? Also, I'm pretty sure people should be allowed to register themselves and their businesses wherever they'd like. Just like you can choose to base your business in the U.S. or France, I can choose to base my business in the Cayman Islands.