Is it wrong for you to not want to be around someone you have already forgiven for causing you harm in some way?

I have had several people in my life cause me harm. I forgave these people for what they did. They did nothing illegal or violent, but they were jerks for doing what they did. But I don't want to be around them anymore. Does this go against what forgiveness is or do I still need to be around these people?