That’s why you avoid them, and hope they’ll avoid you in return. It’s probably the least dickish thing you can do, and you can always just tell them you’ve been busy with other stuff if you do end up wanting to have contact again. Insults, offenses, betrayals, etc. are much harder to come back from.