

That’s always a danger.
You could host the server on its own vlan and use tailscale to tunnel from a VPS to your server. This way your IP isn’t exposed to the public. You can also add crowdsec or similar on the VPS.
I do the same thing with traefik on my dirt cheap VPS.
A person with one watch always knows what time it is; a person with two watches never does.