We are connected to God by our halal intentions towards Him. We can pray 5x a day but if our intentions are to please people then it won't matter. You can ditch people, or people can ditch you, but that doesn't remove God out of your life.
Abraham's slave woman Hajar was kicked out of her home and no one to look to. Did God leave her? No.
When I said you are removing God out of you life if you ditch people, I mean if you leave your mom and dad, or stop talking to your kids or divorce your wife for no reason, you are removing the blessing of God from your life.