All we would have to do is figure out a way to shoo them off. I heard a few conversations about something called an "anchor" which lets them stay in this universe relatively easily, so threatening those would probably be an effective- possibly the only effective measure, since we've clearly seen that shooting them in the face doesn't do much of anything.

Unfortunately I have no idea what either of these "anchors" are or where to find them, so that idea is essentially useless unless you happen to know of them.