Hi @johnsmith101 and welcome!
You are correct. Anybody that learns about a pull payment pointer can potentially pull from it. I have been thinking about adding an auth layer to any payment (because you may want to know where your money is coming from, too) but especially to pull payments. @adrianhopebailie, @matdehaast, and @don have been working on delegated payments which include an auth flow. Additionally, there is a new RFC in the works on Interledger HTTP Authentication Profiles.