I may have discovered what was causing this. I use Ghostery to block a lot of ads and things of that nature, a few of which I unblocked for testing a website. I blocked these again and the box seems to be gone for good... hopefully.