This is fixed in 106.0.4985.0 so far. However, there's another bug where you just click the + button to open a new tab and don't move the mouse. The mouse will be over the new tab but there won't be an X on it. That needs to be fixed still.
There's also another bug with the X not showing when hovering over a tab when a bookmarks bar folder's menu is open. Fix is coming for that one (in 107 probably).