Found the solution, simply right click the controller button in the top left and move it to sidebar. That'll make it so when you close all the tabs it closes the window.