If it can help, I first fixed this issue by logging out, clearing cookies, cache and everything that I could find.
Issue came back quite fast, and I noticed that some chat are still considered as unread if you do not click on the textbox of it.
So if I see I have the icon even if I have no new conversations, I click in the textbox of all recent conversation, or even write in it to be sure, and it works.