Alright, found an extension in the chrome web store.
https://chrome.google.com/webstore/detail/downloads-overwrite-exist/fkomnceojfhfkgjgcijfahmgeljomcfk
When download starts, it will overwrite the existing file without confirmation. Not perfect, but I think I can live with it.