If you are Steal'ing common from within filemanager, would this already work as you need?
As long as common doesn't Steal filemanager, when you build common, it wouldn't rebuild filemanager. But when you build filemanager, it will build and include common within filemanager's resulting build.