I would expect to be able to save components from any file to the <library file>.
This would help me keep track of the created components and re-use them as I design more screens, in separate files.
Currently, I can't keep up with updating the <library file> as I work. I tried but it consumes a massive amount of time.

The extent to which I use this feature is by creating a <library file> that contains all the colours, fonts and a few basic components (inputs, selectors, buttons). As I create more and more screens with many new components, I can't keep up with adding them to the <library file>, SO instead, I just use the <library file> for the basic elements and then create many others in separate files. This means I need to remember and search in what file I created a specific component that I need once again.

I really think this is a must-have for large projects where you use many XD files. You don't know before starting the work what components you'll have. As you create more and more screens in separate files, it gets more and more difficult to create the components in the file where the library is.