As a user who works with design systems and color palettes often, it would speed up my workflow considerably, if I could select a color swatch directly without opening the color palette.
A quick selector that openes on hover and shows the swatches could be a nice solution.
Or color shortcuts.