Photoshop improvements: Recently Used Colors

Sorry it doesn't
This is history of "foreground", "background" colors only. Also partly shape fill and stroke it depends how you change this color.
We need to quick access to all colors picked with color picker.
Shape fill, shape stroke, gradient stops colors, and a lot of layer effects like drop shadow ect.

Next problem is that I need only tiny area with recent colors, but minimal size of this panel is still to tall.

Shape Fill and Stroke colors show in the Swatches panel on my system with the Curvature Pen and Custom Shape tools, but not Gradient and Drop Shadow. It sounds like you may want to see the color history in the Color Picker as it applies to each specific tool.

I suggest thinking about this some more and adding detail as to exactly how you would like it implemented including specific changes to the Color Picker dialog panel. The more details you provide with examples the more likely Adobe will consider implementing it. This will also help others landing here to add additional suggestions.