So this is the way to get a square clone stamp. Rectangular are not available and I doubt Adobe will ever come with such an option. If you really need rectangular stamps you need to use one of the above mentioned ways.
After choosing Clone Stamp inside the editor, click on the small triangle at the RH side of the black brush picture which opens the Brush Preset Picker :
Now click on the small triangle next to the "Default Brushes" and a list of available brushes appears.
Square, fuzzy Stamps are available under "Drop Shadow Brushes"
Square, hard Stamps are available under "Square Brushes"
I agree with all of the other responses. PSE is an economic version of PS CC and for the price I believe it is a very powerful and good value for money product.
The more you experiment the more powerful you can make it. Playing with the brushes will give you more flexibility.