When using the Select and Mask tool I'm not getting theproperties dialog box to open that has the Edge detection, smart radius, globalrefinements, etc. The only thing I see in the properties box is the Pixal LayerProperties.
You have to have a mask actively selected in the Layers panel if you want to use the Properties panel to access the Select and Mask dialog. It won't work if you're trying to enter the S&M dialog on an image layer without a mask. If all you have is a selection on an image layer and a selection tool active, you can use the Options bar to enter Select and Mask.
Maybe this helps? If not, I agree with Hannah that we need to see a screenshot of your active layer and the Properties dialog.