Photoshop: Transforming selections results in antialiasing

  • 1
  • Problem
  • Updated 7 years ago
  • Not a Problem
  • (Edited)
I have a new install of PS on a new Lion Mac. When I use the rectangle selection tool the feathering field always has "0 px" but to actually get no feathering I must manually delete the space to get "0px". Every time I make a new selection the space is back. Looks like a real bug to me. It was making me crazy for a while till I discovered the space is incorrect.
Photo of Big John

Big John

  • 12 Posts
  • 0 Reply Likes
  • disturbed

Posted 7 years ago

  • 1
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 812 Reply Likes
Have you installed all the updates for Photoshop?
Photo of Big John

Big John

  • 12 Posts
  • 0 Reply Likes
I just did and the space is still there.
Photo of Big John

Big John

  • 12 Posts
  • 0 Reply Likes
Here's a screenshot of the issue:

Photo of Jeffrey Tranberry

Jeffrey Tranberry, Sr. Product Manager, Digital Imaging

  • 14046 Posts
  • 1734 Reply Likes
Hi John,

I'm not following you. If I have "0 px" in the feather field, I don't have a feather applied to a the selection I make. Are you simply upset that it reads "0 px" as opposed to "0px" - I so, that's the way it's always worked. We don't even allow "0px" to be a valid option - as soon as you commit the field "0px" it's auto-formatted to "0 px"
Photo of Big John

Big John

  • 12 Posts
  • 0 Reply Likes
When I drag a rectangle and use it to copy some pixels, when I paste them elsewhere there is a feathered edge on the resulting pasted pixels. If I remove that space it works correctly with a sharp copy and paste. Obviously the space is preventing the selection box from taking a sharp pixel copy. If this is not what's happening on your machine then I have no answer, except that this is a fresh, fully updated install on a one week old imac lion box and it's happening to me.
Photo of Jeffrey Tranberry

Jeffrey Tranberry, Sr. Product Manager, Digital Imaging

  • 13996 Posts
  • 1685 Reply Likes
Strange. I can't even get the field to accept "0px" as a valid value. In the document you are pasting into, are you viewing the results at 100% or some other zoom factor? Can you post a screen shot of the entire screen of that result?
Photo of Big John

Big John

  • 12 Posts
  • 0 Reply Likes
I did more testing, and it appears the bug only happens after I use "Transform selection" to resize the selection box. If you create an image and expand the canvas on one side to make some transparency, then drag a selection box the full height, and resize the selection to 10px wide, then use cmnd+opt+shift+right arrow repeatedly, the result should be a continuous extension of the color across the transparent area. This does happen without resizing, but after resizing there seems to be some feathering even tho the field says 0 px.

I use this technique heavily in my web work, and it worked fine until I discovered the selection resize trick and tried it. Suddenly there were flaws appearing due to feathering.

At this point I'm not sure I was right about the space in the field, but the bug is definitely still there and I can reproduce it easily.
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 812 Reply Likes
Transforming the selection has no relation to the feathering option used when creating the selection. But transforming can cause resampling, and that can create antialiased pixels on the edge of the selection.

Yeah, I think you mis-attributed the cause of the antialiased edges to the feathering field.
Photo of Big John

Big John

  • 12 Posts
  • 0 Reply Likes
So, is this a bug or a feature?
Photo of Big John

Big John

  • 12 Posts
  • 0 Reply Likes
...and how do I prevent it from happening?
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 812 Reply Likes
Transforming selections resulting in antialiased edges: that's just how the math works, not a bug.
You could set your resampling to nearest neighbor to reduce it. Or just draw another selection.
Photo of Big John

Big John

  • 12 Posts
  • 0 Reply Likes
So if I draw a selection and it's not quite the right size, I really can't resize it without acquiring anti-aliasing? If I need sharp selections (and I almost always do) then it's the same as not having selection resizing at all. Seems pretty useless to me. But you're right, I can always repeatedly drag a new box and try to get it right eventually, the way I've had to in the past with other editors.

But it sure would be nice to just drag once and then size it to my needs without some side-effect screwing it all up. That's what I thought PS was giving me, but it wasn't. Oh well.
Photo of Big John

Big John

  • 12 Posts
  • 0 Reply Likes
IMO, it would be a useful option to specify that resizing a selection makes a NEW selection mask, rather than interpolating the existing one. Then, the few who want to resize some complex selection could do it, and the rest of us could resize our simple rectangles cleanly.
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 812 Reply Likes
That isn't easy, because most selections aren't just rectangular areas.
Photo of Big John

Big John

  • 12 Posts
  • 0 Reply Likes
So have a code fork that checks to see if it's a simple rectangle without feathering or anti-aliasing, and then create a new mask based on the resize value.
Photo of Big John

Big John

  • 12 Posts
  • 0 Reply Likes
Selecting a simple rectangle is a very common operation, so resizing it should not by default introduce any fuzzyness in the resulting selection. The fact that it does so and is apparently the intended result, is puzzling to me.
Photo of christoph pfaffenbichler

christoph pfaffenbichler, Champion

  • 1208 Posts
  • 166 Reply Likes
A selection can basically be compared to a channel, if you transform that interpolation has to be applied – does not seem puzzling to me when we work in a pixel oriented program.
But like Mr.Cox pointed out you could change the Image Interpolation preference.

Otherwise if a Selection does have a known »right size« you could just set the Rectangular Marquee Tool to those measurements to begin with.
Or use Quick Mask Mode and Image > Adjustments > Posterize to clean up an interpolated Selection.
Photo of Big John

Big John

  • 12 Posts
  • 0 Reply Likes
Okay, you've convinced me. Sorry for bothering you before getting to know the app better. And thanks for all the undeserved attention, it's nice to know there's people like you guys in back of the product if things don't go smooth.