Photoshop: Add options for softer anti-aliasing for Vector Masks/Shape Layers

  • 59
  • Idea
  • Updated 3 years ago
  • Under Consideration
  • (Edited)
*** If this is important to you, please comment below. ***

Vector Masks in Photoshop have sharper anti-aliasing than shapes created other ways. Quite often, I find that the results are too sharp. This is especially true for very small shapes, making it an issue for icon creation.

It’s interesting to note that vector Smart Objects that have been pasted from Illustrator have vastly different anti-aliasing to Shape Layers that have been pasted from Illustrator. The Smart Objects are far heavier and the anti-aliasing seems posterized.

I don’t really have a solution for this, except a suggestion that the Shape Layer/Vector Mask rendering is very close to ideal for me, but I’d prefer slightly softer anti-aliasing. I don’t know how this could be implemented while keeping legacy support. I guess there’s three ways it could be done: A global change or preference, where all documents get the new rendering (breaking legacy rendering), a per document setting or a per object/layer setting. The first breaks compatibility, the second and third add UI and file bloat.

Steps to Reproduce — Create a circular marquee selection at a smallish size, say 9x9 pixels and fill it with white. Create a pixel snapped vector circle that’s the exact same size (you may have to use the rounded rectangle tool with a large radius and Snap To Pixel turned on). Compare the results—the marquee selection bitmap layer is smoother.

Workaround — None. Only really crazy, silly stuff that I’m not usually willing to do because it removes editability.







If you'd like to see the original files, grab them here: antialiastest-597642.zip

View an animated comparison between the various methods.

*** If this is important to you, please comment below. ***
Photo of Marc Edwards

Marc Edwards

  • 130 Posts
  • 21 Reply Likes
  • happy

Posted 8 years ago

  • 59
Photo of Alex

Alex

  • 7 Posts
  • 11 Reply Likes
It would be very helpful to have an "object-related" antialiasing modes for both marquee tools and vector tools.
Photo of Marc Edwards

Marc Edwards

  • 130 Posts
  • 21 Reply Likes
Similar to text objects? That would be great. Definitely one way it could be implemented while keeping legacy support.
Photo of Alex

Alex

  • 7 Posts
  • 11 Reply Likes
Exactly as text objects. We definitely will want to change this setting in different cases. For example, for now, marquee's soft edge is irreplaceable on small objects, while "sharp-edged" smart shapes look better on retina screens. So my vote for "local" settings instead of global trigger.
Photo of Iain Dowling

Iain Dowling

  • 3 Posts
  • 4 Reply Likes
This issue has been a major source of frustration for me, particularly when I have been authoring buttons and small image resources for web sites and mobile devices. The problem is more apparent for mobile devices because the user typically holds the screen closer to their face, so sub-pixel details are more noticeable.

I am a fan of Homer's suggestion of having object-level control over the heaviness of anti-aliasing. Hopefully this gets the attention from Adobe that it deserves.
Photo of Marc Edwards

Marc Edwards

  • 130 Posts
  • 21 Reply Likes
"particularly when I have been authoring buttons and small image resources for web sites and mobile devices"

I agree. It's definitely all about smaller shapes and icons. Larger shapes don't seem to need the softer anti-aliasing.

I think there's a very strong parallel to type rendering, where complex shapes are rendered at tiny sizes, but need every pixel to be carefully considered or they lose their legibility and meaning. There's a huge diversity of type rendering methods for this very reason (and quite a few within Photoshop itself).

And with that thought in mind, there probably isn't a right or wrong answer. Photoshop's text rendering, Cleartype on Windows and Mac OS X's type rendering all have scenarios where they're preferable. This is a fairly specialised UI/web/screen design request, but a very important one as there's no workarounds. If we don't like the shape rendering, there isn't really any way to tune it, so we're stuck with the results.
Photo of Foster Brereton

Foster Brereton, Programmer

  • 42 Posts
  • 14 Reply Likes
This is a super example, with a clear-cut definition of the improvements needed. I wish all requests were more like this one.
Photo of Jeffrey Tranberry

Jeffrey Tranberry, Sr. Product Manager, Digital Imaging

  • 14551 Posts
  • 1897 Reply Likes
I agree. The better case a user makes for their idea the more likely other customers and developers on the product teams will see value in the idea.
Photo of Marc Edwards

Marc Edwards

  • 130 Posts
  • 21 Reply Likes
If there's any more information required, please let me know. If Photoshop becomes better, my job gets easier, so I'll do what I can to help.
Photo of Ric Henri

Ric Henri

  • 1 Post
  • 1 Reply Like
Thanks for, again, bringing this issue to light, millions(?) of designers around the world would thank for such improvements. As mentioned, 50% of the web/UI work these days is for mobile, as these displays resolution increases exponentially, new versions of PS should account for this anti-aliasing issues.

Designers need object level anti-aliasing control! Tired of tweaks, zooming, nudging and other tweaks for better anti-aliasing...
Photo of Octop

Octop

  • 27 Posts
  • 4 Reply Likes
This would be a great addition. However, rather than 'modes' for vectors, which are kind of vague, i'd like to see a slider to control the anti-aliasing, perhaps in the layer style dialog.
Photo of Marc Edwards

Marc Edwards

  • 130 Posts
  • 21 Reply Likes
I'd be up for numeric, slider or "modes". Anything that gives us softer antialiasing is good in my books. I guess the question is how many steps do you think we need? At a guess I'd say 3 would be good, 5 great. And maybe the best way to implement that is a slider.

Thoughts?
Photo of Alex

Alex

  • 7 Posts
  • 11 Reply Likes
Agreed. Slider or numeric control would be even better. But not in the layer style dialogue (too deep I think) but in tool options (like feather for marquee tool)
Photo of Iain Dowling

Iain Dowling

  • 3 Posts
  • 4 Reply Likes
I can see benefits to both approaches, however I am leaning towards grouping it with Layer Styles for a couple of reasons. Firstly, by putting the anti-aliasing controls in the Options Bar, I think there could be confusion over which tool you need to have currently selected (which tool would you allocate?). Secondly, I don't think it is something that you would need to change very often, so "concealing" it in the Layer Styles dialog is not necessarily a problem. Lastly, by making it a Layer Style, you have more UI space to allocate for parameters, such as opacity, sharpness, spread, maybe even a "Use Global" checkbox.
Photo of Marc Edwards

Marc Edwards

  • 130 Posts
  • 21 Reply Likes
The only issue with using the Blending Options (or anything under Layer Styles) is that they're identical for vector layers, bitmap layers, type layers and Smart Objects.

However, I agree that putting it in the Options bar would mean allocating it to a tool, which doesn't make too much sense. As far as I can see, the options bar doesn't change dependent on layer type either.

Or were you talking about it being a new Layer Style item (ie. under Blending Mode)?
Photo of Iain Dowling

Iain Dowling

  • 3 Posts
  • 4 Reply Likes
I was thinking of making it a new Layer Style item; something like "Custom Anti-aliasing", so you could maintain existing behaviour as the default (for backwards compatibility), then use the new Layer Style to indicate that you would like to exercise fine-grained control over anti-aliasing, which you could toggle on and off like any other Layer Style.

This approach, however, is problematic when it comes to Type layers, because it conflics with the existing anti-aliasing control in the Options Bar. Perhaps this new Layer Style would be disabled for Type layers?
Photo of Octop

Octop

  • 27 Posts
  • 4 Reply Likes
I would personally like to see this in Blending Options, so it can be applied to multiple layers via style copying. ideally a slider with anti-alias range (maybe 1-5 pixels as Marc suggested), as word descriptions (crisp, strong, smooth...etc) speak to me less clearly than quantizing the softness. Certain items may benefit from softer anti-alias, while others sharper.

I'm assuming that this would only work for vector objects, so if it could be applied as a layer style, you could modify more than one text layer at a time without having to edit each individually, which would save time in workflow, and I'm always up for that.

Just my 2 cents.
Photo of Marc Edwards

Marc Edwards

  • 130 Posts
  • 21 Reply Likes
Being part of the Layer Styles and therefore easily copied is definitely a huge positive. I have command-shift-C and command-shift-V assigned to copying and pasting Layer Styles and I could see this as a huge plus if the vector anti-aliasing softness was included.
Photo of Octop

Octop

  • 27 Posts
  • 4 Reply Likes
Nice & handy key combos. It would be awesome if that showed up in a future version of Photoshop. Workflow speed is very important.
Photo of Marc Edwards

Marc Edwards

  • 130 Posts
  • 21 Reply Likes
Command-shift-X is great for clear layer style, too.
Photo of Marc Edwards

Marc Edwards

  • 130 Posts
  • 21 Reply Likes
The more I think about this, the more I'd actually like vector shapes to be able to run the full gamut from completely aliased to anti-aliasing similar to the elliptical marquee tool. It'd mean those building retro pixel art could keep everything as vector paths. How awesome would that be?

An example circle, from completely aliased to elliptical marquee tool anti-aliasing. (Doing this with the current tools is possible, but more time consuming and impossible while maintaining vector objects.)



And where we've been talking about placing the option:

Photo of mathias

mathias

  • 159 Posts
  • 17 Reply Likes
Brilliant idea!
Photo of Edward

Edward

  • 92 Posts
  • 23 Reply Likes
Would it not be more appropriate in the masks panel? You can already control feather, just needs another slider for anti-alias.

Photo of Mama Shan Canfield

Mama Shan Canfield

  • 50 Posts
  • 5 Reply Likes
You took the words right out of my mouth! It was my intention to mention to put the options in the Masks Panel....but you've covered that! Best idea for placement of vector aliasing options, in my opinion, too!
Photo of Marc Edwards

Marc Edwards

  • 130 Posts
  • 21 Reply Likes
Didn't know about that! Thanks. Yeah, maybe that's a better place for it. I guess this also demonstrates how spread out the options are for vector masks.
Photo of Matt Stewart

Matt Stewart

  • 1 Post
  • 1 Reply Like
I signed up to Photoshop.com just because of this topic. I'm getting so frustrated because I've been making icons for mobile phones in Illustrator, then bringing them into Photoshop to resize. The anti-aliasing is destroying the graphics. I have to go in with a 1 pixel pencil and fix up every single one.
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 814 Reply Likes
You might want to provide more information about that - because it sounds like you're describing the reverse of what others are asking for here...
Photo of Marc Edwards

Marc Edwards

  • 130 Posts
  • 21 Reply Likes
Matt, I'd love some more info as well. Maybe there's some interim fixes already available, or maybe it's a different type of issue?
Photo of Salman Abbas

Salman Abbas

  • 5 Posts
  • 2 Reply Likes
Any update on this?
Photo of mathias

mathias

  • 190 Posts
  • 39 Reply Likes
A global anti-aliasing switch, just like Illustrator, for Photoshop, would be great!

This way, using a combination of Nearest Neighbor interpo and having all vector AA turned off would provide low-fi workers a great benefit.

Really though, looking at the OP's post - that banded, or "posterized" anti-aliasing should simply NEVER happen. But it's been this way for as long as I can remember.

I wonder if PS's dev's are aware that Photoshop has a reputation for handling AA very poorly in many circumstances. It's a shame. Wish it would improve.
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 823 Reply Likes
The antialiasing of vector shapes and vector files already improved greatly in Photoshop CS6.

And the complaints we've seen about bad AA in Photoshop either have a mistake in their understanding, or are referring to things already fixed.
Photo of Michael Phillips

Michael Phillips

  • 3 Posts
  • 0 Reply Likes
This proposed feature is over a year old? And it still hasn't been implemented? It's totally a no-brainer.

No, the antialiasing is not fixed, although it may be better than previous versions.

I created some original fonts in photoshop, using vector shapes at 300ppi. Even the large letters (36pt or so) printed out with jagged edges, with no option to soften the edges. I had to finally export all vector shapes into Illustrator and print from there, but I shouldn't have had to.
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 814 Reply Likes
Printing is a completely different issue.
You may get resampling depending on the actual resolution of the printer. And most printers only have black and white (or CMY vs White) "pixels" to work with.

Antialiasing can actually cause more problems on many printers (because the printer tries to render the gray values, and can't).
Photo of Michael Phillips

Michael Phillips

  • 3 Posts
  • 0 Reply Likes
Thanks for the explanation. I was using a Brother color laser set to monochrome at 2400 dpi. Surely the halftoning would take care of the pixel displays of gray?

And it would still be very helpful to have the antialiasing just like with type, in a drop-down menu.
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 814 Reply Likes
Actually, the halftoning could make your shapes worse. At high res, you render without antialiasing.
Photo of Michael Phillips

Michael Phillips

  • 3 Posts
  • 0 Reply Likes
OK, so what can be done to make vector shapes print with high resolution, without jagged edges? Or is Illustrator really the only viable option then?

Type doesn't print with jagged edges. Why should vector shapes?
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 814 Reply Likes
First, find out the actual resolution of the printer.
Then render at that resolution, without antialiasing.
Then try with antialiasing and compare.
Photo of Nefaur Khandker

Nefaur Khandker

  • 3 Posts
  • 0 Reply Likes
This is causing issues for our team as well. Our team has a workflow where we design icons for our mobile app in Illustrator, and then place a linked version of that file in Photoshop, where we use Adobe Generator to export all of our assets from this PSD.

The results in Illustrator are pixel-aligned, whereas the results in Photoshop are not. See the attached example.



These results are unacceptable for us because we want to export pixel-aligned assets. Our current options are to either use Illustrator to export all of our assets (which does not have Adobe Generator support) or to rebuild all of our icons in Photoshop using shape layers.
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 813 Reply Likes
Yes, Illustrator antialiasing is less precise than Photoshop's and can make things appear to be pixel aligned when they are not. Also, Illustrator converts it's data on export, and might be losing precision in that conversion. We don't know exactly what the cause is - we just know what when we look at the Illustrator generated files, things land on pixel coordinates like 1.91 and 2.07, which result in antialiasing because they are not on pixel boundaries.
Photo of Jon Fuller

Jon Fuller

  • 106 Posts
  • 13 Reply Likes
Tell them to fix it. Adobe apps need to work together better...
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 813 Reply Likes
We've tried.
Photo of Nefaur Khandker

Nefaur Khandker

  • 3 Posts
  • 0 Reply Likes
Thanks for the info, Chris! That helps explain the behavior at least.
Photo of Marc Edwards

Marc Edwards

  • 130 Posts
  • 21 Reply Likes
The data from Ai looks good to me, unless I'm missing something.

Left = Ai canvas, middle = Apple’s Clipboard Viewer app, right = Ai pasted Smart Object in Ps.

Photo of Jon Fuller

Jon Fuller

  • 106 Posts
  • 13 Reply Likes
Possibly related, but I'm not a big fan of the excess sharpness that is applied to scaled-down smart objects -- compared to if they were just raster layers and then scaled.
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 814 Reply Likes
Not related at all.
If you scale down a raster smart object, it will have some sharpening applied.
If you scale down a vector smart object, it just gets re-rasterized at the new size.
Photo of Rodolfo Rubens

Rodolfo Rubens

  • 1 Post
  • 0 Reply Likes
Also, don't forget to give us ability to completely disable anti alias to make it look like they were drew using the pencil tool, as show on the Marc Edwards picture, when drawing perfect 45o lines some times it's much better to not have anti-aliasing!