Add face recognition algorithms similar to what Aperture has. Perhaps include something along the lines of people tagging so that people aren't just key words in a photo.
If Adobe decides to add face recognition, I hope they do a better job than they did with Photoshop Elements, which had severe problems with its recognition.
I'm all for this IF sufficient Dev resources are committed to to it's being implemented well. In other words, if it's going to be half-assed, don't bother. There's plenty of other things to tweak and improve closer to LR's core functions.
First a face detection would be OK
-> I mean just detect faces with out saying who is there can help a lot.
Then add face recognition
But the most important thing is to be able to tag an image region (like a face for instance)
1. Start to make it general : ability to associate a region (saved in xmp) with a Keyword (normal kind of keyword we have already)
2. then these Keywords can be used for face taging
3. then in the database some of these keywords can be associated to face characteristic allowing an algorithm to make face recognition.
Finally the most important in face recognition is not how well the algorithm performs, but how well results are presented so that you can easily correct any errors or validate good guess.
Maybe we can group faces by similarities (with a given threshold) like Picasa seem to do.
A same person may then appear in different such group (because the person got older, or ...) and a same keyword (person keyword for instance) would then be associated with 2 or more face characteristic. this way we can eleminate the problems of photoshop element who get worse at guessing people faces when you have tagged too many faces of the same person (old faces pollute the ability to detect youger faces and vice versa)
regards
Eric
Presentation is very important and something Picasa has got right (even over iPhoto). The annoying thing about Picasa Face recognition is that you can't tweak anything, or even tell if it's still running... until you suddenly get new results (try the Grouping Faces feature)
LR integration with Keywords is crucial, and possibly a way to integrate EXISTING Keywords to help with the detection would be great.
I typically take have a shooting spree where I have taken dozens of photos with the same people, so presenting that / building an algorithm should prefer that I would hope.
You're right and I've confused the issue... For EXISTING metadata (e.g. IPTC:Person Shown, Keywords) of photos I'd like the algorithm/interface to give great weight to those edits I've already made for those fields. When "tagging" a Face on the photo, I think it should be stored in IPTC:Person Shown field.
Understood, though I suspect I also misread you! Do you know if Picasa uses existing IPTC as part of the algorithm? I can see why it would make sense (and fit LR's style), but I hadn't seen an example of software doing this.
no they don't. At least in the version I use,I did not checked if it got updated.
By the way the anoying thing with PICASA and IPTC: person shown is that there is no hierarchy
-> whenyou have 1000 of names it becomes a real pain to find the one you need for tagging (though PICASA lets you type it and auto complete, which helps a lot)
"By the way the anoying thing with PICASA and IPTC: person shown is that there is no hierarchy"
Lightroom could still add hierarchy without it being IPTC-supported (as it does with collections for instance). The trouble is, family structures are often messy, and can be fluid over time.
I would hate if the face information were not stored in the XMP data.
That is already one of the biggest issue with PICASA, and is why I do not use it anymore.
Note that Microsoft already created a XMP structure to store face information. (It is just a little bit too much microsoft centric since one of the record is related to a MSN account, but you do not need to fill it)
By the way it would be nice if lightroom could read these tags (and write them) as well.
PS: As for family structure I have no problem with them so far, keeping it simple, with just the family name.
But maybe we could invent a XMP tag that would allow for complex tree such as family tree ...
Remember that XMP is only the language or syntax used. There's an IPTC standard field - Person Shown. Build out from there - and the X stands for (user) extensible.
The Metadata Working Group, of which Adobe and Microsoft are members, has already defined a standard method for representing regions and metadata associated with those regions. In particular, keywords, map locations, and persons (as already defined by the standards) can be associated with regions. This all gets stored in XMP metadata, of course.
See "Guidelines for Handling Image Metadata, Version 2", section 5.9 for more details.
that is exactly what we need!
I would like then LR to offer us the ability to tag region and link them with Keywords(whether it is a face region or not),( and Person In Photo IPTC/XMP tags when we specify it is a face region).
That would be the firrst step towards face recognition.
Second step would be to extract faces (just face recognition - no person identification) , so that it would improve manual face tagging.
Third step would add the face recognition.
When I speak in term of step I mean version of lightroom, as I understand that developping all that at once may take time and may not be free of bugs.
So we can take time to get there.
I would hope though that if Adobe do add this feature (please do) that they do not get tempted to incorporate this into social networking sites... I do not like the idea that people can tag your picture on line without your permission. let alone be able to publish your photo online.....
So Apple has it, and it looks like Google will be using it. It sure would be easier to keyword photos with face recognition, assuming it worked well. Instead of viewing it as a gimmick for the masses, as opposed to serious photographers, try viewing it as a workflow automation improvement that will become a requirement over time.
Fast forward a few years. Imagine LR creates an XML file, or mini DB of attributes attached to each photo, derived from things like image processing, GPS coordinates and date/time. Potential keywords could be presented for easy selection like: Dad, Julie, John Warnock, Christmas, Wedding, River, Mountains, Horse, Fog, Dawn, Yosemite, Motorcycle, Woman, Rain, Tornado, Fire, Red, Fruit ... You get the idea.
This kind of stuff will be prevalent for still and video captures in a few years, by all the computer-based applications (desktop and laptop and maybe tablet).
I decided to take a second look at face recognition after my disastrous first test with PS Elements 8. I tried out Windows Live Photo Gallery's version.
I gave it a folder of about a thousand images, pretty much all of them of people (my kids, mostly). It took about 10 minutes, and came up with a bunch of faces for me to identify. That took a few minutes. Done right?
I then went through image-by-image and had a look. It missed about 75% of the people in the images, not identifying them as faces at all. This seemed to be caused by the fact that the full face wasn't in the image, either in profile, partially obstructed, wearing sun glasses, or something else.
I then proceeded to manually do what it hadn't done. It took a little over two hours.
Since I had to touch each image manually anyway, I estimate that the face recognition technology only saved me a few minutes over doing the whole thing myself manually.
I think this is a difficult problem and I don't see a way to overcome it.
From my experience : picasa is doing a much better job you Should try it.
Saved me several weeeks of work. But it also has difficulties to detect profile...
"I think this is a difficult problem and I don't see a way to overcome it." - this is the crux of innovation. Adobe needs to find/acquire/dedicate the resources to make this possible.
To Eric's comment: Picasa is a good alternative for now, but the workflow is a large burden. I used Picasa to index photos, recognize faces and then added standard Keywords for those images. In LR, I had to re-read metadata from this files and then assign the "Person Shown" in IPTC metadata. Ridiculous
"From my experience : picasa is doing a much better job you Should try it. "
Okay, I tried Picasa on the same images.
It did much, much worse than Windows Live Photo Gallery, it was slower, harder to use, and very buggy. It missed a lot more really obvious faces it had no reason to miss plus it missed all the one WLPG missed too.
Took half an hour to scan the faces versus ten minutes, and would have taken at least 10 hours to manually enter them all versus 2.5 hours. I gave up and uninstalled it.
Interesting. I just gave Elements 9 a try. It did slightly better than Windows Live Photo Gallery, missing fewer of the ones wearing sunglasses. It still missed a lot of the profile and partial face shots, but it also ran by-far the fastest at about 1.5 minutes for the same set of images, and it only took about half an hour to manually add the ones it missed. Interesting, especially given how badly Elements 8 did on a completely different set of images.
My PSE9 catalog contains over 99,000 images. A large number of these photos are of youth soccer players from a variety of teams, and many photos contain multiple faces (e.g. team photos). As many players are unknown to me, it is very helpful to tag a face with a player's name, and to find all photos (e.g. action photos) containing that player.
Yes, at first PSE9's face recognition appears to work substantially better than PSE8. However, my experience is that it simply stops working after some number of images are processed. If you re-run Find->Find People For Tagging, nothing happens. Worse, other features are impacted: you cannot use F11 for full screen (must exit and restart app), and you must re-load thumbnails for many images (right click->Update Thumbnail, may or may not work).
Note that part of the job could be done outside lightroom.
more and more camera have this feature, to adjust focus on the faces, even dslr (in live view mode). some of them store this info in metadata. So if lightroom could read that, this could help save computing time?
It would be an excellent tool. At this point I use Picasa to detect and tag people but unfortunately the work around is not good for DNGs. I work events, set photography for films and publicity and face detection would save me lots of time. I even considered to migrate to Apple Aperture but the face detection in Picasa was better and the tools easier to use.
I have not used facial recognition tools and found the above conversation fascinating. My only reservations are that the technology does not appear to be reliable at this point and that LR still has basic stuff to get right, such as the keywording and performance rendering previews.
This seems to me to be an ideal function to be addressed by a third party addin.
Responses
kylet_2319110
3 Messages
•
126 Points
10 years ago
This reply was created from a merged topic originally titled
Face recognition & People Tagging..
0
0
John_R_Ellis
Champion
•
5.5K Messages
•
97.4K Points
10 years ago
1
0
bend_2312379
24 Messages
•
4.4K Points
10 years ago
1
eric_bayard
136 Messages
•
4.7K Points
10 years ago
-> I mean just detect faces with out saying who is there can help a lot.
Then add face recognition
But the most important thing is to be able to tag an image region (like a face for instance)
1. Start to make it general : ability to associate a region (saved in xmp) with a Keyword (normal kind of keyword we have already)
2. then these Keywords can be used for face taging
3. then in the database some of these keywords can be associated to face characteristic allowing an algorithm to make face recognition.
Finally the most important in face recognition is not how well the algorithm performs, but how well results are presented so that you can easily correct any errors or validate good guess.
Maybe we can group faces by similarities (with a given threshold) like Picasa seem to do.
A same person may then appear in different such group (because the person got older, or ...) and a same keyword (person keyword for instance) would then be associated with 2 or more face characteristic. this way we can eleminate the problems of photoshop element who get worse at guessing people faces when you have tagged too many faces of the same person (old faces pollute the ability to detect youger faces and vice versa)
regards
Eric
0
brettveenstra
3 Messages
•
128 Points
10 years ago
LR integration with Keywords is crucial, and possibly a way to integrate EXISTING Keywords to help with the detection would be great.
I typically take have a shooting spree where I have taken dozens of photos with the same people, so presenting that / building an algorithm should prefer that I would hope.
8
eric_bayard
136 Messages
•
4.7K Points
10 years ago
ability to tag a picture region
0
0
John_R_Ellis
Champion
•
5.5K Messages
•
97.4K Points
10 years ago
See "Guidelines for Handling Image Metadata, Version 2", section 5.9 for more details.
1
steve_mcginty
7 Messages
•
286 Points
10 years ago
I would hope though that if Adobe do add this feature (please do) that they do not get tempted to incorporate this into social networking sites... I do not like the idea that people can tag your picture on line without your permission. let alone be able to publish your photo online.....
0
steve_mcginty
7 Messages
•
286 Points
10 years ago
This reply was created from a merged topic originally titled
Lightroom: add face recognition.
0
0
dan_zemke
5 Messages
•
454 Points
9 years ago
http://blogs.wsj.com/digits/2011/07/2...
So Apple has it, and it looks like Google will be using it. It sure would be easier to keyword photos with face recognition, assuming it worked well. Instead of viewing it as a gimmick for the masses, as opposed to serious photographers, try viewing it as a workflow automation improvement that will become a requirement over time.
Fast forward a few years. Imagine LR creates an XML file, or mini DB of attributes attached to each photo, derived from things like image processing, GPS coordinates and date/time. Potential keywords could be presented for easy selection like: Dad, Julie, John Warnock, Christmas, Wedding, River, Mountains, Horse, Fog, Dawn, Yosemite, Motorcycle, Woman, Rain, Tornado, Fire, Red, Fruit ... You get the idea.
This kind of stuff will be prevalent for still and video captures in a few years, by all the computer-based applications (desktop and laptop and maybe tablet).
So Adobe - lead or follow?
Dan
1
yr4uku
4 Messages
•
106 Points
9 years ago
to test for large photo facial expressions
0
0
lee_jay_fingersh
946 Messages
•
13.8K Points
9 years ago
I gave it a folder of about a thousand images, pretty much all of them of people (my kids, mostly). It took about 10 minutes, and came up with a bunch of faces for me to identify. That took a few minutes. Done right?
I then went through image-by-image and had a look. It missed about 75% of the people in the images, not identifying them as faces at all. This seemed to be caused by the fact that the full face wasn't in the image, either in profile, partially obstructed, wearing sun glasses, or something else.
I then proceeded to manually do what it hadn't done. It took a little over two hours.
Since I had to touch each image manually anyway, I estimate that the face recognition technology only saved me a few minutes over doing the whole thing myself manually.
I think this is a difficult problem and I don't see a way to overcome it.
6
0
eric_bayard
136 Messages
•
4.7K Points
9 years ago
more and more camera have this feature, to adjust focus on the faces, even dslr (in live view mode). some of them store this info in metadata. So if lightroom could read that, this could help save computing time?
0
0
vlad_plaiasu
9 Messages
•
460 Points
9 years ago
Please implement it.
0
rory_hill
248 Messages
•
4.1K Points
9 years ago
This seems to me to be an ideal function to be addressed by a third party addin.
0
0