Lightroom: Automatic image tagging

  • 21
  • Idea
  • Updated 4 months ago
  • (Edited)
[Update: See this reply for a list of third-party plugins providing automatic image tagging: https://feedback.photoshop.com/photoshop_family/topics/automatic-image-tagging-with-google-cloud-vis... -- John Ellis]

Hi all,
who of you likes organising and tagging pictures in your image catalogue? I don't. I'd love to keep my focus on taking photos. When I read about the Google Cloud Vision API, I thought this cloud be the solution for my problem that I don't like organising, but love having things organised. Cloud Vision is a service that takes an image and returns a set of keywords, places, brancs, etc. it detected. Why not let Google do the work?
So I started learning Lua and Lightroom development to create such a Plugin by myself (hold on, it is nothing there yet).
When I started thinking about the actual plug-in, the question came up what would other Lightroom users expect it to look like? And what other features do you expect? So I forwarding these questions to you. What do you think of it? Do you think this plug-in is helpful to you? What are your expectations?
Photo of Hendrik M Halkow

Hendrik M Halkow

  • 6 Posts
  • 2 Reply Likes

Posted 2 years ago

  • 21
Photo of Victoria Bampton - Lightroom Queen

Victoria Bampton - Lightroom Queen, Champion

  • 4296 Posts
  • 1563 Reply Likes
Have you tried the new image analysis search feature in Lightroom Web yet? It runs along similar ideas, except the keywords aren't actually visible.
Photo of Hendrik M Halkow

Hendrik M Halkow

  • 6 Posts
  • 2 Reply Likes
Thank you Victoria for sharing this. I didn't know about this. looks like Adobe is working on this. What I miss is that the keywords are suggested based on the content, which is what you mentioned, too.
And of course, the integration of this feature into Lightroom: I want all of my images to be indexed by this feature (besides, I also want all of my images to be available in Lightroom Web and Mobile).

@Adobe: What are your plans about this feature? Could you please add the automatic keyword suggestion to it?
Photo of John R. Ellis

John R. Ellis, Champion

  • 3690 Posts
  • 963 Reply Likes
"So I started learning Lua and Lightroom development to create such a Plugin by myself (hold on, it is nothing there yet)."

That sounds like an excellent idea! Please keep us apprised of your progress.

I think it's likely that over time, the Google service will be higher quality and more capable than the Adobe service -- Google has more technical resources and a bigger market to support their efforts, as well as access to orders of magnitude more training data for its machine-learning algorithms.  Just compare the quality of face recognition provided by Google and Facebook with that provided by LR (technology Adobe licensed from a third party).

The Lightroom SDK forum is a good place to ask questions about implementing plugins.  
Photo of Kostadin Luchansky

Kostadin Luchansky

  • 22 Posts
  • 5 Reply Likes
@Hendrik M Halkow, sounds amazing! Both thumbs up! I've been hoping for and looking forward for such a solution for years. That sounds like a really excellent idea! Please keep us posted of your progress. I've subscribed to this post. Have a nice day :)
Photo of yann.jouanique

yann.jouanique

  • 1 Post
  • 0 Reply Likes
Hello all!

I just came across this thread and thought I'd let you know I did come up with a plugin that does just that.
Lightroom CC users can grab it from https://creative.adobe.com/addons/products/16333
Sorry though, it's commercial...
Photo of John R. Ellis

John R. Ellis, Champion

  • 3690 Posts
  • 963 Reply Likes
I've just released the Any Vision plugin, which also uses Google Cloud Vision to automatically tag your photos with labels, landmarks, logos, face expressions, and dominant colors and extract embedded text (OCR).

In addition to Any Vision, there are three other products with significantly different feature sets and pricing: Cloud Tagger, Excire, and Lightroom Web search.
Photo of awespark

awespark

  • 5 Posts
  • 0 Reply Likes
This reply was created from a merged topic originally titled Lightroom: Auto keywording similar to google photos and apple photos.

After much deliberation I chose Lightroom to be my primary photo management software over Google Photos and Apple Photos because among other things I am a photographer / videographer and love that it's keywords are ITCP level metadata (instead of some proprietary system).

But one thing I really fell in love with when testing google / apple photos was their auto keywording features.  Being able to have all "cars" "landscapes" "animals" etc automatically keyworded makes searching and adding my own keywords exponentially faster.

I found this cool plugin http://wordroom.org/ but it only keywords 1 photo at a time when you manually tell it to.

I know lightroom is in a bit of a different ballgame than google / apple photos but I'm guessing there are a lot more people like me who use Lightroom over those options but would love to see stay competitive in the auto-keywording area as well.

Thanks!
Photo of Laith Alsulaiman

Laith Alsulaiman

  • 2 Posts
  • 0 Reply Likes
This reply was created from a merged topic originally titled Automating keywording and tagging process using machine learning.

Keywording is a painful and cumbersome process especially in these days where many photographers have fat LR catalogs with hundred thousands of photos that are increasing every day with little or no time to pick, rate, keyword, and tag.

LR has automated face detection and geotagging features, and both are integrated nicely with keywording panel at the library module. However, many users may need to search for a specific photo based on some general concepts such as scene type (e.g. landscape, architecture), or more specific objects such as Eiffel tower.

At the moment, I'm aware of the experimental search feature which is available only at online version of LR but this is limited only for searching on synced collections without offering any option to keyword them.


I think it will be great if Adobe extends the artificial intelligence based search feature to desktop and mobile versions of LR  and adds the ability to suggest keywords and apply them into photos.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3690 Posts
  • 963 Reply Likes
Here's an updated list of products for automatic image tagging within Lightroom. If you know of others, please post them here.

Cloud Tagger uses Google Cloud Vision, and it is intended as an aid to careful keywording of smaller numbers of photos rather than searching large catalogs. It currently has no recurring charges.

MyKeyworder is intended as an aid to careful keywording of smaller numbers of photos rather than searching large catalogs.  It has recurring charges.

Wordroom is intended for careful keywording of individual stock photos. It's currently in beta and it's not clear what it will cost in the future.

Excire lets you search your entire catalog for specific content or for similar images. It doesn't use the cloud and it has a higher one-time upfront cost with no recurring charges.

LrTag is similar to Excire, but it has a monthly charge.

Any Vision uses Google Cloud Vision to automatically tag your photos with labels, landmarks, logos, face expressions, and dominant colors and extract embedded text (OCR). It has recurring charges.

The newly released Lightroom CC (but not LR Classic) uses similar technology, but it doesn't search your desktop catalog, just photos you've synced with Lightroom CC, Web, and Mobile.

[Updated to reflect the release of Project Nimbus as Lightroom CC.]

[Updated to add Wordroom and LrTag.]
(Edited)
Photo of Kimberly Teske Fetrow

Kimberly Teske Fetrow

  • 12 Posts
  • 0 Reply Likes
THANK YOU for this!
Photo of Frederic Rychen

Frederic Rychen

  • 2 Posts
  • 0 Reply Likes
This reply was created from a merged topic originally titled keywords generation.

Is it possible to implement photo recognition to generate keywords?

I have seen this kind of functionality on an online photo community (I don't know if Adobe already provide it to its online one). I was thinking that this functionality will be a powerful tool for non professional photographer to improve their use of a software like Lightroom that provide tools to take advantage of keywords in photos.

I use Lightroom for long time, but in order to exploit my photo database I need an continuing improved set of keywords. As I develop my way to practice photography I'm more demanding about Lightroom features that are powerful like collections... But one limitation is brain and memory (speak for myself only) because I'm facing a problem : how can I extract photos that share the same features in order to work on them and see if I can make a coherent collection.

I'm not so wise to implement a complete dictionary of keywords, and giving coherently keywords to all my photos is a task that will take too long time and probably will be pledge by inefficiencies in the keywords definition.

So will it be possible to implement IA photo recognition and to generate set of keywords on each photo of a given database, like face recognition does. It will save precious time and give the opportunity to all users to discover common elements in their photographic practice. A analysis module could also provide automatic collection themes based on the most shared keywords or can provide interface that allows the user to select a specific theme and explore differently its production.

Exploiting past photos is a growing problem and this kind of feature will be helpful and powerful if connected to other LR existing functionalities. It will help me (and probably some others) to focus on ideas and photo selection/improvement.

Thanks 
Frederic
Photo of Richard Polhill

Richard Polhill

  • 4 Posts
  • 0 Reply Likes
This reply was created from a merged topic originally titled Object/Product Recognition..

Currently, Lightroom can recognize faces and suggest People to label images. It would be really useful to be able to apply the same technology to objects or products. 

I take a lot of photographs of classic cars and it would be really helpful for Lightroom to be able to suggest specific makes/models. I'm sure that train, airplane and boat enthusiasts will have a similar use. Or marketers can use it to find appearances of their products in media.

Additionally, it can recognize specific cities by their buildings or landmarks. 

Also statues and artworks.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3676 Posts
  • 961 Reply Likes
Google's Cloud Vision service does some of this. For example, it generally recognizes and labels the presence of "car", "antique", and "classic" or "vintage" car, and it correctly identified a Volkswagen Beetle, a Ford Model A, and a Mercedes Benz 500K. But more often it fails to identify the make or model.

It also usually identifies the presence of other categories -- trains, planes, etc.   And it often identifies buildings and landmarks by their map location.  

See this post for an overview of services that provide automatic tagging in LR: https://feedback.photoshop.com/photoshop_family/topics/automatic-image-tagging-with-google-cloud-vis...
Photo of Kimberly Teske Fetrow

Kimberly Teske Fetrow

  • 12 Posts
  • 0 Reply Likes
This reply was created from a merged topic originally titled Sensei for Lightroom Classic.

Can you please give us professionals that use Lightroom Classic Sensei too? I hate it when you give all the new tools to the shiny new penny and don't continue to do so with the older "classic" versions. This is alarmingly similar to my experience with purchasing Lightroom 6 only to discover it didn't have all the features of the CC version...so I did switch.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3676 Posts
  • 961 Reply Likes
See this previous post for plugins that provide automatic tagging and search: https://feedback.photoshop.com/photoshop_family/topics/automatic-image-tagging-with-google-cloud-vis...
Photo of John R. Ellis

John R. Ellis, Champion

  • 3690 Posts
  • 963 Reply Likes
In a merged topic, Victoria Bampton Champion wrote, "Sensei is all done in the cloud, so you'd have to get your photos up there. Since you have Classic though, you can sync photos to the cloud (it just sends small smart previews) so that Sensei searches them, and you can use the Sensei search on the mobile apps or web browser."
Photo of Jens Koopmann

Jens Koopmann

  • 15 Posts
  • 13 Reply Likes
I understand Sensei only works for online pictures. But why not enable search with sensei from Lightroom Classic for all synched photos? I dont want to change the application just for a search.
Photo of Nahuai Badiola

Nahuai Badiola

  • 2 Posts
  • 0 Reply Likes
This reply was created from a merged topic originally titled Add Adobe Sensei capabilities to Lightroom Classic CC.

In my opinion, Adobe Sensei is one of the best new features and I'd love to use it on my  Lightroom Classic CC library.

Is there any technical limitations not to have it on this version? 

Thanks!
Photo of Thomas Kaester

Thomas Kaester

  • 2 Posts
  • 2 Reply Likes
If you wanne benfit from local image analysis and similarity search for lightroom, check www.excire.com

P.S.: Sorry, did not see, that John already mentioned it. Anyway, you really should test it ;-)
(Edited)
Photo of PhotoCitizen

PhotoCitizen

  • 6 Posts
  • 1 Reply Like
This reply was created from a merged topic originally titled Lightroom Classic: Sensei Search.

Adobe's artificial intelligence (Sensei) search works surprisingly well in their iOS Lightroom app. When will Sensei search be incorporated into the desktop Lightroom Classic?
Photo of Victoria Bampton - Lightroom Queen

Victoria Bampton - Lightroom Queen, Champion

  • 4296 Posts
  • 1563 Reply Likes
This reply was created from a merged topic originally titled Lightroom Classic: Sensei Search.

It's unlikely to happen, because it's a cloud feature and they've said that Classic will be focused on desktop features rather than cloud ones. Of course you can sync photos to the cloud from Classic and use the web interface on your desktop to search for photos. Just go to https://lightroom.adobe.com

Note: This conversation was created from a reply on: Lightroom Classic: Sensei Search.
Photo of Antoine Hlmn

Antoine Hlmn

  • 175 Posts
  • 36 Reply Likes
I think this approach is really sad. Many photographers have a workflow that is difficulty compatible with the new Lightroom version: lack of fundamental features, need for constant connection with a high bandwidth and maybe the most important, the need of control on their images. Retaining features for their « cloud » apps from the desktop app for commercial reasons might be extremely frustrating for photographers.
Photo of Antoine Hlmn

Antoine Hlmn

  • 176 Posts
  • 36 Reply Likes
I’d love to see some automatic keywording tool based on Sensei (or any other AI) in LR classic. Like the author, I like to focus on taking pictures rather than organizing them. That’s why I almost never use keywords... Here’s where the software part can shine
Photo of Thomas Kaester

Thomas Kaester

  • 2 Posts
  • 2 Reply Likes
You really should test the Excire Search Plugin (www.excire.com). It allows for automatic keywording and image similarity search based on AI. Furthermore it runs completely local on your machine, no cloud service is integrated or necessary. It will be continuously extended and improved. Rigt now, it supports more than 500 classes/scenes. New classes will come this year and user feedback and suggestiona are appreciated.

Sorry for double posting, but it seems that my above comment wasn't read.

Best regards,

Thomas (Excire Project Manager)
(Edited)
Photo of Antoine Hlmn

Antoine Hlmn

  • 176 Posts
  • 36 Reply Likes
Thanks Thomas
Photo of Jason Walls

Jason Walls

  • 1 Post
  • 0 Reply Likes
I've read the previous replies, but I'm not sure that my situation fits any of the solutions... I'm not looking for general keywords such as car and racetrack, I'm looking for the driver's name after linking that tag to a number style or photo of his car.  Each night will have a dozen+ photos of 80+ different drivers, and there will be 80+ races per year.  Below is my original request on another forum that lead me here:

This may or may not be something that will be useful to the majority, but... looking at the "facial recognition" features gave me an idea.  As a motorsports photographer, it would save an incredible amount of time if we could get lightroom to "recognize" a particular number style or feature on a car and label it with the driver's name the way that it recognizes a face and labels it with the person's name.  When shooting 1000+ shots per night over 4-6 different divisions, it could take hours to label each car with the driver's name that would unlock the ability for customers to search for their particular car when looking to purchase photos.  IF Lightroom could do that work for us, it would be a game changer, and IMO it doesn't seem like that much of a stretch from the "facial recognition" ability.   Thanks for your consideration.