Uploaded image for project: 'PUBLIC - Liferay Portal Community Edition'
  1. PUBLIC - Liferay Portal Community Edition
  2. LPS-81878

As a marketing manager I want to automate images tagging with recognition capabilities so I can improve image discovery and categorization

    Details

    • Type: Story
    • Status: Closed
    • Priority: Major
    • Resolution: Completed
    • Affects Version/s: None
    • Fix Version/s: Master
    • Component/s: DM, DM > Auto Tagging
    • Labels:
    • Sprint:
      August_Collab, 7.2 - S1, September, 7.2 - S2 - PM Review, Col | Sprint 4 | Nov19 - Nov30, Col | Sprint 5 | Dec3 - Dec14, Col | Sprint 6 | Dec17 - Jan18, Col | Sprint 9 | Feb18 - Mar5, Col | Sprint10 | Mar6 - Mar15, Col | Sprint 3 | Nov5 - Nov16

      Description

      Context

      Content marketing relies on tons of user-generated or crawled content (images, documents...). But this content will probably come without any structured meta-data or tags so you can't really take advantage of them. Let's say you are creating a marketing campaign for the next Symposium and this event will take place in a venue close to a beach. You want to use images... with a beach, right? With no meta data, finding an image with a beach within your repository will be a hard task (or impossible task depending on the volume of data and how it is organized). To improve asset discovery you and your team will need to manually annotate all the images in your repository using expensive human labor.

      With auto-tagging, you can assign relevant tags to all these images in an automated fashion improving assets discovery and categorization and in the end... the productivity of your team and company.

      Goal

      The goal of this user story is to implement auto-tagging for images

      Details

      Assets supported: images

      Features to identify in an image (it will depend on the provider [local, external] capabilities):

      • Basic descriptive metadata for concepts and objects.
      • Industry specifics: eg, if your industry is food, the user will receive specific food-related tags.
      • Sentiment: analyze facial attributes to auto-tag images based on emotions like happiness, surprise, and confusion
      • Demographics: tag images of people based on age, ethnicity, and gender
      • Flag: tag questionable or inappropriate images.

      Other features to identify: it will depend on the provider configuration.

      We will provide two auto-tagging modes:

      • On-premise (optional, low priority): based on a pre-trained AI model. This mode targets basic use cases (playground). The customer will need to train the model on their own. The default model is based on imageNet training.
      • External service: advanced feature recognition. We will integrate with external services. We will provide integration OOTB with: Google Vision API and Microsoft Cognitive services.

      Configuration:

      • At portal level we will be able to configure:
        • Enable/Disable auto-tagging
        • Auto-tagging provider selection
        • Provider configuration (API configuration, features to detect, confidence threshold)
        • Max number of auto-tags generated per image (the more tags the more indexing the system will need to do)
      • At instance level we will be able to configure:
        • Enable/Disable auto-tagging (if auto-tagging is disabled at portal level, we won't be able to enable it at this level)
        • Auto-tagging provider selection
        • Provider configuration (API configuration, features to detect, confidence threshold)
        • Max number of auto-tags generated per image (the more tags the more indexing the system will need to do)
      • At site level we will be able to configure:
        • Enable/Disable auto-tagging (if auto-tagging is disabled at portal or instance level, we won't be able to enable it at this level)

      Is there any option to apply auto-tagging to all the images we had before enabling auto-tagging?

      Yes, through the command line.

      Acceptance criteria

      • When a user uploads a supported media asset to a site with auto-tagging enabled the DM system will process the media asset(s) and create tags automatically storing these tags in the asset metadata.
      • Auto-tagging will be applied to images with no tags. If an image has tags, auto-tagging won't be applied.
      • The user should be able to remove all auto-tags.

      UX work needed

      • Remove all tags interaction (when disabling? anytime?)

      What could come next:

      • Documents auto-tagging
      • Assign less relevance to auto-tags
      • Automatic actions based on auto-tags: e.g, if an image is tagged as inappropriate then auto-delete.

       

        Attachments

        1. AssertAutoTag.PNG
          AssertAutoTag.PNG
          91 kB
        2. ATLocalization.PNG
          ATLocalization.PNG
          686 kB
        3. autotag.PNG
          autotag.PNG
          806 kB
        4. Google Cloud Vision.png
          Google Cloud Vision.png
          284 kB
        5. GoogleCloudVisionGeneratedTags.PNG
          GoogleCloudVisionGeneratedTags.PNG
          990 kB
        6. TensorFlow.PNG
          TensorFlow.PNG
          59 kB

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Packages

                  Version Package
                  Master