Details

    • Branch Version/s:
      6.1.x
    • Backported to Branch:
      Committed
    • Fix Priority:
      4
    • Similar Issues:
      Show 5 results 

      Description

      I have an installation with 7 companies and one of them has 1,3 millions of users.

      When I tried to do an users reindex the system goes OutOf Memory.
      My JVM is configured with "-Xmx1024m". I know it is few memory but IMHO the reindex process will have to load a bunch of records, update the SearchEngine, and discard it.

      During my analisys I notice that UserIndexer use an ActionableDynamicQuery to fill up an array of DocumentImpl. After that all the records of the selected company are processed the array is submitted to IndexerWriter.

      I've used an ext (indexer-ext-6.2.0.1.war) to help me:

      • speedup the processing of companies with few users (look UserActionableDynamicQueryExt)
      • increase logging

      The files "catalina-02.out", "heap-histo-02.log" and JVisualVM snapshot "application-1376574827825.apps" shows how heap usage increases constantly for the company with millions of users.

      In 6.1 strategy was to invoke the IndexWriter for every bunch of records.

      I've extended my ext to call IndexWriter every 5000 DocumentImpl.
      The files "catalina-03.out" and "heap-histo-03.log" show how heap usage is lower.

      1. catalina-02.out
        129 kB
        Mauro Mariuzzo
      2. catalina-03.out
        141 kB
        Mauro Mariuzzo
      3. heap-histo-02.log
        226 kB
        Mauro Mariuzzo
      4. heap-histo-03.log
        748 kB
        Mauro Mariuzzo
      1. Screen Shot 2013-10-22 at 5.27.42 PM.png
        309 kB
      2. Screen Shot 2013-10-22 at 5.27.54 PM.png
        366 kB

        Issue Links

          Activity

          Hide
          Michael Han added a comment -

          Mauro, I think this should be resolved for 6.2 across all indexers. The default sizing for the ActionableDynamicQuery has been set to 10,000 but we could probably make that configurable if need

          Show
          Michael Han added a comment - Mauro, I think this should be resolved for 6.2 across all indexers. The default sizing for the ActionableDynamicQuery has been set to 10,000 but we could probably make that configurable if need
          Hide
          Mauro Mariuzzo added a comment -

          Hi Michael,

          for Users the code that worry me is inside reindexUsers

          ActionableDynamicQuery actionableDynamicQuery = new UserActionableDynamicQuery() {

          @Override
          protected void performAction(Object object) throws PortalException {
          User user = (User)object;

          if (!user.isDefaultUser())

          { Document document = getDocument(user); documents.add(document); }

          }
          };

          ActionableDynamicQuery load users 10,000 a time, but they are all added to documents.

          When all users are processed, documents (the whole big list) are given to searchEngine.

          Show
          Mauro Mariuzzo added a comment - Hi Michael, for Users the code that worry me is inside reindexUsers ActionableDynamicQuery actionableDynamicQuery = new UserActionableDynamicQuery() { @Override protected void performAction(Object object) throws PortalException { User user = (User)object; if (!user.isDefaultUser()) { Document document = getDocument(user); documents.add(document); } } }; ActionableDynamicQuery load users 10,000 a time, but they are all added to documents. When all users are processed, documents (the whole big list) are given to searchEngine.
          Hide
          Mauro Mariuzzo added a comment -

          Maybe I've misunderstood you comment. I agree that a solution have to be applied on all indexes

          Show
          Mauro Mariuzzo added a comment - Maybe I've misunderstood you comment. I agree that a solution have to be applied on all indexes
          Hide
          Michael Han added a comment -

          Sorry Mauro, I misread the bug filing. We are using the ActionableDynamicQuery in all Indexers but we're using it wrong. Will address

          Show
          Michael Han added a comment - Sorry Mauro, I misread the bug filing. We are using the ActionableDynamicQuery in all Indexers but we're using it wrong. Will address
          Hide
          Mauro Mariuzzo added a comment -

          If you want I can propose a fix using the standard git mechanism. I've have already done it for my tests.

          Show
          Mauro Mariuzzo added a comment - If you want I can propose a fix using the standard git mechanism. I've have already done it for my tests.
          Hide
          Michael Han added a comment -

          No worries, it's already done...

          Show
          Michael Han added a comment - No worries, it's already done...
          Hide
          James Falkner added a comment -

          CDS Global seems to have run into a very similar issue (see screenshots from Eclipse Memory Analyzer) - _documents array aren't getting properly cleaned up as evidenced by the 1.9GB of held-onto memory.

          Re-opening and assigning to Mike Han as requested.

          Show
          James Falkner added a comment - CDS Global seems to have run into a very similar issue (see screenshots from Eclipse Memory Analyzer) - _documents array aren't getting properly cleaned up as evidenced by the 1.9GB of held-onto memory. Re-opening and assigning to Mike Han as requested.

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since last comment:
                1 year, 40 weeks ago

                Development

                  Structure Helper Panel