Uploaded image for project: 'PUBLIC - Liferay Portal Community Edition'
  1. PUBLIC - Liferay Portal Community Edition
  2. LPS-83306

VerifyGroup.verifySites is slow when there are many organizations without sites

    Details

      Description

      Some Liferay environments have 1,000,000 or more organizations, and most of them do not have sites. The way it's currently written, the verify will iterate over 1,000,000 organizations and run 2,000,000 queries in order to determine that it doesn't need to do anything.

      Steps to Reproduce

      1. Setup a remote database (database not running on the local machine or on a local VM or docker container)
      2. Create many organizations – use the Groovy script below in the scripting console to create 100,000 organizations without a site
        import com.liferay.portal.kernel.model.OrganizationConstants
        import com.liferay.portal.kernel.search.IndexStatusManagerThreadLocal
        import com.liferay.portal.kernel.security.auth.PrincipalThreadLocal
        import com.liferay.portal.kernel.service.OrganizationLocalServiceUtil
        
        def organizationCount = OrganizationLocalServiceUtil.getOrganizationsCount()
        def targetOrganizationCount = 100000
        
        IndexStatusManagerThreadLocal.setIndexReadOnly(true)
        
        if (organizationCount < targetOrganizationCount) {
        	organizationCount.upto(targetOrganizationCount, {
        		def organizationName = "Sample " + it
        
        		System.out.println(organizationName)
        
        		OrganizationLocalServiceUtil.addOrganization(
        			PrincipalThreadLocal.getUserId(),
        			OrganizationConstants.DEFAULT_PARENT_ORGANIZATION_ID,
        			organizationName, false);
        	})
        }
      3. Run VerifyGroup and observe how long it takes to complete
        import com.liferay.portal.kernel.dao.orm.EntityCacheUtil
        import com.liferay.portal.kernel.dao.orm.FinderCacheUtil
        import com.liferay.portal.verify.VerifyGroup
        
        FinderCacheUtil.clearCache()
        EntityCacheUtil.clearCache()
        
        new VerifyGroup().verify()

      In a local database, this completes very quickly (approximately 15 seconds for 100,000 organizations, or 3 minutes for 1,000,000 organizations). In a remote database, this completes very slowly (it did not complete within 2 hours for 1,000,000 organizations).

      It's only when there is network latency (which is what would happen in a real, non-development environment) that the verify process takes much longer.

        Attachments

          Activity

            People

            Assignee:
            summer.zhang Summer Zhang
            Reporter:
            minhchau.dang Minhchau Dang
            Participants of an Issue:
            Recent user:
            Csaba Turcsan
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:
              Days since last comment:
              3 years, 19 weeks, 2 days ago

                Packages

                Version Package
                7.1.10 DXP FP1
                7.1.1 CE GA2
                7.1.10.1 SP1
                7.1.X
                Master