Unable to delete a Site with many Folders or Files

• Type: Bug
• Status: Closed
• Resolution: Fixed
• Affects Version/s: 7.0.6 CE GA7, 7.0.0 DXP FP51, 7.1.0 RC1, Master
• Fix Version/s: Master
• Component/s: Documents & Media
• Labels:
  • QA-TBD
• Fix Priority: 3
• Git Pull Request: https://github.com/brianchandotcom/liferay-portal/pull/68973

I have a site with few thousands of folders inside the Document Media Library and I'm unable to delete it.

I think the strategy used to remove folders or files inside a tree of folders isn't suitable for big amounts of elements. Let me explain it better.

When you choose to remove a Site you land to

            GroupLocalServiceImpl.deleteGroup(Group)

which as this code

            // Repositories

            dlAppLocalService.deleteAllRepositories(group.getGroupId());

Method deleteAllRepositories calls deleteRepository

    protected void deleteRepository(LocalRepository localRepository)
        throws PortalException {

        long repositoryId = localRepository.getRepositoryId();

        if (!_isExternalRepository(localRepository)) {
            dlAppHelperLocalService.deleteRepositoryFileEntries(repositoryId);

            localRepository.deleteAll();
        }

        repositoryLocalService.deleteRepository(repositoryId);
    }

Contents have to be removed only when stored inside Liferay, this is right.

Here the code of DLAppHelperLocalServiceImpl.deleteRepositoryFileEntries()

    public void deleteRepositoryFileEntries(long repositoryId)
        throws PortalException {

        LocalRepository localRepository =
            RepositoryProviderUtil.getLocalRepository(repositoryId);

        List<FileEntry> fileEntries = localRepository.getRepositoryFileEntries(
            UserConstants.USER_ID_DEFAULT,
            DLFolderConstants.DEFAULT_PARENT_FOLDER_ID, QueryUtil.ALL_POS,
            QueryUtil.ALL_POS, null);

        for (FileEntry fileEntry : fileEntries) {
            deleteFileEntry(fileEntry);
        }
    }

In this method there is the first problem. It collects all the FileEntry of the repository. If we have a site with millions of entries we probably saturate JVM memory.

A better approch could be to use an ActionableDynamicQuery?

But, let's go forward. How FileEntry are collected?

In this scenario LocalRespository is a LiferayRepository, and the method calls DLFileEntryServiceImpl.getGroupFileEntries()

    public List<DLFileEntry> getGroupFileEntries(
            long groupId, long userId, long rootFolderId, int start, int end,
            OrderByComparator<DLFileEntry> obc)
        throws PortalException {

        List<Long> folderIds = dlFolderService.getFolderIds(
            groupId, rootFolderId);

        if (folderIds.isEmpty()) {
            return Collections.emptyList();
        }
        else if (userId <= 0) {
            return dlFileEntryPersistence.filterFindByG_F(
                groupId, ArrayUtil.toLongArray(folderIds), start, end, obc);
        }
        else {
            return dlFileEntryPersistence.filterFindByG_U_F(
                groupId, userId, ArrayUtil.toLongArray(folderIds), start, end,
                obc);
        }
    }

The first action is to collect all the forlderIds. In my use case ~3000.

Than it calls dlFileEntryPersistence.filterFindByG_F which build an sql having a where clause containing

"DLFileEntry.folderId = ? OR DLFileEntry.folderId = ? OR DLFileEntry.folderId = ? ....."

 with ~3000 filter parameters.

The SQL command exceed the database limit and throws an JDBCException. This is the second problem.