7.1.X, 7.2.X, Master
The root cause of this problem is connected with LPS-72400. When trying to retrieve records beyond the index.max_result_window, ES throws an error and seems to only return results from before 10,000. Liferay ingests these results and returns them as the last results. So if results 10,000-10,020 should be shown on page 501, it will only display 9,980-10,000 which is the same as page 500.
Furthermore, ES7 is using a new parameter "track_total_hits" that needs to be set in order to return accurate result numbers after 10,000 results. See: [reference|
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/search-request-track-total-hits.html#search-request-track-total-hits]. So in liferay versions after ES7, liferay acts as if there are no more than 10,000 results and will only paginate to page 500.
Steps to Reproduce:
- Go to Control Panel > Configuration > Server Administration > Script. Paste and execute this groovy script to create 10K+ User Groups. (It took about 7 min for the user groups to be created)
- Go to Control Panel > Configuration > System Setting > Foundation > Elasticsearch. Uncheck "Log exceptions only" and Save.
- Go to Control Panel > Users > User Groups
- Click on page 501 of the search container (note )
Click on page 501 of the search container, no errors in console, user group "Test User Group 10001" is on that page
In versions of liferay using ES 7+, such as master, page 501 doesn't exist. In older versions clicking on page 501 has the same results as page 500.
An exception is thrown in the logs:
This exception comes from this call which creates a query with size > 10000.