Elasticsearch client dropping documents without reporting server errors

• Type: Regression Bug
• Status: Closed
• Resolution: Fixed
• Affects Version/s: 7.1.X, 7.2.X, Master
• Fix Version/s: 7.1.10 DXP FP17, 7.1.10.4 SP4, 7.1.X, 7.2.0 GA1, 7.2.X, Master
• Component/s: Search Infrastructure > Elasticsearch Connector
• Labels:
  • ee-ts
  • liferay-dxp-71-sp4
  • liferay-fixpack-dxp-17-7110
  • search-line-of-work-assigned
• Branch Version/s:
  7.1.x
• Backported to Branch:
  Committed
• Fix Priority: 4
• Epic Link: Low Level Search in 7.2
• Sprint: Search | S02 Sprint 12, Search | S02 Sprint 13, Search | S02 Sprint 14
• Git Pull Request: https://github.com/brianchandotcom/liferay-portal/pull/69805

Steps to reproduce

Start a vanilla Elasticsearch 6.5 server, without language analyzer plugins. (This is to force errors when indexing Liferay documents.)

Add any content with multi language fields (web articles, document files etc.)

Perform a full reindex.

Observed: Elasticsearch errors in the Elasticsearch console (missing language analyzers).

Observed: Documents are not indexed; zero search results.

Expected: Elasticsearch errors in Liferay logs, when "log exceptions only" is enabled; Liferay exceptions in Liferay logs, when "log exceptions only" is disabled.

Observed: No indication of why documents are missing; No errors of any kind in Liferay logs.

 

Alternate steps to reproduce (doesnt require configuration of elastic server)

Go to Control Panel > Configuration > System Settings > Search > Elasticsearch 6,  paste into Additional Type Mappings:

{
   "LiferayDocumentType":{
      "properties":{
         "expando__custom_fields__test":{
            "format":"yyyyMMddHHmmss",
            "store":true,
            "type":"date"
         }
      }
   }
}

Go to Control Panel > Configuration > Search, Reindex all search indexes.

Go to Control Panel > Configuration > Custom Fields > Document > Add a Text Field called "test", under Advanced Properties, select searchable AS TEXT

Go to Document and Media > Add a Document ("File Upload") called "doc" and in the Test custom field, put the word text

Publish the document and see ERROR in tomcat logs

 

For additional logging, Go to Control Panel > Configuration > Server Administration > Log Levels, set "com.liferay.portal.search.elasticsearch6" to WARN