Uploaded image for project: 'PUBLIC - Liferay Portal Community Edition'
  1. PUBLIC - Liferay Portal Community Edition
  2. LPS-93762

Reindex does not consider SYSTEM company index: this can prevent index replication on Elasticsearch

    Details

      Description

      Background: As of Elasticsearch 5.x, it is not possible to define the values of shards / replicas in "elasticsearch.yml".

      1. Download & unpack Elasticsearch 6.5.x/6.1.x
      2. Set
        config/elasticsearch.yml
        cluster.name: LiferayElasticsearchCluster
        node.max_local_storage_nodes: 3
        
      1. Start a 3-node ES cluster
        1. ./bin/elasticsearch
        2. ./bin/elasticsearch
        3. ./bin/elasticsearch
      2. Create & put it under LIFERAY_HOME/osgi/configs
        com.liferay.portal.search.elasticsearch6.configuration.ElasticsearchConfiguration.config
        transportAddresses=["localhost:9300","localhost:9301","localhost:9302"]
        operationMode="REMOTE"
        
      1. Start Liferay
      2. Go to System Settings - Elasticsearch 6
      3. Set "1" for "Number of Index Replicas"

        The number of replicas for each Liferay index. If unset, no replicas are used. Changes to this value take effect after a full reindex.

        Default is zero ("0")

      1. Go to Control Panel - Configuration - Search: "Reindex all search indexes."
      2. Check Elasticsearch Cluster status through curl -X GET "localhost:9200/_shard_stores?status=green&human&pretty" or using elasticsearch-head

      Expected Result: index with name "liferay-0" appears under 2 nodes (replicated)
      Actual Result: index with name "liferay-0" appears under only 1 node (not replicated)

      Notes for 7.0.x

      • Does not affect the default Elasticsearch 2 connector because there is no such option for the connector, but this can be configured at elasticsearch.yml level
      • To test with ES6, you need to the deploy the "Liferay Connector to Elasticsearch 6" app (portal-search-elasticsearch6 modules) and disable the default portal-search-elasticsearch module.
      • Reindex is in the Server Admin

      QA Notes
      Test solution with

      • index.on.startup=true and
      • Solr 7/5
        also

      Workarounds

      Workaround 1

      1. Stop Liferay
      2. Stop ES cluster
      3. Delete "company-0" from the node it holds
      4. Create com.liferay.portal.search.elasticsearch6.configuration.ElasticsearchConfiguration.config with the required "replica/sharding" settings (or set it through the UI before shutting down Liferay)
      5. Start ES cluster
      6. Start Liferay

      Workaround 2

      1. Update the "Index Settings" for the SYSTEM company "liferay-0" using the REST API: https://www.elastic.co/guide/en/elasticsearch/reference/6.5/indices-update-settings.html
      2. Update com.liferay.portal.search.elasticsearch6.configuration.ElasticsearchConfiguration.config with the required "replica/sharding" settings or set it through the UI before shutting down Liferay so it'll still take effect in the future in case they'd wipe out the indexes for whatever reasons

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Days since last comment:
                  34 weeks, 1 day ago

                  Packages

                  Version Package
                  7.1.10 DXP FP13
                  7.2.10 DXP FP1
                  7.2.1 CE GA2
                  Master