Uploaded image for project: 'PUBLIC - Liferay Portal Community Edition'
  1. PUBLIC - Liferay Portal Community Edition
  2. LPS-96061

Property values in OSGi configuration (.config) files are truncated after LPS-93457

    Details

      Description

      The scenario below uses the Elasticsearch 6 config, though it could be reproduced with any configuration where for example we can expect to get Java regexp values.

      Steps to reproduce:

      1. Create a DXP 7.1 bundle patched with FP11
      2. Add com.liferay.portal.search.elasticsearch6.configuration.ElasticsearchConfiguration.config osgi/configs
      3. Go to Control Panel > Configuration > Search
      4. Go to System Settings > Search > Elasticsearch 6: observe that the "Additional Index Configuration" JSON got truncated:
         "remove_orphan_punctation_1" : {
                  "type": "pattern_replace",
                  "pattern": "((\\s)*[\\u2000-\\u206F\\u2E00-\\u2E7F\\\\'!\"#$%&()*+,.\\-\\/:;<=>?@\\[\\]^_`{|}~]*(\\s)+)",
                  "replacement": " "
              },
        
      5. Reindex All Search indexes

      Actual behavior: You receive an exception. Check the config file and you will notice that the value of the property additionalIndexConfigurations is truncated.

      2019-05-21 09:40:13.351 ERROR [liferay/background_task-1][SearchEngineInitializer:154] Error encountered while reindexing 
      java.lang.IllegalStateException: SettingsException[Failed to load settings from [{_ "analysis": {_ "filter": {_ "autocompleteFilter": {_ "max_shingle_size": "3",_ "min_shingle_size": "2",_ "type": "shingle",_ "filler_token": ""_ },_ "remove_whitespaces": {_ "type": "pattern_replace",_ "pattern": "s{2,}",_ "replacement": " "_ },_ "dutch_stop": {_ "type": "stop",_ "stopwords": "_dutch_"_ },_ "german_stop": {_ "type": "stop",_ "stopwords": "_german_"_ },_ "french_stop": {_ "type": "stop",_ "stopwords": "_french_"_ },_ "english_stop": {_ "type": "stop",_ "stopwords": "_english_"_ },_ "dutch_stemmer": {_ "type": "stemmer",_ "language": "dutch"_ },_ "dutch_override": {_ "type": "stemmer_override",_ "rules": [_ "fiets=>fiets",_ "bromfiets=>bromfiets",_ "ei=>eier",_ "kind=>kinder"_ ]_ },_ "remove_orphan_punctation_1" : {_ "type": "pattern_replace",_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"]]; nested: MarkedYAMLException[while scanning a double-quoted scalar_ in 'reader', line 46, column 22:_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"_ ^_found unknown escape character '(39)_ in 'reader', line 46, column 36:_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"_ ^__ at [Source: java.io.StringReader@67790e61; line: 46, column: 20]]; nested: ScannerException[while scanning a double-quoted scalar_ in 'reader', line 46, column 22:_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"_ ^_found unknown escape character '(39)_ in 'reader', line 46, column 36:_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"_ ^_]; [Sanitized] 
       at com.liferay.portal.search.elasticsearch6.internal.ElasticsearchSearchEngine.initialize(ElasticsearchSearchEngine.java:116) 
       at com.liferay.portal.kernel.search.SearchEngineProxyWrapper.initialize(SearchEngineProxyWrapper.java:95) 
       at com.liferay.portal.search.internal.SearchEngineHelperImpl.initialize(SearchEngineHelperImpl.java:214) 
       at com.liferay.portal.kernel.search.SearchEngineHelperUtil.initialize(SearchEngineHelperUtil.java:83) 
       at com.liferay.portal.search.internal.SearchEngineInitializer.doReIndex(SearchEngineInitializer.java:107) 
       at com.liferay.portal.search.internal.SearchEngineInitializer.reindex(SearchEngineInitializer.java:67) 
       at com.liferay.portal.search.internal.SearchEngineInitializer.reindex(SearchEngineInitializer.java:63) 
       at com.liferay.portal.search.internal.background.task.ReindexPortalBackgroundTaskExecutor.reindex(ReindexPortalBackgroundTaskExecutor.java:56) 
       at com.liferay.portal.search.internal.background.task.ReindexBackgroundTaskExecutor.execute(ReindexBackgroundTaskExecutor.java:54) 
       at com.liferay.portal.background.task.internal.SerialBackgroundTaskExecutor.execute(SerialBackgroundTaskExecutor.java:62) 
       at com.liferay.portal.kernel.backgroundtask.DelegatingBackgroundTaskExecutor.execute(DelegatingBackgroundTaskExecutor.java:41) 
       at com.liferay.portal.background.task.internal.ThreadLocalAwareBackgroundTaskExecutor.execute(ThreadLocalAwareBackgroundTaskExecutor.java:72) 
       at com.liferay.portal.background.task.internal.messaging.BackgroundTaskMessageListener.doReceive(BackgroundTaskMessageListener.java:132) 
       at com.liferay.portal.kernel.messaging.BaseMessageListener.receive(BaseMessageListener.java:26) 
       at com.liferay.portal.kernel.messaging.InvokerMessageListener.receive(InvokerMessageListener.java:74) 
       at com.liferay.portal.kernel.messaging.ParallelDestination$1.run(ParallelDestination.java:52) 
       at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask._runTask(ThreadPoolExecutor.java:752) 
       at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask.run(ThreadPoolExecutor.java:664) 
       at java.lang.Thread.run(Thread.java:748) 
      Caused by: SettingsException[Failed to load settings from [{_ "analysis": {_ "filter": {_ "autocompleteFilter": {_ "max_shingle_size": "3",_ "min_shingle_size": "2",_ "type": "shingle",_ "filler_token": ""_ },_ "remove_whitespaces": {_ "type": "pattern_replace",_ "pattern": "s{2,}",_ "replacement": " "_ },_ "dutch_stop": {_ "type": "stop",_ "stopwords": "_dutch_"_ },_ "german_stop": {_ "type": "stop",_ "stopwords": "_german_"_ },_ "french_stop": {_ "type": "stop",_ "stopwords": "_french_"_ },_ "english_stop": {_ "type": "stop",_ "stopwords": "_english_"_ },_ "dutch_stemmer": {_ "type": "stemmer",_ "language": "dutch"_ },_ "dutch_override": {_ "type": "stemmer_override",_ "rules": [_ "fiets=>fiets",_ "bromfiets=>bromfiets",_ "ei=>eier",_ "kind=>kinder"_ ]_ },_ "remove_orphan_punctation_1" : {_ "type": "pattern_replace",_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"]]; nested: MarkedYAMLException[while scanning a double-quoted scalar_ in 'reader', line 46, column 22:_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"_ ^_found unknown escape character '(39)_ in 'reader', line 46, column 36:_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"_ ^__ at [Source: java.io.StringReader@67790e61; line: 46, column: 20]]; nested: ScannerException[while scanning a double-quoted scalar_ in 'reader', line 46, column 22:_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"_ ^_found unknown escape character '(39)_ in 'reader', line 46, column 36:_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"_ ^_]; [Sanitized] 
       at org.elasticsearch.common.settings.Settings$Builder.loadFromSource(Settings.java:1178) 
       at com.liferay.portal.search.elasticsearch6.internal.settings.SettingsBuilder.loadFromSource(SettingsBuilder.java:49) 
       at com.liferay.portal.search.elasticsearch6.internal.index.CompanyIndexFactory.loadAdditionalIndexConfigurations(CompanyIndexFactory.java:184) 
       at com.liferay.portal.search.elasticsearch6.internal.index.CompanyIndexFactory.setSettings(CompanyIndexFactory.java:299) 
       at com.liferay.portal.search.elasticsearch6.internal.index.CompanyIndexFactory.createIndex(CompanyIndexFactory.java:151) 
       at com.liferay.portal.search.elasticsearch6.internal.index.CompanyIndexFactory.createIndices(CompanyIndexFactory.java:74) 
       at com.liferay.portal.search.elasticsearch6.internal.ElasticsearchSearchEngine.initialize(ElasticsearchSearchEngine.java:110) 
       ... 18 more 
      Caused by: com.fasterxml.jackson.dataformat.yaml.snakeyaml.error.MarkedYAMLException: while scanning a double-quoted scalar_ in 'reader', line 46, column 22:_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"_ ^_found unknown escape character '(39)_ in 'reader', line 46, column 36:_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"_ ^__ at [Source: java.io.StringReader@67790e61; line: 46, column: 20] [Sanitized] 
       at com.fasterxml.jackson.dataformat.yaml.snakeyaml.error.MarkedYAMLException.from(MarkedYAMLException.java:27) 
       at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:343) 
       at org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:53) 
       at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:724) 
       at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:730) 
       at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:730) 
       at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:730) 
       at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:699) 
       at org.elasticsearch.common.settings.Settings.access$500(Settings.java:84) 
       at org.elasticsearch.common.settings.Settings$Builder.loadFromSource(Settings.java:1176) 
       ... 24 more 
      Caused by: while scanning a double-quoted scalar_ in 'reader', line 46, column 22:_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"_ ^_found unknown escape character '(39)_ in 'reader', line 46, column 36:_ "pattern": "((s)*[ÔÇÇ-Ôü»Ô©Ç-Ô╣┐\'!"_ ^_ [Sanitized] 
       at org.yaml.snakeyaml.scanner.ScannerImpl.scanFlowScalarNonSpaces(ScannerImpl.java:1906) 
       at org.yaml.snakeyaml.scanner.ScannerImpl.scanFlowScalar(ScannerImpl.java:1843) 
       at org.yaml.snakeyaml.scanner.ScannerImpl.fetchFlowScalar(ScannerImpl.java:1029) 
       at org.yaml.snakeyaml.scanner.ScannerImpl.fetchDouble(ScannerImpl.java:1011) 
       at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:396) 
       at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:226) 
       at org.yaml.snakeyaml.parser.ParserImpl$ParseFlowMappingValue.produce(ParserImpl.java:761) 
       at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:157) 
       at org.yaml.snakeyaml.parser.ParserImpl.getEvent(ParserImpl.java:167) 
       at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:340) 
       ... 32 more 

      Expected behavior: You can reindex without issues. The config file does not get truncated.


      Works on DXP FP10 or below. The "original JSON" for "additionalIndexConfigurations" looks like this:

      {
        "analysis": {
          "filter": {
            "autocompleteFilter": {
              "max_shingle_size": "3",
              "min_shingle_size": "2",
              "type": "shingle",
              "filler_token": ""
            },
            "remove_whitespaces": {
              "type": "pattern_replace",
              "pattern": "\\s{2,}",
              "replacement": " "
            },
            "dutch_stop": {
              "type": "stop",
              "stopwords": "_dutch_"
            },
            "german_stop": {
              "type": "stop",
              "stopwords": "_german_"
            },
            "french_stop": {
              "type": "stop",
              "stopwords": "_french_"
            },
            "english_stop": {
              "type": "stop",
              "stopwords": "_english_"
            },
            "dutch_stemmer": {
              "type": "stemmer",
              "language": "dutch"
            },
            "dutch_override": {
              "type": "stemmer_override",
              "rules": [
                "fiets=>fiets",
                "bromfiets=>bromfiets",
                "ei=>eier",
                "kind=>kinder"
              ]
            },
            "remove_orphan_punctation_1" : {
                "type": "pattern_replace",
                "pattern": "((\\s)*[\\u2000-\\u206F\\u2E00-\\u2E7F\\\\'!\"#$%&()*+,.\\-\\/:;<=>?@\\[\\]^_`{|}~]*(\\s)+)",
                "replacement": " "
            },
            "remove_orphan_punctation_2" : {
                "type": "pattern_replace",
                "pattern": "((\\s)+[\\u2000-\\u206F\\u2E00-\\u2E7F\\\\'!\"#$%&()*+,.\\-\\/:;<=>?@\\[\\]^_`{|}~]*(\\s)*)",
                "replacement": " "
            },
            "remove_orphan_punctation_3" : {
                "type": "pattern_replace",
                "pattern": "[\\u2000-\\u206F\\u2E00-\\u2E7F\\\\'!\"#$%&()*+,.\\-\\/:;<=>?@\\[\\]^_`{|}~]*$",
                "replacement": " "
            },
            "remove_orphan_punctation_4" : {
                "type": "pattern_replace",
                "pattern": "(^[\\u2000-\\u206F\\u2E00-\\u2E7F\\\\'!\"#$%&()*+,.\\-\\/:;<=>?@\\[\\]^_`{|}~]*)",
                "replacement": " "
            }
          },
          "analyzer": {
            "autocomplete-nl": {
              "filter": [
                "lowercase",
                "asciifolding",
                "dutch_stop",
                "autocompleteFilter",
                "remove_orphan_punctation_1",
                "remove_orphan_punctation_2",
                "remove_orphan_punctation_3",
                "remove_orphan_punctation_4",
                "remove_whitespaces",
                "trim"
              ],
              "char_filter": [
                "html_strip"
              ],
              "type": "custom",
              "tokenizer": "whitespace"
            },
            "autocomplete-fr": {
              "filter": [
                "lowercase",
                "french_stop",
                "autocompleteFilter",
                "remove_orphan_punctation_1",
                "remove_orphan_punctation_2",
                "remove_orphan_punctation_3",
                "remove_orphan_punctation_4",
                "remove_whitespaces",
                "trim"
              ],
              "char_filter": [
                "html_strip"
              ],
              "type": "custom",
              "tokenizer": "whitespace"
            },
            "autocomplete-en": {
              "filter": [
                "lowercase",
                "asciifolding",
                "english_stop",
                "autocompleteFilter",
                "remove_orphan_punctation_1",
                "remove_orphan_punctation_2",
                "remove_orphan_punctation_3",
                "remove_orphan_punctation_4",
                "remove_whitespaces",
                "trim"
              ],
              "char_filter": [
                "html_strip"
              ],
              "type": "custom",
              "tokenizer": "whitespace"
            },
            "autocomplete-de": {
              "filter": [
                "lowercase",
                "asciifolding",
                "german_stop",
                "autocompleteFilter",
                "remove_orphan_punctation_1",
                "remove_orphan_punctation_2",
                "remove_orphan_punctation_3",
                "remove_orphan_punctation_4",
                "remove_whitespaces",
                "trim"
              ],
              "char_filter": [
                "html_strip"
              ],
              "type": "custom",
              "tokenizer": "whitespace"
            },
            "dutch": {
              "tokenizer": "standard",
              "filter": [
                "lowercase",
                "asciifolding",
                "dutch_stop",
                "dutch_override",
                "dutch_stemmer"
              ]
            }
          }
        }
      }
      

      Caused by LPS-93457.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Days since last comment:
                  13 weeks, 4 days ago

                  Packages

                  Version Package
                  7.2.10 DXP FP1
                  7.2.X
                  Master