Uploaded image for project: 'PUBLIC - Liferay Portal Community Edition'
  1. PUBLIC - Liferay Portal Community Edition
  2. LPS-118944

Sidecar fails to restart when updating Elasticsearch 7 System Settings with incorrect syntax

    Details

      Description

      Steps to reproduce:

      1. Before starting Portal, within osgi/config, deploy the ElasticsearchConfiguration.config file with the following (uses incorrect syntax):
      com.liferay.portal.search.elasticsearch7.configuration.ElasticsearchConfiguration.config
      sidecarJVMOptions="-Xms256m|-Xmx512m"
      1. Start Portal with Sidecar
      2. Navigate to System Settings > Search > Elasticsearch 7
      3. Update any setting
      4. Save

      Expected result: Sidecar restarts and the configuration with the incorrect syntax gracefully fails to apply.

      Actual result: Sidecar fails to restart and errors are continuously thrown in the console:

      2020-08-12 01:00:24.211 ERROR [CM Event Dispatcher (Fire ConfigurationEvent: pid=com.liferay.portal.search.elasticsearch7.configuration.ElasticsearchConfiguration)][ElasticsearchConnectionManager:84] Elasticsearch sidecar could not be started. Search will be unavailable. Manual installation of Elasticsearch and activation of remote mode is recommended.
      java.lang.RuntimeException: java.io.IOException: Broken pipe
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.Sidecar.startElasticsearch(Sidecar.java:347)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.Sidecar.start(Sidecar.java:112)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarManager.lambda$applyConfigurations$0(SidecarManager.java:134)
      	at com.liferay.portal.search.elasticsearch7.internal.connection.ElasticsearchConnection.connect(ElasticsearchConnection.java:61)
      	at com.liferay.portal.search.elasticsearch7.internal.connection.ElasticsearchConnectionManager.addElasticsearchConnection(ElasticsearchConnectionManager.java:78)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarManager.applyConfigurations(SidecarManager.java:141)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarManager.onElasticsearchConfigurationUpdate(SidecarManager.java:71)
      	at java.lang.Iterable.forEach(Iterable.java:75)
      	at com.liferay.portal.search.elasticsearch7.internal.configuration.ElasticsearchConfigurationWrapper.activate(ElasticsearchConfigurationWrapper.java:237)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:228)
      	at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41)
      	at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:664)
      	at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:510)
      	at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:317)
      	at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:307)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.invokeModifiedMethod(SingleComponentManager.java:837)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.modify(SingleComponentManager.java:792)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:710)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:674)
      	at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.configurationUpdated(ConfigurableComponentHolder.java:435)
      	at org.apache.felix.scr.impl.manager.RegionConfigurationSupport.configurationEvent(RegionConfigurationSupport.java:317)
      	at org.apache.felix.scr.impl.manager.RegionConfigurationSupport$2.configurationEvent(RegionConfigurationSupport.java:119)
      	at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.sendEvent(ConfigurationManager.java:1685)
      	at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.run(ConfigurationManager.java:1637)
      	at org.apache.felix.cm.impl.UpdateThread.run0(UpdateThread.java:138)
      	at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:105)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.io.IOException: Broken pipe
      	at java.io.FileOutputStream.writeBytes(Native Method)
      	at java.io.FileOutputStream.write(FileOutputStream.java:326)
      	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
      	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
      	at java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputStream.java:1823)
      	at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:719)
      	at java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputStream.java:1823)
      	at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:719)
      	at com.liferay.petra.process.local.LocalProcessChannel.write(LocalProcessChannel.java:88)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.Sidecar.startElasticsearch(Sidecar.java:328)
      	... 30 more
      2020-08-12 01:00:24.214 WARN  [com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarMainProcessCallable@27dac88a[-Xms256m|-Xmx512m -Des.path.conf=/home/joshchong/liferay/bundles/master-stable-sidecar-restart-new/tomcat-9.0.33/temp/sidecar8557400666572841077/config -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dio.netty.allocator.type=unpooled -Dio.netty.allocator.numDirectArenas=0 -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/home/joshchong/liferay/bundles/master-stable-sidecar-restart-new/tomcat-9.0.33/temp/sidecar8557400666572841077 -Djava.security.policy=jar:file:/home/joshchong/liferay/bundles/master-stable-sidecar-restart-new/osgi/state/org.eclipse.osgi/204/0/bundleFile!/META-INF/sidecar.policy -Djna.nosys=true]-][Sidecar:585] Sidecar Elasticsearch process is aborted
      java.util.concurrent.ExecutionException: com.liferay.petra.process.TerminationProcessException: Subprocess terminated with exit code 1
      	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
      	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.Sidecar$RestartFutureListener.complete(Sidecar.java:581)
      	at com.liferay.petra.concurrent.DefaultNoticeableFuture$OnceFutureListener.complete(DefaultNoticeableFuture.java:111)
      	at com.liferay.petra.concurrent.DefaultNoticeableFuture.done(DefaultNoticeableFuture.java:85)
      	at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:384)
      	at java.util.concurrent.FutureTask.setException(FutureTask.java:251)
      	at com.liferay.petra.concurrent.DefaultNoticeableFuture.setException(DefaultNoticeableFuture.java:79)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:271)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: com.liferay.petra.process.TerminationProcessException: Subprocess terminated with exit code 1
      	at com.liferay.petra.process.local.LocalProcessExecutor$SubprocessReactor.call(LocalProcessExecutor.java:309)
      	at com.liferay.petra.process.local.LocalProcessExecutor$SubprocessReactor.call(LocalProcessExecutor.java:161)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	... 1 more
      2020-08-12 01:00:24.215 INFO  [com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarMainProcessCallable@27dac88a[-Xms256m|-Xmx512m -Des.path.conf=/home/joshchong/liferay/bundles/master-stable-sidecar-restart-new/tomcat-9.0.33/temp/sidecar8557400666572841077/config -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dio.netty.allocator.type=unpooled -Dio.netty.allocator.numDirectArenas=0 -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/home/joshchong/liferay/bundles/master-stable-sidecar-restart-new/tomcat-9.0.33/temp/sidecar8557400666572841077 -Djava.security.policy=jar:file:/home/joshchong/liferay/bundles/master-stable-sidecar-restart-new/osgi/state/org.eclipse.osgi/204/0/bundleFile!/META-INF/sidecar.policy -Djna.nosys=true]-][Sidecar:593] Restarting sidecar Elasticsearch process
      2020-08-12 01:00:24.219 WARN  [SCR Component Actor][SidecarManager:111] Liferay is configured to use Elasticsearch engine running in a child process of current process named as sidecar. Do NOT use sidecar in production. Sidecar is useful for development and demonstration purposes. Refer to the documentation for details on the limitations of sidecar. Remote Elasticsearch connections can be configured in the Control Panel.
      2020-08-12 01:00:24.219 INFO  [SCR Component Actor][Sidecar:99] Starting sidecar Elasticsearch
      2020-08-12 01:00:24.220 ERROR [CM Event Dispatcher (Fire ConfigurationEvent: pid=com.liferay.portal.search.elasticsearch7.configuration.ElasticsearchConfiguration)][ElasticsearchConfigurationWrapper:93] bundle com.liferay.portal.search.elasticsearch7.impl:2.0.31 (204)[com.liferay.portal.search.elasticsearch7.internal.configuration.ElasticsearchConfigurationWrapper(399)] : The activate method has thrown an exception 
      java.lang.RuntimeException: java.io.IOException: Broken pipe
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.Sidecar.startElasticsearch(Sidecar.java:347)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.Sidecar.start(Sidecar.java:112)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarManager.lambda$applyConfigurations$0(SidecarManager.java:134)
      	at com.liferay.portal.search.elasticsearch7.internal.connection.ElasticsearchConnection.connect(ElasticsearchConnection.java:61)
      	at com.liferay.portal.search.elasticsearch7.internal.connection.ElasticsearchConnectionManager.addElasticsearchConnection(ElasticsearchConnectionManager.java:78)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarManager.applyConfigurations(SidecarManager.java:141)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarManager.onElasticsearchConfigurationUpdate(SidecarManager.java:71)
      	at java.lang.Iterable.forEach(Iterable.java:75)
      	at com.liferay.portal.search.elasticsearch7.internal.configuration.ElasticsearchConfigurationWrapper.activate(ElasticsearchConfigurationWrapper.java:237)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:228)
      	at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41)
      	at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:664)
      	at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:510)
      	at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:317)
      	at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:307)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.invokeModifiedMethod(SingleComponentManager.java:837)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.modify(SingleComponentManager.java:792)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:710)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:674)
      	at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.configurationUpdated(ConfigurableComponentHolder.java:435)
      	at org.apache.felix.scr.impl.manager.RegionConfigurationSupport.configurationEvent(RegionConfigurationSupport.java:317)
      	at org.apache.felix.scr.impl.manager.RegionConfigurationSupport$2.configurationEvent(RegionConfigurationSupport.java:119)
      	at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.sendEvent(ConfigurationManager.java:1685)
      	at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.run(ConfigurationManager.java:1637)
      	at org.apache.felix.cm.impl.UpdateThread.run0(UpdateThread.java:138)
      	at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:105)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.io.IOException: Broken pipe
      	at java.io.FileOutputStream.writeBytes(Native Method)
      	at java.io.FileOutputStream.write(FileOutputStream.java:326)
      	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
      	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
      	at java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputStream.java:1823)
      	at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:719)
      	at java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputStream.java:1823)
      	at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:719)
      	at com.liferay.petra.process.local.LocalProcessChannel.write(LocalProcessChannel.java:88)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.Sidecar.startElasticsearch(Sidecar.java:328)
      	... 30 more
      

      Note: This is not reproducible when using the correct syntax. In the above example this would be sidecarJVMOptions=["-Xms256m","-Xmx512m"]. This is not reproducible when Portal is connected to a remote Elasticsearch.

      Reproduced on:
      Portal master git commit: ef37423dea70b0ca7e76324e745a14e4872ff9f1
      7.3 CE GA4

        Attachments

          Activity

            People

            Assignee:
            andre.oliveira André Ricardo Barreto de Oliveira
            Reporter:
            joshua.chong Joshua Chong
            Participants of an Issue:
            Recent user:
            Liferay JIRA Bot
            Engineering Assignee:
            Wade Cao
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Days since last comment:
              3 weeks, 2 days ago
              Development End Date:
              Development Start Date:

                Packages

                Version Package