Uploaded image for project: 'PUBLIC - Liferay Portal Community Edition'
  1. PUBLIC - Liferay Portal Community Edition
  2. LPS-107980

Sidecar is unable to start elasticsearch server

    Details

      Description

      Steps to reproduce:
      1. Set property "cluster.link.enabled=true" (portal-ext)
      2. Set database connection properties (HSQL not supported)
      3. Create config file
      osgi/configs/com.liferay.portal.store.file.system.configuration.FileSystemStoreConfiguration.config
      4. Set config inside of new file
          -   rootDir="/Users/username/liferay/master-bundle-1/data/document_library"
      5. Make a copy of the tomcat bundle in a new location (all configs should be the same)
      6. Set second bundle port to 9080
           -   sed -i "s/8005/9005/; s/8080/9080/; s/8009/9009/; s/8443/9443/g" ../conf/server.xml
      7. Start first Portal server

      Expected Results:
      Portal starts up with no issues

      Actual Results:
      Portal fails to startup

      2020-01-31 20:18:34.305 INFO  [main][Sidecar:129] Starting sidecar
      2020-01-31 20:18:35.140 INFO  [main][Sidecar:624] Sidecar properties : {bootstrap.memory_lock=false,bootstrap.system_call_filter=false,cluster.initial_master_nodes=liferay-d8994093-db44-68bf-376f-2ca3777840ae,cluster.name=LiferayElasticsearchCluster,cluster.routing.allocation.disk.threshold_enabled=false,discovery.seed_hosts=172.16.22.206,http.cors.allow-origin=/https?:\/\/localhost(:[0-9]+)?/,http.cors.enabled=true,network.host=172.16.22.206,node.data=true,node.ingest=true,node.master=true,node.name=liferay-d8994093-db44-68bf-376f-2ca3777840ae,node.store.allow_mmap=false,path.data=/home/tai/Documents/Daily_Builds/liferay-portal-bundle-tomcat/bundles/data/elasticsearch7/indices,path.logs=/home/tai/Documents/Daily_Builds/liferay-portal-bundle-tomcat/bundles/logs,path.repo=/home/tai/Documents/Daily_Builds/liferay-portal-bundle-tomcat/bundles/data/elasticsearch7/repo}
      ERROR: [1] bootstrap checks failed
      [1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
      2020-01-31 20:18:41.931 ERROR [com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarMainProcessCallable@2f4ffb35[-Xms1g -Xmx1g -XX:+AlwaysPreTouch -Des.distribution.flavor=default -Des.distribution.type=tar -Des.path.conf=/home/tai/Documents/Daily_Builds/liferay-portal-bundle-tomcat/bundles/tomcat-9.0.17/temp/20200131201834941KSVIOXTT/config -Des.path.home=/home/tai/Documents/Daily_Builds/liferay-portal-bundle-tomcat/bundles/elasticsearch7 -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dio.netty.allocator.type=unpooled -Dio.netty.allocator.numDirectArenas=0 -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/home/tai/Documents/Daily_Builds/liferay-portal-bundle-tomcat/bundles/tomcat-9.0.17/temp/20200131201834941KSVIOXTT -Djava.security.policy=jar:file:/home/tai/Documents/Daily_Builds/liferay-portal-bundle-tomcat/bundles/osgi/state/org.eclipse.osgi/162/0/bundleFile!/META-INF/sidecar/security.policy -Djna.nosys=true]-][Sidecar:205] Unable to start elasticsearch server
      java.util.concurrent.ExecutionException: com.liferay.petra.process.ProcessException: Unable to start elasticsearch server
      	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
      	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.Sidecar.lambda$start$2(Sidecar.java:198)
      	at com.liferay.petra.concurrent.DefaultNoticeableFuture$OnceFutureListener.complete(DefaultNoticeableFuture.java:111)
      	at com.liferay.petra.concurrent.DefaultNoticeableFuture.done(DefaultNoticeableFuture.java:85)
      	at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:384)
      	at java.util.concurrent.FutureTask.setException(FutureTask.java:251)
      	at com.liferay.petra.concurrent.DefaultNoticeableFuture.setException(DefaultNoticeableFuture.java:79)
      	at com.liferay.petra.concurrent.AsyncBroker.takeWithException(AsyncBroker.java:85)
      	at com.liferay.petra.process.local.ResponseProcessCallable.call(ResponseProcessCallable.java:40)
      	at com.liferay.petra.process.local.ResponseProcessCallable.call(ResponseProcessCallable.java:25)
      	at com.liferay.petra.process.local.LocalProcessExecutor$SubprocessReactor.call(LocalProcessExecutor.java:254)
      	at com.liferay.petra.process.local.LocalProcessExecutor$SubprocessReactor.call(LocalProcessExecutor.java:161)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: com.liferay.petra.process.ProcessException: Unable to start elasticsearch server
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.StartSidecarProcessCallable.call(StartSidecarProcessCallable.java:47)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.StartSidecarProcessCallable.call(StartSidecarProcessCallable.java:29)
      	at com.liferay.petra.process.local.RequestProcessCallable.call(RequestProcessCallable.java:40)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	... 1 more
      Caused by: java.lang.reflect.InvocationTargetException
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.ElasticsearchServerUtil.start(ElasticsearchServerUtil.java:127)
      	at com.liferay.portal.search.elasticsearch7.internal.sidecar.StartSidecarProcessCallable.call(StartSidecarProcessCallable.java:44)
      	... 6 more
      Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "exitVM.78")
      	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
      	at java.security.AccessController.checkPermission(AccessController.java:886)
      	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
      	at java.lang.SecurityManager.checkExit(SecurityManager.java:761)
      	at org.elasticsearch.secure_sm.SecureSM.innerCheckExit(SecureSM.java:253)
      	at org.elasticsearch.secure_sm.SecureSM.checkExit(SecureSM.java:206)
      	at java.lang.Runtime.exit(Runtime.java:108)
      	at java.lang.System.exit(System.java:973)
      	at org.elasticsearch.cli.Command.exit(Command.java:140)
      	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:94)
      	... 12 more
      Exception in thread "com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarMainProcessCallable@2f4ffb35[-Xms1g -Xmx1g -XX:+AlwaysPreTouch -Des.distribution.flavor=default -Des.distribution.type=tar -Des.path.conf=/home/tai/Documents/Daily_Builds/liferay-portal-bundle-tomcat/bundles/tomcat-9.0.17/temp/20200131201834941KSVIOXTT/config -Des.path.home=/home/tai/Documents/Daily_Builds/liferay-portal-bundle-tomcat/bundles/elasticsearch7 -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dio.netty.allocator.type=unpooled -Dio.netty.allocator.numDirectArenas=0 -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/home/tai/Documents/Daily_Builds/liferay-portal-bundle-tomcat/bundles/tomcat-9.0.17/temp/20200131201834941KSVIOXTT -Djava.security.policy=jar:file:/home/tai/Documents/Daily_Builds/liferay-portal-bundle-tomcat/bundles/osgi/state/org.eclipse.osgi/162/0/bundleFile!/META-INF/sidecar/security.policy -Djna.nosys=true]-" java.lang.RuntimeException: Sidecar process is not started
      	at com.liferay.portal.search.elasticsearch7.internal.connection.SidecarElasticsearchConnectionManager$RestartFutureListener.complete(SidecarElasticsearchConnectionManager.java:155)
      	at com.liferay.petra.concurrent.DefaultNoticeableFuture$OnceFutureListener.complete(DefaultNoticeableFuture.java:111)
      	at com.liferay.petra.concurrent.DefaultNoticeableFuture.done(DefaultNoticeableFuture.java:85)
      	at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:384)
      	at java.util.concurrent.FutureTask.set(FutureTask.java:233)
      	at com.liferay.petra.concurrent.DefaultNoticeableFuture.set(DefaultNoticeableFuture.java:74)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:274)
      	at java.lang.Thread.run(Thread.java:748)
      

      Full stack: https://gist.github.com/tainguyen100/915f73ddceb41bede6a064beea134129

        Attachments

          Activity

            People

            • Assignee:
              tina.tian Tina Tian
              Reporter:
              tai.nguyen Tai Nguyen
              Participants of an Issue:
              Recent user:
              Tai Nguyen
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since last comment:
                27 weeks ago

                Packages

                Version Package