-
Type:
Bug
-
Status: Closed
-
Resolution: Fixed
-
Affects Version/s: 6.1.20 EE GA2, 6.1.30 EE GA3, 6.2.10 EE GA1, 7.0.0 M3
-
Branch Version/s:6.2.x, 6.1.x
-
Backported to Branch:Committed
-
Story Points:28
-
Fix Priority:5
-
Databases:Oracle Database 11.1
-
Git Pull Request:
To reproduce this issue you have to config the following environment:
- Cluster
- Sharding
- Oracle
- Scheduler Enabled
Steps to reproduce:
1- Start node 1
2- After the complete start, start the node 2
The second node suddenly shutdowns trying to get the scheduled jobs from the node one:
10:14:02,811 INFO [localhost-startStop-1][BaseDB:481] Database supports case sensitive queries 10:14:03,391 INFO [Incoming-2,LIFERAY-CONTROL-CHANNEL,ALBERTOCHAPARRO-7033][LicenseManager:?] Failed to find directory D:\servers\cluster\6.2\liferay-portal-6.2-ee-sp2-node2\data\license 10:14:23,389 ERROR [localhost-startStop-1][MainServlet:209] com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.scheduler.SchedulerException: Unable to initialize scheduler com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.scheduler.SchedulerException: Unable to initialize scheduler at com.liferay.portal.events.StartupAction.run(StartupAction.java:74) at com.liferay.portal.servlet.MainServlet.processStartupEvents(MainServlet.java:1238) at com.liferay.portal.servlet.MainServlet.init(MainServlet.java:206) at javax.servlet.GenericServlet.init(GenericServlet.java:160) at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1280) at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1193) at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1088) at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5176) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5460) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:633) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:656) at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1635) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: com.liferay.portal.kernel.scheduler.SchedulerException: Unable to initialize scheduler at com.liferay.portal.scheduler.ClusterSchedulerEngine.initialize(ClusterSchedulerEngine.java:244) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:115) at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:48) at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111) at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56) at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111) at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56) at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111) at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56) at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111) at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56) at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111) at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56) at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111) at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56) at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111) at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56) at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111) at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56) at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111) at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56) at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111) at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56) at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111) at com.liferay.portal.spring.aop.ServiceBeanAopProxy.invoke(ServiceBeanAopProxy.java:175) at com.sun.proxy.$Proxy488.initialize(Unknown Source) at com.liferay.portal.scheduler.SchedulerEngineHelperImpl.initialize(SchedulerEngineHelperImpl.java:574) at com.liferay.portal.kernel.scheduler.SchedulerEngineHelperUtil.initialize(SchedulerEngineHelperUtil.java:237) at com.liferay.portal.events.StartupAction.doRun(StartupAction.java:198) at com.liferay.portal.ee.license.StartupAction.doRun(Unknown Source) at com.liferay.portal.events.StartupAction.run(StartupAction.java:68) ... 19 more Caused by: com.liferay.portal.kernel.scheduler.SchedulerException: Unable to load scheduled jobs from cluster node ALBERTOCHAPARRO-51674 at com.liferay.portal.scheduler.ClusterSchedulerEngine.callMaster(ClusterSchedulerEngine.java:561) at com.liferay.portal.scheduler.ClusterSchedulerEngine.initialize(ClusterSchedulerEngine.java:236) ... 53 more Caused by: java.util.concurrent.TimeoutException at com.liferay.portal.kernel.cluster.FutureClusterResponses.get(FutureClusterResponses.java:88) at com.liferay.portal.scheduler.ClusterSchedulerEngine.callMaster(ClusterSchedulerEngine.java:553) ... 54 more
Technical Notes:
During node 2 startup this server gets the node 1 ID executing a SELECT FOR UPDATE over lock_ table and it generates a lock on that table since the commit doesn't release it (just in Oracle). Then this node calls node 1 to get the scheduled jobs and the same query is executed in node 1 which waits until the lock is released so after ten seconds a timeout is triggered shutting down node 2.
- causes
-
LPS-48194 Sharding:Portal is unable to start caused by NoSuchUserException during the verify process (VerifyGroup)
- Closed
- is duplicated by
-
LPS-46319 Second node startup fails because it's unable to initialize scheduler (fix for master)
- Closed
- is related to
-
LPS-22698 LockLocalService fails to lock up row as expected
- Closed
- relates
-
LPE-12857 Second node in cluster fails to startup
-
- Closed
-