Uploaded image for project: 'PUBLIC - Liferay Portal Community Edition'
  1. PUBLIC - Liferay Portal Community Edition
  2. LPS-47443

Scheduler deadlock during quartz clusterCheckIn()

    Details

      Description

      A deadlock exception occurs during a cluster check in for the Quartz Scheduler.

      Steps to reproduce:

      1. Setup a basic two-node cluster with the following properties:
        #
        # SQL Server
        #
        jdbc.default.driverClassName=net.sourceforge.jtds.jdbc.Driver
        jdbc.default.url=jdbc:jtds:sqlserver://172.16.10.72/lportal
        jdbc.default.username=sa
        jdbc.default.password=password
        
        #
        # Clusterlink Properties
        #
        cluster.link.enabled=true
        ehcache.cluster.link.replication.enabled=true
        
        # this property will increase frequency of quartz scheduler checkin
        persisted.scheduler.org.quartz.jobStore.clusterCheckinInterval=25
        
        # for some reason enabling serializable transaction levels gives us the issues more readily (can be reproduced without)
        persisted.scheduler.org.quartz.jobStore.txIsolationLevelSerializable=true
        
      2. Start up the cluster
      3. Wait

      Expected
      No deadlock
      Actual
      Deadlock Exception is thrown

      Note
      This is a known unresolved issue with Quartz https://jira.terracotta.org/jira/browse/QTZ-413

      D:\bundles\cluster\node1-trunk>cd tomcat-7.0.42\bin
      
      D:\bundles\cluster\node1-trunk\tomcat-7.0.42\bin>catalina run
      Using CATALINA_BASE:   "D:\bundles\cluster\node1-trunk\tomcat-7.0.42"
      Using CATALINA_HOME:   "D:\bundles\cluster\node1-trunk\tomcat-7.0.42"
      Using CATALINA_TMPDIR: "D:\bundles\cluster\node1-trunk\tomcat-7.0.42\temp"
      Using JRE_HOME:        "C:\Program Files\Java\jdk1.7.0_67\"
      Using CLASSPATH:       "D:\bundles\cluster\node1-trunk\tomcat-7.0.42\bin\bootstrap.jar;D:\bundles\cluster\node1-trunk\tomcat-7.0.42\bin\tomcat-juli.jar"
      Sep 05, 2014 9:46:19 PM org.apache.catalina.core.AprLifecycleListener init
      INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\Program Files\Java\jdk1.7.0_67\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\apache-ant\bin;C:\Program Files\Java\jdk1.7.0_67\\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\MacType;C:\Program Files\MySQL\MySQL Server 5.5\bin;.
      Sep 05, 2014 9:46:19 PM org.apache.catalina.startup.SetAllPropertiesRule begin
      WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'redirectport' to '9443' did not find a matching property.
      Sep 05, 2014 9:46:19 PM org.apache.catalina.startup.SetAllPropertiesRule begin
      WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'redirectport' to '9443' did not find a matching property.
      Sep 05, 2014 9:46:19 PM org.apache.coyote.AbstractProtocol init
      INFO: Initializing ProtocolHandler ["http-bio-9080"]
      Sep 05, 2014 9:46:19 PM org.apache.coyote.AbstractProtocol init
      INFO: Initializing ProtocolHandler ["ajp-bio-9009"]
      Sep 05, 2014 9:46:19 PM org.apache.catalina.startup.Catalina load
      INFO: Initialization processed in 708 ms
      Sep 05, 2014 9:46:19 PM org.apache.catalina.core.StandardService startInternal
      INFO: Starting service Catalina
      Sep 05, 2014 9:46:19 PM org.apache.catalina.core.StandardEngine startInternal
      INFO: Starting Servlet Engine: Apache Tomcat/7.0.42
      Sep 05, 2014 9:46:19 PM org.apache.catalina.startup.HostConfig deployDescriptor
      INFO: Deploying configuration descriptor D:\bundles\cluster\node1-trunk\tomcat-7.0.42\conf\Catalina\localhost\ROOT.xml
      Loading jar:file:/D:/bundles/cluster/node1-trunk/tomcat-7.0.42/webapps/ROOT/WEB-INF/lib/portal-impl.jar!/system.properties
      Loading file:/D:/bundles/cluster/node1-trunk/tomcat-7.0.42/webapps/ROOT/WEB-INF/classes/system-ext.properties
      Loading jar:file:/D:/bundles/cluster/node1-trunk/tomcat-7.0.42/webapps/ROOT/WEB-INF/lib/portal-impl.jar!/portal.properties
      Loading file:/D:/bundles/cluster/node1-trunk/portal-ext.properties
      Sep 05, 2014 9:46:32 PM org.apache.catalina.core.ApplicationContext log
      INFO: Initializing Spring root WebApplicationContext
      21:46:37,116 INFO  [localhost-startStop-1][DialectDetector:71] Determine dialect for Microsoft SQL Server 10
      21:46:37,140 INFO  [localhost-startStop-1][DialectDetector:136] Found dialect com.liferay.portal.dao.orm.hibernate.SQLServer2008Dialect
      21:46:50,275 WARN  [localhost-startStop-1][ResourceActionsImpl:617] Cannot load META-INF/resource-actions/default.xml
      21:46:51,250 INFO  [localhost-startStop-1][ClusterBase:149] Autodetecting JGroups outgoing IP address and interface for www.google.com:80
      21:46:51,279 INFO  [localhost-startStop-1][ClusterBase:165] Setting JGroups outgoing IP address to 172.16.20.71 and interface to net0
      
      -------------------------------------------------------------------
      GMS: address=LIFERAY-ALEX-50454, cluster=LIFERAY-CONTROL-CHANNEL, physical address=172.16.20.71:65055
      -------------------------------------------------------------------
      21:46:51,958 INFO  [localhost-startStop-1][BaseReceiver:64] Accepted view [LIFERAY-ALEX-21910|1] [LIFERAY-ALEX-21910, LIFERAY-ALEX-50454]
      21:46:51,978 INFO  [localhost-startStop-1][ClusterBase:93] Create a new channel with properties UDP(bind_addr=172.16.20.71;oob_thread_pool_keep_alive_time=30000;timer_keep_alive_time=5000;external_port=0;oob_thread_pool_enabled=true;max_bundle_size=64000;diagnostics_ttl=8;physical_addr_max_fetch_attempts=10;receive_on_all_interfaces=false;mcast_port=23301;thread_pool_min_threads=2;thread_pool_keep_alive_time=30000;thread_pool_max_threads=10;enable_diagnostics=true;ucast_send_buf_size=100000;ip_ttl=8;oob_thread_pool_queue_max_size=500;enable_bundling=true;thread_pool_queue_enabled=true;suppress_time_different_cluster_warnings=60000;timer_rejection_policy=run;diagnostics_port=7500;oob_thread_pool_max_threads=10;wheel_size=200;disable_loopback=false;logical_addr_cache_max_size=500;ip_mcast=true;tick_time=50;logical_addr_cache_expiration=120000;thread_pool_rejection_policy=Discard;suppress_time_different_version_warnings=60000;oob_thread_pool_min_threads=2;who_has_cache_timeout=2000;port_range=50;stats=true;mcast_send_buf_size=100000;id=21;mcast_recv_buf_size=500000;diagnostics_addr=224.0.75.75;bind_port=0;tos=8;oob_thread_pool_rejection_policy=discard;loopback=true;oob_thread_pool_queue_enabled=true;name=UDP;enable_unicast_bundling=true;thread_pool_enabled=true;log_discard_msgs_version=true;thread_naming_pattern=cl;ucast_recv_buf_size=64000;timer_max_threads=10;timer_queue_max_size=500;discard_incompatible_packets=true;ergonomics=true;bundler_capacity=200000;max_bundle_timeout=20;mcast_group_addr=239.255.0.1;bind_interface_str=;timer_min_threads=4;log_discard_msgs=true;thread_pool_queue_max_size=500;bundler_type=new;timer_type=old)_:PING(id=6;return_entire_cache=false;num_initial_members=20;break_on_coord_rsp=true;force_sending_discovery_rsps=true;stagger_timeout=0;stats=true;name=PING;ergonomics=true;timeout=2000;num_initial_srv_members=0)_:MERGE3(id=54;max_participants_in_merge=100;stats=true;name=MERGE3;min_interval=10000;ergonomics=true;max_interval=30000)_:FD_SOCK(bind_addr=172.16.20.71;external_port=0;port_range=50;stats=true;suspect_msg_interval=5000;client_bind_port=0;ergonomics=true;num_tries=3;id=3;get_cache_timeout=1000;sock_conn_timeout=1000;bind_interface_str=;name=FD_SOCK;keep_alive=true;start_port=0)_:FD_ALL(id=29;interval=3000;stats=true;name=FD_ALL;ergonomics=true;msg_counts_as_heartbeat=false;timeout=10000;timeout_check_interval=2000)_:VERIFY_SUSPECT(id=13;bind_addr=172.16.20.71;bind_interface_str=;stats=true;name=VERIFY_SUSPECT;num_msgs=1;ergonomics=true;use_mcast_rsps=false;use_icmp=false;timeout=1500)_:pbcast.NAKACK2(use_mcast_xmit_req=false;use_mcast_xmit=false;suppress_time_non_member_warnings=60000;max_msg_batch_size=500;xmit_from_random_member=false;stats=true;xmit_table_max_compaction_time=30000;log_not_found_msgs=true;ergonomics=true;discard_delivered_msgs=true;print_stability_history_on_failed_xmit=false;id=57;become_server_queue_size=50;max_rebroadcast_timeout=2000;xmit_table_msgs_per_row=2000;xmit_table_num_rows=100;name=NAKACK2;log_discard_msgs=true;xmit_table_resize_factor=1.2;xmit_interval=1000)_:UNICAST2(max_retransmit_time=60000;max_bytes=10000000;max_msg_batch_size=500;stats=true;xmit_table_max_compaction_time=60000;exponential_backoff=300;max_stable_msgs=5;log_not_found_msgs=true;ergonomics=true;stable_interval=60000;id=40;conn_expiry_timeout=60000;xmit_table_msgs_per_row=2000;xmit_table_automatic_purging=true;xmit_table_num_rows=100;name=UNICAST2;timeout=400,800,1600,3200;xmit_interval=1000;xmit_table_resize_factor=1.2;use_range_based_retransmitter=true)_:pbcast.STABLE(id=16;desired_avg_gossip=50000;max_bytes=8000000;stats=true;cap=0.1;name=STABLE;ergonomics=true;stability_delay=1000)_:pbcast.GMS(print_local_addr=true;stats=true;max_bundling_time=50;log_collect_msgs=true;resume_task_timeout=20000;log_view_warnings=true;num_prev_views=20;ergonomics=true;use_flush_if_present=true;print_physical_addrs=true;merge_timeout=5000;id=14;num_prev_mbrs=50;leave_timeout=1000;view_bundling=true;name=GMS;join_timeout=3000;handle_concurrent_startup=true;view_ack_collection_timeout=2000;max_join_attempts=0)_:UFC(id=45;max_block_time=5000;max_credits=2000000;stats=true;ignore_synchronous_response=true;min_credits=800000;name=UFC;min_threshold=0.4;ergonomics=true)_:MFC(id=44;max_block_time=5000;max_credits=2000000;stats=true;ignore_synchronous_response=true;min_credits=800000;name=MFC;min_threshold=0.4;ergonomics=true)_:FRAG2(id=5;frag_size=61440;stats=true;name=FRAG2;ergonomics=true)_:RSVP(id=55;stats=true;name=RSVP;resend_interval=2000;throw_exception_on_timeout=true;ergonomics=true;ack_on_delivery=true;timeout=10000)_ [Sanitized]
      
      -------------------------------------------------------------------
      GMS: address=LIFERAY-ALEX-58261, cluster=LIFERAY-TRANSPORT-CHANNEL-0, physical address=172.16.20.71:65056
      -------------------------------------------------------------------
      21:46:52,361 INFO  [localhost-startStop-1][BaseReceiver:64] Accepted view [LIFERAY-ALEX-12684|1] [LIFERAY-ALEX-12684, LIFERAY-ALEX-58261]
      21:46:52,379 INFO  [localhost-startStop-1][ClusterBase:93] Create a new channel with properties UDP(bind_addr=172.16.20.71;oob_thread_pool_keep_alive_time=30000;timer_keep_alive_time=5000;external_port=0;oob_thread_pool_enabled=true;max_bundle_size=64000;diagnostics_ttl=8;physical_addr_max_fetch_attempts=10;receive_on_all_interfaces=false;mcast_port=23302;thread_pool_min_threads=2;thread_pool_keep_alive_time=30000;thread_pool_max_threads=10;enable_diagnostics=true;ucast_send_buf_size=100000;ip_ttl=8;oob_thread_pool_queue_max_size=500;enable_bundling=true;thread_pool_queue_enabled=true;suppress_time_different_cluster_warnings=60000;timer_rejection_policy=run;diagnostics_port=7500;oob_thread_pool_max_threads=10;wheel_size=200;disable_loopback=false;logical_addr_cache_max_size=500;ip_mcast=true;tick_time=50;logical_addr_cache_expiration=120000;thread_pool_rejection_policy=Discard;suppress_time_different_version_warnings=60000;oob_thread_pool_min_threads=2;who_has_cache_timeout=2000;port_range=50;stats=true;mcast_send_buf_size=100000;id=21;mcast_recv_buf_size=500000;diagnostics_addr=224.0.75.75;bind_port=0;tos=8;oob_thread_pool_rejection_policy=discard;loopback=true;oob_thread_pool_queue_enabled=true;name=UDP;enable_unicast_bundling=true;thread_pool_enabled=true;log_discard_msgs_version=true;thread_naming_pattern=cl;ucast_recv_buf_size=64000;timer_max_threads=10;timer_queue_max_size=500;discard_incompatible_packets=true;ergonomics=true;bundler_capacity=200000;max_bundle_timeout=20;mcast_group_addr=239.255.0.2;bind_interface_str=;timer_min_threads=4;log_discard_msgs=true;thread_pool_queue_max_size=500;bundler_type=new;timer_type=old)_:PING(id=6;return_entire_cache=false;num_initial_members=20;break_on_coord_rsp=true;force_sending_discovery_rsps=true;stagger_timeout=0;stats=true;name=PING;ergonomics=true;timeout=2000;num_initial_srv_members=0)_:MERGE3(id=54;max_participants_in_merge=100;stats=true;name=MERGE3;min_interval=10000;ergonomics=true;max_interval=30000)_:FD_SOCK(bind_addr=172.16.20.71;external_port=0;port_range=50;stats=true;suspect_msg_interval=5000;client_bind_port=0;ergonomics=true;num_tries=3;id=3;get_cache_timeout=1000;sock_conn_timeout=1000;bind_interface_str=;name=FD_SOCK;keep_alive=true;start_port=0)_:FD_ALL(id=29;interval=3000;stats=true;name=FD_ALL;ergonomics=true;msg_counts_as_heartbeat=false;timeout=10000;timeout_check_interval=2000)_:VERIFY_SUSPECT(id=13;bind_addr=172.16.20.71;bind_interface_str=;stats=true;name=VERIFY_SUSPECT;num_msgs=1;ergonomics=true;use_mcast_rsps=false;use_icmp=false;timeout=1500)_:pbcast.NAKACK2(use_mcast_xmit_req=false;use_mcast_xmit=false;suppress_time_non_member_warnings=60000;max_msg_batch_size=500;xmit_from_random_member=false;stats=true;xmit_table_max_compaction_time=30000;log_not_found_msgs=true;ergonomics=true;discard_delivered_msgs=true;print_stability_history_on_failed_xmit=false;id=57;become_server_queue_size=50;max_rebroadcast_timeout=2000;xmit_table_msgs_per_row=2000;xmit_table_num_rows=100;name=NAKACK2;log_discard_msgs=true;xmit_table_resize_factor=1.2;xmit_interval=1000)_:UNICAST2(max_retransmit_time=60000;max_bytes=10000000;max_msg_batch_size=500;stats=true;xmit_table_max_compaction_time=60000;exponential_backoff=300;max_stable_msgs=5;log_not_found_msgs=true;ergonomics=true;stable_interval=60000;id=40;conn_expiry_timeout=60000;xmit_table_msgs_per_row=2000;xmit_table_automatic_purging=true;xmit_table_num_rows=100;name=UNICAST2;timeout=400,800,1600,3200;xmit_interval=1000;xmit_table_resize_factor=1.2;use_range_based_retransmitter=true)_:pbcast.STABLE(id=16;desired_avg_gossip=50000;max_bytes=8000000;stats=true;cap=0.1;name=STABLE;ergonomics=true;stability_delay=1000)_:pbcast.GMS(print_local_addr=true;stats=true;max_bundling_time=50;log_collect_msgs=true;resume_task_timeout=20000;log_view_warnings=true;num_prev_views=20;ergonomics=true;use_flush_if_present=true;print_physical_addrs=true;merge_timeout=5000;id=14;num_prev_mbrs=50;leave_timeout=1000;view_bundling=true;name=GMS;join_timeout=3000;handle_concurrent_startup=true;view_ack_collection_timeout=2000;max_join_attempts=0)_:UFC(id=45;max_block_time=5000;max_credits=2000000;stats=true;ignore_synchronous_response=true;min_credits=800000;name=UFC;min_threshold=0.4;ergonomics=true)_:MFC(id=44;max_block_time=5000;max_credits=2000000;stats=true;ignore_synchronous_response=true;min_credits=800000;name=MFC;min_threshold=0.4;ergonomics=true)_:FRAG2(id=5;frag_size=61440;stats=true;name=FRAG2;ergonomics=true)_:RSVP(id=55;stats=true;name=RSVP;resend_interval=2000;throw_exception_on_timeout=true;ergonomics=true;ack_on_delivery=true;timeout=10000)_ [Sanitized]
      Starting Liferay Portal Community Edition 7.0.0 CE M1 (Wilberforce / Build 7000 / May 1, 2014)
      21:47:02,037 INFO  [localhost-startStop-1][BaseDB:482] Database does not support case sensitive queries
      21:47:02,468 INFO  [localhost-startStop-1][ServerDetector:124] Server supports hot deploy
      21:47:02,473 INFO  [localhost-startStop-1][PluginPackageUtil:1003] Reading plugin package for the root context
      21:47:09,866 INFO  [localhost-startStop-1][AutoDeployDir:139] Auto deploy scanner started for D:\bundles\cluster\node1-trunk\deploy
      21:47:12,521 ERROR [localhost-startStop-1][JDBCExceptionReporter:82] Data truncation
      21:47:12,522 ERROR [localhost-startStop-1][JDBCExceptionReporter:82] String or binary data would be truncated.
      21:47:12,526 ERROR [localhost-startStop-1][PortalInstances:500] com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.exception.SystemException: com.liferay.portal.kernel.dao.orm.ORMException: org.hibernate.exception.DataException: Could not execute JDBC batch update
      com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.exception.SystemException: com.liferay.portal.kernel.dao.orm.ORMException: org.hibernate.exception.DataException: Could not execute JDBC batch update
              at com.liferay.portal.events.AddDefaultLayoutPrototypesAction.run(AddDefaultLayoutPrototypesAction.java:54)
              at com.liferay.portal.events.AddDefaultDataAction.run(AddDefaultDataAction.java:40)
              at com.liferay.portal.kernel.events.SimpleAction.processLifecycleEvent(SimpleAction.java:26)
              at com.liferay.portal.events.EventsProcessorUtil._process(EventsProcessorUtil.java:129)
              at com.liferay.portal.events.EventsProcessorUtil.process(EventsProcessorUtil.java:74)
              at com.liferay.portal.util.PortalInstances._initCompany(PortalInstances.java:494)
              at com.liferay.portal.util.PortalInstances.initCompany(PortalInstances.java:93)
              at com.liferay.portal.servlet.MainServlet.initCompanies(MainServlet.java:760)
              at com.liferay.portal.servlet.MainServlet.init(MainServlet.java:332)
              at javax.servlet.GenericServlet.init(GenericServlet.java:160)
              at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1280)
              at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1193)
              at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1088)
              at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5176)
              at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5460)
              at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
              at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
              at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
              at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:633)
              at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:656)
              at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1635)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
              at java.util.concurrent.FutureTask.run(FutureTask.java:262)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: com.liferay.portal.kernel.exception.SystemException: com.liferay.portal.kernel.dao.orm.ORMException: org.hibernate.exception.DataException: Could not execute JDBC batch update
              at com.liferay.portal.service.persistence.impl.BasePersistenceImpl.processException(BasePersistenceImpl.java:250)
              at com.liferay.portal.service.persistence.impl.ResourcePermissionPersistenceImpl.findByC_N_S_P(ResourcePermissionPersistenceImpl.java:2660)
              at com.liferay.portal.service.persistence.impl.ResourcePermissionPersistenceImpl.findByC_N_S_P(ResourcePermissionPersistenceImpl.java:2482)
              at com.liferay.portal.service.impl.ResourceLocalServiceImpl.addResources(ResourceLocalServiceImpl.java:855)
              at com.liferay.portal.service.impl.ResourceLocalServiceImpl.addResources(ResourceLocalServiceImpl.java:260)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:115)
              at com.liferay.portal.spring.transaction.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:53)
              at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:55)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ServiceBeanAopProxy.invoke(ServiceBeanAopProxy.java:169)
              at com.sun.proxy.$Proxy15.addResources(Unknown Source)
              at com.liferay.portal.service.impl.LayoutLocalServiceImpl.addLayout(LayoutLocalServiceImpl.java:413)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:115)
              at com.liferay.portal.service.impl.LayoutLocalServiceVirtualLayoutsAdvice.invoke(LayoutLocalServiceVirtualLayoutsAdvice.java:181)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.service.impl.LayoutLocalServiceStagingAdvice.invoke(LayoutLocalServiceStagingAdvice.java:139)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.transaction.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:53)
              at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:55)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ServiceBeanAopProxy.invoke(ServiceBeanAopProxy.java:169)
              at com.sun.proxy.$Proxy115.addLayout(Unknown Source)
              at com.liferay.portal.service.impl.LayoutPrototypeLocalServiceImpl.addLayoutPrototype(LayoutPrototypeLocalServiceImpl.java:108)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:115)
              at com.liferay.portal.spring.transaction.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:53)
              at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:55)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
              at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
              at com.liferay.portal.spring.aop.ServiceBeanAopProxy.invoke(ServiceBeanAopProxy.java:169)
              at com.sun.proxy.$Proxy163.addLayoutPrototype(Unknown Source)
              at com.liferay.portal.service.LayoutPrototypeLocalServiceUtil.addLayoutPrototype(LayoutPrototypeLocalServiceUtil.java:93)
              at com.liferay.portal.events.AddDefaultLayoutPrototypesAction.addLayoutPrototype(AddDefaultLayoutPrototypesAction.java:121)
              at com.liferay.portal.events.AddDefaultLayoutPrototypesAction.addBlogPage(AddDefaultLayoutPrototypesAction.java:63)
              at com.liferay.portal.events.AddDefaultLayoutPrototypesAction.doRun(AddDefaultLayoutPrototypesAction.java:206)
              at com.liferay.portal.events.AddDefaultLayoutPrototypesAction.run(AddDefaultLayoutPrototypesAction.java:51)
              ... 25 more
      Caused by: com.liferay.portal.kernel.dao.orm.ORMException: org.hibernate.exception.DataException: Could not execute JDBC batch update
              at com.liferay.portal.dao.orm.hibernate.ExceptionTranslator.translate(ExceptionTranslator.java:33)
              at com.liferay.portal.dao.orm.hibernate.QueryImpl.list(QueryImpl.java:125)
              at com.liferay.portal.dao.orm.hibernate.QueryImpl.list(QueryImpl.java:104)
              at com.liferay.portal.kernel.dao.orm.QueryUtil.list(QueryUtil.java:59)
              at com.liferay.portal.service.persistence.impl.ResourcePermissionPersistenceImpl.findByC_N_S_P(ResourcePermissionPersistenceImpl.java:2641)
              ... 109 more
      Caused by: org.hibernate.exception.DataException: Could not execute JDBC batch update
              at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:102)
              at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
              at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:275)
              at org.hibernate.jdbc.AbstractBatcher.prepareStatement(AbstractBatcher.java:114)
              at org.hibernate.jdbc.AbstractBatcher.prepareStatement(AbstractBatcher.java:109)
              at org.hibernate.jdbc.AbstractBatcher.prepareBatchStatement(AbstractBatcher.java:244)
              at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2412)
              at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2875)
              at org.hibernate.action.EntityInsertAction.execute(EntityInsertAction.java:79)
              at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:273)
              at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:265)
              at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:184)
              at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
              at com.liferay.portal.dao.orm.hibernate.event.NestableAutoFlushEventListener.onAutoFlush(NestableAutoFlushEventListener.java:62)
              at org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:1185)
              at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1261)
              at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102)
              at com.liferay.portal.dao.orm.hibernate.QueryImpl.list(QueryImpl.java:113)
              ... 112 more
      Caused by: java.sql.BatchUpdateException: Data truncation
              at net.sourceforge.jtds.jdbc.JtdsStatement.executeBatch(JtdsStatement.java:947)
              at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:1135)
              at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)
              at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)
              ... 127 more
      Sep 05, 2014 9:47:13 PM org.apache.catalina.core.ApplicationContext log
      INFO: Initializing Spring FrameworkServlet 'Remoting Servlet'
      Sep 05, 2014 9:47:14 PM org.apache.catalina.startup.HostConfig deployDirectory
      INFO: Deploying web application directory D:\bundles\cluster\node1-trunk\tomcat-7.0.42\webapps\marketplace-portlet
      21:47:16,421 INFO  [localhost-startStop-1][HotDeployImpl:217] Deploying marketplace-portlet from queue
      21:47:16,423 INFO  [localhost-startStop-1][PluginPackageUtil:1006] Reading plugin package for marketplace-portlet
      Loading file:/D:/bundles/cluster/node1-trunk/tomcat-7.0.42/temp/0-marketplace-portlet/WEB-INF/classes/portlet.properties
      Loading file:/D:/bundles/cluster/node1-trunk/tomcat-7.0.42/temp/0-marketplace-portlet/WEB-INF/classes/service.properties
      Loading file:/D:/bundles/cluster/node1-trunk/tomcat-7.0.42/temp/0-marketplace-portlet/WEB-INF/classes/portlet.properties
      Sep 05, 2014 9:47:16 PM org.apache.catalina.core.ApplicationContext log
      INFO: Initializing Spring root WebApplicationContext
      Loading file:/D:/bundles/cluster/node1-trunk/tomcat-7.0.42/temp/0-marketplace-portlet/WEB-INF/classes/service.properties
      Loading file:/D:/bundles/cluster/node1-trunk/tomcat-7.0.42/temp/0-marketplace-portlet/WEB-INF/classes/service.properties
      Loading file:/D:/bundles/cluster/node1-trunk/tomcat-7.0.42/temp/0-marketplace-portlet/WEB-INF/classes/service.properties
      21:47:17,222 INFO  [localhost-startStop-1][HookHotDeployListener:569] Registering hook for marketplace-portlet
      Loading file:/D:/bundles/cluster/node1-trunk/tomcat-7.0.42/temp/0-marketplace-portlet/WEB-INF/classes/portal.properties
      21:47:17,355 INFO  [localhost-startStop-1][HookHotDeployListener:650] Hook for marketplace-portlet is available for use
      21:47:17,360 INFO  [localhost-startStop-1][PortletHotDeployListener:238] Registering portlets for marketplace-portlet
      Loading file:/D:/bundles/cluster/node1-trunk/tomcat-7.0.42/temp/0-marketplace-portlet/WEB-INF/classes/portlet.properties
      21:47:18,165 INFO  [localhost-startStop-1][PortletHotDeployListener:392] 3 portlets for marketplace-portlet are available for use
      Sep 05, 2014 9:47:18 PM org.apache.coyote.AbstractProtocol start
      INFO: Starting ProtocolHandler ["http-bio-9080"]
      Sep 05, 2014 9:47:18 PM org.apache.coyote.AbstractProtocol start
      INFO: Starting ProtocolHandler ["ajp-bio-9009"]
      Sep 05, 2014 9:47:18 PM org.apache.catalina.startup.Catalina start
      INFO: Server startup in 58264 ms
      21:48:06,352 ERROR [QuartzScheduler_PersistedQuartzSchedulerEngineInstance-LIFERAY-ALEX1409953612956_ClusterManager][PortalJobStore:120] ClusterManager: Error managing cluster: Failure updating scheduler state when checking-in: Transaction (Process ID 91) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
      org.quartz.JobPersistenceException: Failure updating scheduler state when checking-in: Transaction (Process ID 91) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction. [See nested exception: java.sql.SQLException: Transaction (Process ID 91) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.]
              at org.quartz.impl.jdbcjobstore.JobStoreSupport.clusterCheckIn(JobStoreSupport.java:3406)
              at org.quartz.impl.jdbcjobstore.JobStoreSupport.doCheckin(JobStoreSupport.java:3253)
              at org.quartz.impl.jdbcjobstore.JobStoreSupport$ClusterManager.manage(JobStoreSupport.java:3870)
              at org.quartz.impl.jdbcjobstore.JobStoreSupport$ClusterManager.run(JobStoreSupport.java:3907)
      Caused by: java.sql.SQLException: Transaction (Process ID 91) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
              at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:368)
              at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2820)
              at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2258)
              at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:632)
              at net.sourceforge.jtds.jdbc.JtdsStatement.processResults(JtdsStatement.java:584)
              at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL(JtdsStatement.java:546)
              at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.executeUpdate(JtdsPreparedStatement.java:504)
              at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:147)
              at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.updateSchedulerState(StdJDBCDelegate.java:2930)
              at org.quartz.impl.jdbcjobstore.JobStoreSupport.clusterCheckIn(JobStoreSupport.java:3400)
              ... 3 more
      

      @see https://www.liferay.com/community/forums/-/message_boards/message/38541081 Have not gotten any responses there so posting it here

        Attachments

          Activity

            People

            • Assignee:
              support-lep@liferay.com SE Support
              Reporter:
              bitsofinfo bitsofinfo (Inactive)
              Participants of an Issue:
              Recent user:
              Minhchau Dang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Days since last comment:
                6 years, 5 days ago

                Packages

                Version Package