Uploaded image for project: 'PUBLIC - Liferay Portal Community Edition'
  1. PUBLIC - Liferay Portal Community Edition
  2. LPS-75089

Can't shutdown WildFly when using a server managed database connection

    Details

    • Fix Priority:
      2
    • Application Servers:
      Wildfly 10.0

      Description

      Steps to reproduce

      • Use the Wildfly package (e.g. liferay-ce-portal-wildfly-7.0-ga4-..)
      • Define a datasource in wildfly-10.0.0/standalone/configuration/standalone.xml, e.g.:
        <datasources>
          ...
          <datasource jndi-name="java:/jdbc/liferay" pool-name="liferay-pool" enabled="true" use-java-context="true">
            <connection-url>jdbc:hsqldb:liferay_db;hsqldb.write_delay=false</connection-url>
            <driver>hsqldb</driver>
            <security>
              <user-name>sa</user-name>
              <password></password>
            </security>
          </datasource>
          <drivers>
            ...
            <driver name="hsqldb" module="com.liferay.portal">
              <xa-datasource-class>org.hsqldb.jdbc.pool.JDBCXADataSource</xa-datasource-class>
            </driver>
          </drivers>
        </datasources>
        
      • Define a jndi datasource in portal-ext.properties, e.g.:
        jdbc.default.jndi.name=jdbc/liferay
        
      • Start the server
      • Wait until the server is up and running
      • Send the server a shutdown signal (kill or Ctrl+C or console)

      Current result

      • The server doesn't stop and prints a lot of errors in the log:
        ERROR [ServerService Thread Pool -- 280][JDBCExceptionReporter:234] javax.resource.ResourceException: IJ000470: You are trying to use a connection factory that has been shut down: java:/jdbc/liferay
        ERROR [Framework Event Dispatcher: Equinox Container: 4022a98e-a0a9-0017-135f-fd693010d531][com_liferay_portal_osgi_web_portlet_tracker:97] FrameworkEvent ERROR 
        com.liferay.portal.kernel.exception.SystemException: com.liferay.portal.kernel.dao.orm.ORMException: org.hibernate.exception.GenericJDBCException: could not execute query
        	at com.liferay.portal.kernel.service.persistence.impl.BasePersistenceImpl.processException(BasePersistenceImpl.java:265)
        	at com.liferay.portal.service.persistence.impl.CompanyPersistenceImpl.findAll(CompanyPersistenceImpl.java:1975)
        	at com.liferay.portal.service.persistence.impl.CompanyPersistenceImpl.findAll(CompanyPersistenceImpl.java:1884)
        	at com.liferay.portal.service.persistence.impl.CompanyPersistenceImpl.findAll(CompanyPersistenceImpl.java:1850)
        	at com.liferay.portal.service.impl.CompanyLocalServiceImpl.getCompanies(CompanyLocalServiceImpl.java:576)
        	at sun.reflect.GeneratedMethodAccessor722.invoke(Unknown Source)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:498)
        	at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:163)
        	at com.liferay.portal.spring.transaction.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:54)
        	at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:58)
        	at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:137)
        	at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
        	at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:137)
        	at com.liferay.portal.spring.aop.ServiceBeanAopProxy.invoke(ServiceBeanAopProxy.java:169)
        	at com.sun.proxy.$Proxy71.getCompanies(Unknown Source)
        	at com.liferay.portal.osgi.web.portlet.tracker.internal.PortletTracker.removedService(PortletTracker.java:219)
        	at com.liferay.portal.osgi.web.portlet.tracker.internal.PortletTracker.removedService(PortletTracker.java:103)
        	at org.osgi.util.tracker.ServiceTracker$Tracked.customizerRemoved(ServiceTracker.java:967)
        	at org.osgi.util.tracker.ServiceTracker$Tracked.customizerRemoved(ServiceTracker.java:1)
        	at org.osgi.util.tracker.AbstractTracked.untrack(AbstractTracked.java:341)
        	at org.osgi.util.tracker.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:909)
        	at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:109)
        	at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:917)
        	at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
        	at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
        	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:862)
        	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:801)
        	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.unregister(ServiceRegistrationImpl.java:222)
        	at org.apache.felix.scr.impl.manager.AbstractComponentManager$3.unregister(AbstractComponentManager.java:908)
        	at org.apache.felix.scr.impl.manager.AbstractComponentManager$3.unregister(AbstractComponentManager.java:873)
        	at org.apache.felix.scr.impl.manager.RegistrationManager.changeRegistration(RegistrationManager.java:139)
        	at org.apache.felix.scr.impl.manager.AbstractComponentManager.unregisterService(AbstractComponentManager.java:950)
        	at org.apache.felix.scr.impl.manager.AbstractComponentManager.doDeactivate(AbstractComponentManager.java:805)
        	at org.apache.felix.scr.impl.manager.AbstractComponentManager.deactivateInternal(AbstractComponentManager.java:787)
        	at org.apache.felix.scr.impl.manager.DependencyManager.deactivateComponentManager(DependencyManager.java:2225)
        	at org.apache.felix.scr.impl.manager.DependencyManager.access$500(DependencyManager.java:55)
        	at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.removedService(DependencyManager.java:1082)
        	at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.removedService(DependencyManager.java:968)
        	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerRemoved(ServiceTracker.java:1241)
        	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerRemoved(ServiceTracker.java:1136)
        	at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.untrack(ServiceTracker.java:996)
        	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:1175)
        	at org.apache.felix.scr.impl.BundleComponentActivator$ListenerInfo.serviceChanged(BundleComponentActivator.java:127)
        	at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:109)
        	at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:917)
        	at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
        	at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
        	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:862)
        	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:801)
        	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.unregister(ServiceRegistrationImpl.java:222)
        	at com.liferay.portal.spring.extender.internal.bean.ApplicationContextServicePublisher.unregister(ApplicationContextServicePublisher.java:86)
        	at com.liferay.portal.spring.extender.internal.context.ModuleApplicationContextRegistrator.stop(ModuleApplicationContextRegistrator.java:93)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:498)
        	at org.apache.felix.dm.InvocationUtil.invokeMethod(InvocationUtil.java:111)
        	at org.apache.felix.dm.InvocationUtil.invokeCallbackMethod(InvocationUtil.java:66)
        	at org.apache.felix.dm.impl.ComponentImpl.invokeCallbackMethod(ComponentImpl.java:769)
        	at org.apache.felix.dm.impl.ComponentImpl.invoke(ComponentImpl.java:760)
        	at org.apache.felix.dm.impl.ComponentImpl.unbindService(ComponentImpl.java:737)
        	at org.apache.felix.dm.impl.ComponentImpl.access$500(ComponentImpl.java:54)
        	at org.apache.felix.dm.impl.ComponentImpl$14.run(ComponentImpl.java:247)
        	at org.apache.felix.dm.impl.SerialExecutor.runTask(SerialExecutor.java:137)
        	at org.apache.felix.dm.impl.SerialExecutor.runTasks(SerialExecutor.java:119)
        	at org.apache.felix.dm.impl.SerialExecutor.execute(SerialExecutor.java:85)
        	at org.apache.felix.dm.impl.ComponentImpl.calculateStateChanges(ComponentImpl.java:252)
        	at org.apache.felix.dm.impl.ComponentImpl.stop(ComponentImpl.java:459)
        	at org.apache.felix.dm.DependencyManager.remove(DependencyManager.java:177)
        	at com.liferay.portal.spring.extender.internal.context.ModuleApplicationContextExtender$ModuleApplicationContextExtension.destroy(ModuleApplicationContextExtender.java:153)
        	at org.apache.felix.utils.extender.AbstractExtender$2.run(AbstractExtender.java:290)
        	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        	at org.apache.felix.utils.extender.AbstractExtender.destroyExtension(AbstractExtender.java:312)
        	at org.apache.felix.utils.extender.AbstractExtender.stop(AbstractExtender.java:125)
        	at com.liferay.portal.spring.extender.internal.context.ModuleApplicationContextExtender.deactivate(ModuleApplicationContextExtender.java:76)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:498)
        	at org.apache.felix.scr.impl.inject.BaseMethod.invokeMethod(BaseMethod.java:224)
        	at org.apache.felix.scr.impl.inject.BaseMethod.access$500(BaseMethod.java:39)
        	at org.apache.felix.scr.impl.inject.BaseMethod$Resolved.invoke(BaseMethod.java:617)
        	at org.apache.felix.scr.impl.inject.BaseMethod.invoke(BaseMethod.java:501)
        	at org.apache.felix.scr.impl.inject.ActivateMethod.invoke(ActivateMethod.java:302)
        	at org.apache.felix.scr.impl.inject.ActivateMethod.invoke(ActivateMethod.java:294)
        	at org.apache.felix.scr.impl.manager.SingleComponentManager.disposeImplementationObject(SingleComponentManager.java:343)
        	at org.apache.felix.scr.impl.manager.SingleComponentManager.deleteComponent(SingleComponentManager.java:152)
        	at org.apache.felix.scr.impl.manager.AbstractComponentManager.doDeactivate(AbstractComponentManager.java:813)
        	at org.apache.felix.scr.impl.manager.AbstractComponentManager.deactivateInternal(AbstractComponentManager.java:787)
        	at org.apache.felix.scr.impl.manager.DependencyManager.deactivateComponentManager(DependencyManager.java:2225)
        	at org.apache.felix.scr.impl.manager.DependencyManager.access$500(DependencyManager.java:55)
        	at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.removedService(DependencyManager.java:1082)
        	at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.removedService(DependencyManager.java:968)
        	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerRemoved(ServiceTracker.java:1241)
        	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerRemoved(ServiceTracker.java:1136)
        	at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.untrack(ServiceTracker.java:996)
        	at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:1175)
        	at org.apache.felix.scr.impl.BundleComponentActivator$ListenerInfo.serviceChanged(BundleComponentActivator.java:127)
        	at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:109)
        	at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:917)
        	at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
        	at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
        	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:862)
        	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:801)
        	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.unregister(ServiceRegistrationImpl.java:222)
        	at com.liferay.registry.internal.ServiceRegistrationWrapper.unregister(ServiceRegistrationWrapper.java:77)
        	at com.liferay.portal.servlet.MainServlet.destroy(MainServlet.java:150)
        	at io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:115)
        	at org.wildfly.extension.undertow.security.RunAsLifecycleInterceptor.handle(RunAsLifecycleInterceptor.java:65)
        	at org.wildfly.extension.undertow.security.RunAsLifecycleInterceptor.destroy(RunAsLifecycleInterceptor.java:89)
        	at io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:101)
        	at io.undertow.servlet.core.ManagedServlet$DefaultInstanceStrategy.invokeDestroy(ManagedServlet.java:254)
        	at io.undertow.servlet.core.ManagedServlet$DefaultInstanceStrategy.stop(ManagedServlet.java:246)
        	at io.undertow.servlet.core.ManagedServlet.stop(ManagedServlet.java:145)
        	at io.undertow.servlet.core.DeploymentManagerImpl.stop(DeploymentManagerImpl.java:548)
        	at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.stopContext(UndertowDeploymentService.java:141)
        	at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$2.run(UndertowDeploymentService.java:121)
        	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        	at java.lang.Thread.run(Thread.java:745)
        	at org.jboss.threads.JBossThread.run(JBossThread.java:320)
        Caused by: com.liferay.portal.kernel.dao.orm.ORMException: org.hibernate.exception.GenericJDBCException: could not execute query
        	at com.liferay.portal.dao.orm.hibernate.ExceptionTranslator.translate(ExceptionTranslator.java:33)
        	at com.liferay.portal.dao.orm.hibernate.QueryImpl.list(QueryImpl.java:139)
        	at com.liferay.portal.dao.orm.hibernate.QueryImpl.list(QueryImpl.java:110)
        	at com.liferay.portal.kernel.dao.orm.QueryUtil.list(QueryUtil.java:59)
        	at com.liferay.portal.service.persistence.impl.CompanyPersistenceImpl.findAll(CompanyPersistenceImpl.java:1956)
        	... 123 more
        Caused by: org.hibernate.exception.GenericJDBCException: could not execute query
        	at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:140)
        	at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:128)
        	at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
        	at org.hibernate.loader.Loader.doList(Loader.java:2545)
        	at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2276)
        	at org.hibernate.loader.Loader.list(Loader.java:2271)
        	at org.hibernate.hql.classic.QueryTranslatorImpl.list(QueryTranslatorImpl.java:940)
        	at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:196)
        	at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1268)
        	at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102)
        	at com.liferay.portal.dao.orm.hibernate.QueryImpl.list(QueryImpl.java:127)
        	... 126 more
        Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000470: You are trying to use a connection factory that has been shut down: java:/jdbc/liferay
        	at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:146)
        	at org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:66)
        	at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy$LazyConnectionInvocationHandler.getTargetConnection(LazyConnectionDataSourceProxy.java:403)
        	at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy$LazyConnectionInvocationHandler.invoke(LazyConnectionDataSourceProxy.java:376)
        	at com.sun.proxy.$Proxy16.prepareStatement(Unknown Source)
        	at org.hibernate.jdbc.AbstractBatcher.getPreparedStatement(AbstractBatcher.java:534)
        	at org.hibernate.jdbc.AbstractBatcher.getPreparedStatement(AbstractBatcher.java:452)
        	at org.hibernate.jdbc.AbstractBatcher.prepareQueryStatement(AbstractBatcher.java:161)
        	at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1700)
        	at org.hibernate.loader.Loader.doQuery(Loader.java:801)
        	at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:274)
        	at org.hibernate.loader.Loader.doList(Loader.java:2542)
        	... 133 more
        Caused by: javax.resource.ResourceException: IJ000470: You are trying to use a connection factory that has been shut down: java:/jdbc/liferay
        	at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:735)
        	at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:138)
        	... 144 more
        ... (many of these)
      • You need to use a hard kill (kill -9) to stop the process.
      • There is no difference with other types of databases or newer versions of WildFly

      Expected result

      • The process does stop when sending the shutdown signal

      Analysis

      WildFly closes the database connection before sending the "destroy" signal to the servlets. I understand that this is somehow unexpected (I would expect the inverse order as well). But I think that the undeployment should be smart enough to expect a closed connection. Such a situation can happen on other application servers as well, for example in a configuration with an external database were the operating system closes all open connections before sending the kill signal to the open applications.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                2 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Days since last comment:
                  1 year, 37 weeks, 4 days ago

                  Packages

                  Version Package