Tomcat does not shut down properly

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 4.2.1
• Fix Version/s: 4.2.2, 4.3.0
• Component/s: None
• Labels: None
• Environment: Linux (Ubuntu Dapper) on a 1.5.0_10 with the bundle liferay-portal-tomcat-jdk5-4.2.1
• Similar Issues: Show 5 results ; ;

  LEP-4949	Tomcat does not shut down cleanly in JDK 6
  LEP-2346	Improve Tomcat development process
  LEP-3782	ant deploy does not copy ROOT.xml to tomcat
  LEP-2990	Provide Tomcat 6.0.13 bundle for Liferay 4.3.0-final
  LEP-2034	Possible memory leak in Liferay/Tomcat

I have to sudo kill -9 the process. A JPDA debug shows 4 remaining thread :

• Thread [TP-Processor4] (Running)
• Thread [http-8080-Processor24] (Running)
• Thread [Timer-0] (Running)
• Thread [com.liferay.portal.kernel.deploy.AutoDeployScanner] (Running)
• Thread [HSQLDB Timer @1f71773] (Running) (this one is not present when using MySQL of course)
• Thread [DestroyJavaVM] (Running)

The console says :
Using CATALINA_BASE: /home/vpernin/java/liferay/421/temp
Using CATALINA_HOME: /home/vpernin/java/liferay/421/temp
Using CATALINA_TMPDIR: /home/vpernin/java/liferay/421/temp/temp
Using JRE_HOME: /usr/java/jdk1.5.0_10
18 janv. 2007 15:37:46 org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:8009
18 janv. 2007 15:37:46 org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/39 config=null
18 janv. 2007 15:37:46 org.apache.catalina.storeconfig.StoreLoader load
INFO: Find registry server-registry.xml at classpath resource
18 janv. 2007 15:37:46 org.apache.catalina.startup.Catalina start
INFO: Server startup in 51836 ms
18 janv. 2007 15:38:17 org.apache.coyote.http11.Http11BaseProtocol pause
INFO: Suspension de Coyote HTTP/1.1 sur http-8080
18 janv. 2007 15:38:18 org.apache.catalina.core.StandardService stop
INFO: Arrêt du service Catalina
15:38:18,704 INFO [[/tunnel-web]:646] Closing Spring root WebApplicationContext
15:38:18,715 INFO [[/tunnel-web]:646] Closing WebApplicationContext of Spring FrameworkServlet 'SpringServlet'
18 janv. 2007 15:38:23 org.apache.coyote.http11.Http11BaseProtocol destroy
INFO: Arrêt de Coyote HTTP/1.1 sur http-8080
18 janv. 2007 15:38:23 org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: Failed shutdown of Apache Portable Runtime

This last APR message is just an info, desactivating the listener in server.xml suppress the message, but the server remain on.

[Vladislav Pernin]

It looks like the remaining Timer Thread contains two ThreadGroup :

• QuartzSchedulerefaultQuartzScheduler
• org.subethamail.smtp.server.SMTPServer ConnectionHandler Group

I tried to overload the set the property org.quartz.threadPool.makeThreadsDaemons to false that was initially to true.

I didn't find any smtp related parameter.

It had no effect.

[Brian Chan]

Works for me in same env.

[Vladislav Pernin]

Sorry, but it looks like I'm not alone :
http://www.liferay.com/web/guest/devzone/forums/message_boards/message/24510

Is there any change between 4.2.0 and 4.2.1 that could have a consequence on Quartz timers or Smtp listener ?

[Vladislav Pernin]

I made some more investigation :

• I tested on Fedora Core 3 Linux, 2.6.9 kernel, Tomcat shut down properly
• I was running under a SMP kernel
• I tested both 420 and 421 under a mono processor kernel on ubuntu dapper, still with jdk 1.5.0_10, and it looks like the 421 throw a Thread called "Thread [Timer-0] (Running)" which remain activ when trying to shut down tomcat, whereas the 420 does not throw a such thread.

Vladislav Pernin

[Vladislav Pernin]

Brian, which environment do you use : Linux, so which distribution and kernel, Windows, which version, jdk ?

[Felix Krull]

Same problem here:

Java 1.5.0_10-b03
liferay 4.2.1
debian sarge, kernel 2.4.25

[Ryan Russo]

Newbie to liferay so apologize in advance if commenting to this issue is inappropriate but, found this while googling and wanted to report I get the same behavior running the liferay 4.2.1 Tomcat 5.5.17 bundle under
Linux <HOST> 2.6.9-42.0.8.ELsmp #1 SMP Tue Jan 30 12:33:47 EST 2007 i686 athlon i386 GNU/Linux

I first noticed this same behavior running this liferay bundle under Windows XP pro.
My boss and me set the above bundle up separately and experienced this behavior (me on Windows and he on the above linux).

In sum: Tomcat process hangs until I kill Tomcat's PID with kill <PID>

AFTER SHUTDOWN with Tomcat no longer bound to the HTTP port I still see in ps:
<USER_NAME> 29588 1 1 16:42 ? 00:00:43 /usr/java/jdk1.5.0_09/bin/java -Xms128m -Xmx768m -Dfile.encoding=UTF8 -Duser.timezone=GMT -Djava.security.auth.login.config=/opt/portal/liferay421/conf/jaas.config -Djava.endorsed.dirs=/opt/portal/liferay421/common/endorsed -classpath :/opt/portal/liferay421/bin/bootstrap.jar:/opt/portal/liferay421/bin/commons-logging-api.jar -Dcatalina.base=/opt/portal/liferay421 -Dcatalina.home=/opt/portal/liferay421 -Djava.io.tmpdir=/opt/portal/liferay421/temp org.apache.catalina.startup.Bootstrap start

I assume the thing running under port 1099 relates to something started by liferay since I run multiple instances of Tomcat on a variety of platforms and have never seen this port being utlized (RMI ?).

I tracked it back to Tomcat using lsof -i tcp:1099

Did an plain kill <PID>

Then the Tomcat process was no longer running.

Repeatable every time.

[Brian Chan]

Checked in. Thanks James.

[Pan Zupa]

We have the same problem. We usebundle version 4.2.1 with tomcat in many environments: windows xp, Red Hat, Gentoo.
I tried to build liferay myself. I check out 4.2.x branch of you SVN and build it (ant all) into tomcat from your bundle. Still the same error: Fail to shutdown.

[Vladislav Pernin]

I just try the 4.2.2 jdk5 tomcat bundle, when shuting down, it says :
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable.run(ThreadPool.java:559)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
16:18:45,711 ERROR [MainServlet:937] com.liferay.portal.struts.ActionException: java.lang.IllegalThreadStateException
at com.liferay.portal.events.EventsProcessor._process(EventsProcessor.java:155)
at com.liferay.portal.events.EventsProcessor.process(EventsProcessor.java:76)
at com.liferay.portal.servlet.MainServlet.destroy(MainServlet.java:929)
at org.apache.catalina.core.StandardWrapper.unload(StandardWrapper.java:1331)
at org.apache.catalina.core.StandardWrapper.stop(StandardWrapper.java:1666)
at org.apache.catalina.core.ContainerBase.removeChild(ContainerBase.java:892)
at org.apache.catalina.core.StandardContext.removeChild(StandardContext.java:3095)
at org.apache.catalina.startup.ContextConfig.stop(ContextConfig.java:1082)
at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:276)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at org.apache.catalina.core.StandardContext.stop(StandardContext.java:4355)
at org.apache.catalina.core.ContainerBase.removeChild(ContainerBase.java:892)
at org.apache.catalina.startup.HostConfig.undeployApps(HostConfig.java:1164)
at org.apache.catalina.startup.HostConfig.stop(HostConfig.java:1135)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:312)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at org.apache.catalina.core.ContainerBase.stop(ContainerBase.java:1054)
at org.apache.catalina.core.ContainerBase.stop(ContainerBase.java:1066)
at org.apache.catalina.core.StandardEngine.stop(StandardEngine.java:447)
at org.apache.catalina.core.StandardService.stop(StandardService.java:512)
at org.apache.catalina.core.StandardServer.stop(StandardServer.java:743)
at org.apache.catalina.startup.Catalina.stop(Catalina.java:601)
at org.apache.catalina.startup.Catalina.start(Catalina.java:576)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:294)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:432)
Caused by: java.lang.IllegalThreadStateException
at java.lang.ThreadGroup.destroy(ThreadGroup.java:750)
at com.liferay.portal.events.GlobalShutdownAction.run(GlobalShutdownAction.java:164)
at com.liferay.portal.events.EventsProcessor._process(EventsProcessor.java:152)
... 28 more

22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable.run(ThreadPool.java:559)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:45 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Unexpected exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
at java.lang.Thread.run(Thread.java:595)
22 avr. 2007 16:18:47 org.apache.coyote.http11.Http11BaseProtocol destroy
INFO: Arrêt de Coyote HTTP/1.1 sur http-8080
22 avr. 2007 16:18:47 org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: Failed shutdown of Apache Portable Runtime
Failed to shut down: java.lang.NullPointerException

Has the problem been resolved with a ThreadGroup destroy ? I hope not and the diagnostic has gone a little bit futher than ours.

[Vladislav Pernin]

Hi all,

I have just downloaded the 4.3RC version and I saw the same stack when shutting down Liferay. What is the status of this issue ?

[General Chen]

I got the issue also: com.liferay.portal.struts.ActionException: java.lang.IllegalThreadStateException.
My env is Ubuntu 7.04, 2.6.20-16-generic #2 SMP Thu Jun 7 20:19:32 UTC 2007 i686 GNU/Linux,
java version "1.6.0_01"
Java(TM) SE Runtime Environment (build 1.6.0_01-b06)
Java HotSpot(TM) Client VM (build 1.6.0_01-b06, mixed mode, sharing)

I add following line into portal-ext.properties, the issue fixed.
shutdown.programmatically.exit=false