UnknownChannelHubException when Channel Hub tries to handle an incoming message from another node at startup

• Type: Bug
• Status: Closed
• Resolution: Fixed
• Affects Version/s: 6.2.10 EE GA1, 6.2.X EE
• Fix Version/s: 6.2.X EE, 7.0.0 Beta 5
• Component/s: Collaboration, User Notifications
• Labels:
  • ee-ts
  • liferay-fixpack-portal-93-6210
• Branch Version/s:
  6.2.x
• Backported to Branch:
  Committed
• Story Points: 1.25
• Fix Priority: 3
• Git Pull Request: https://github.com/brianchandotcom/liferay-portal/pull/35018

Description
In a cluster when a user is logged in and meanwhile a second node is starting up, an UnknownChannelHubException can be seen in the logs.

Reproduction
(May be refined later)
1. Build up a 2 node cluster environment
2. Deploy Notifications portlet (both nodes)
3. Stop node 1
4. With a user log in to node 2 and keep the session open (and the browser) for the next step.
5. Startup node 1

6. Watch the log:

08:45:31,336 ERROR [Incoming-2,LIFERAY-CONTROL-CHANNEL,E6520-GCDT-25509][ClusterRequestReceiver:211] Unable to invoke method {arguments=[20157, 20201, [email protected]28], methodKey=com.liferay.portal.kernel.notifications.ChannelHubManagerUtil.storeNot
ificationEvent(long,long,com.liferay.portal.kernel.notifications.NotificationEvent)}
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.liferay.portal.kernel.util.MethodHandler.invoke(MethodHandler.java:61)
        at com.liferay.portal.cluster.ClusterRequestReceiver.processClusterRequest(ClusterRequestReceiver.java:206)
        at com.liferay.portal.cluster.ClusterRequestReceiver.doReceive(ClusterRequestReceiver.java:73)
        at com.liferay.portal.cluster.BaseReceiver.receive(BaseReceiver.java:62)
        at org.jgroups.JChannel.invokeCallback(JChannel.java:749)
        at org.jgroups.JChannel.up(JChannel.java:710)
        at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1025)
        at org.jgroups.protocols.RSVP.up(RSVP.java:188)
        at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
        at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)
        at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
        at org.jgroups.protocols.pbcast.GMS.up(GMS.java:896)
        at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:245)
        at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:453)
        at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:763)
        at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:574)
        at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:147)
        at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:187)
        at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288)
        at org.jgroups.protocols.MERGE3.up(MERGE3.java:290)
        at org.jgroups.protocols.Discovery.up(Discovery.java:359)
        at org.jgroups.protocols.TP.passMessageUp(TP.java:1263)
        at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1825)
        at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1793)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
Caused by: com.liferay.portal.kernel.notifications.UnknownChannelHubException: No channel exists with company id 20157
        at com.liferay.portal.notifications.ChannelHubManagerImpl.getChannelHub(ChannelHubManagerImpl.java:261)
        at com.liferay.portal.notifications.ChannelHubManagerImpl.getChannelHub(ChannelHubManagerImpl.java:251)
        at com.liferay.portal.notifications.ChannelHubManagerImpl.storeNotificationEvent(ChannelHubManagerImpl.java:377)

If you want to reproduce the issue more than once, just shut down node 1 and then restart, the issue is consistently reproducible.

Expected result
Node starts up without errors.

Other details
In LPS-15045 a feature called Channel Hub was introduced which "integrated user login/logout and portal instance creation/deletion"
With LPS-49691 the channel hub was made cluster aware.