Uploaded image for project: 'PUBLIC - Liferay Portal Community Edition'
  1. PUBLIC - Liferay Portal Community Edition
  2. LPS-62426

Master node election process may result in multiple master nodes if the nodes lose connectivity during election

    Details

    • Fix Priority:
      4

      Description

      running these two scripts might demonstrate an inconsistency between the in memory "isMaster" and the lock table

      import com.liferay.portal.kernel.cluster.ClusterExecutorUtil;
      import com.liferay.portal.service.LockLocalServiceUtil;
      
      import com.liferay.portal.kernel.cluster.Address;
      import com.liferay.portal.model.Lock;
      import com.liferay.portal.kernel.util.Validator;
      
      try {
       	String _LOCK_CLASS_NAME = "com.liferay.portal.cluster.ClusterMasterExecutorImpl";
      
      	String localAddress = AddressSerializerUtil.serialize(
      		ClusterExecutorUtil.getLocalClusterNodeAddress());
      
      	String masterAddress = null;
      
      	for (int i = 0; i < 10; i++) {
      		try {
      			Lock lock = null;
      
      			if (masterAddress == null) {
      				lock = LockLocalServiceUtil.lock(
      					_LOCK_CLASS_NAME, _LOCK_CLASS_NAME,
      					localAddress);
      			}
      			else {
      				lock = LockLocalServiceUtil.lock(
      					_LOCK_CLASS_NAME, _LOCK_CLASS_NAME, masterAddress,
      					localAddress);
      			}
      
      			masterAddress = lock.getOwner();
      
      			Address address = AddressSerializerUtil.deserialize(masterAddress);
      
      			if (ClusterExecutorUtil.isClusterNodeAlive(address)) {
      				out.println("master is alive");
      
      				break;
      			}
      			else {
      				out.println("master is not alive");
      			}
      		}
      		catch (Exception e) {
      			out.println("Master Address: " + masterAddress);
      			out.println("Unable to acquire memory scheduler cluster lock" + e.getMessage());
      		}
      	}
      
      	out.println("Local Address:" +  AddressSerializerUtil.deserialize(localAddress));
      	out.println("Master Address: " +  AddressSerializerUtil.deserialize(masterAddress));
      }
      catch (Exception e) {
      	e.printStackTrace();
      
      	out.println(e.getMessage());
      }
      
      out.println(com.liferay.portal.kernel.cluster.ClusterMasterExecutorUtil.isMaster());
      

      this can occur if the cluster temporarily loses connectivity during the master election process because a node only attempts to connect once before nominating itself as the master node.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                andrew.betts Andrew Betts
                Reporter:
                andrew.betts Andrew Betts
                Participants of an Issue:
                Recent user:
                Esther Sanz
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Days since last comment:
                  3 years, 26 weeks, 4 days ago

                  Packages

                  Version Package
                  6.2.X EE
                  7.0.0 Alpha 4