Details

      Description

      Configuring a component from control panel in a clustered environment only applies to the cluster node that handles the request.

      Steps to reproduce

      1. Configure a 2-node cluster from ee-7.0.x branch as follows

      • Enable cluster link in portal-ext.properties (both nodes):
        cluster.link.enabled=true
      • make them point to the same database
      • in node 2 set the gogo console to open a unused port:
        module.framework.properties.osgi.console=localhost:11312

      2. Although this would apply to any component, I'll illustrate it with S3 store:

      • Configure the use of S3 store for the document library in portal-ext.properties (both nodes):
        dl.store.impl=com.liferay.portal.store.s3.S3Store

      3. Start the cluster, first node 1, then node 2.

      • Make sure "Liferay Portal Cluster Multiple" module is active.
      • Make sure "Liferay Portal Cluster Single" module is not active. Otherwise, there is no cluster. If necessary, stop this bundle on both nodes

      4. Open 2 telnet sessions, one against node 1 (11311) and other for node 2 (11312)

      5. On each telnet, obtain the bundle id for S3 Store:

      g! lb | grep S3     
        122|Active     |    1|Liferay Portal Store S3 (2.0.2)
      

      6. Now list S3 bundle components on all telnet sessions (use the bundle Id returned by prevoius command on each node):

      scr:list 122

      Checkpoint: both nodes output something like

      g! scr:list 122
       BundleId Component Name Default State
          Component Id State      PIDs (Factory PID)
       [ 122]   com.liferay.portal.store.s3.S3FileCacheImpl  enabled
       [ 122]   com.liferay.portal.store.s3.S3KeyTransformerImpl  enabled
          [ 466] [active      ] 
       [ 122]   com.liferay.portal.store.s3.S3Store  enabled
      

      Note that only component 466 is active

      7. Open 2 browsers and access one node from each. Start session with Administrator role

      8. From node 1, go to Control panel | Configuration | System Settings | Foundation | S3StoreConfiguration

      9. Provide values for "Access Key", "Bucket Name" and "Secret Key" fields, then save.
      Now, some components declared by the S3 module should have been activated across all cluster nodes

      10. Repeat step 6

      Results

      Expected behavior:

      • Both nodes have activated the S3 components

      Observed behavior:

      • Only node 1 activated the S3 components:
      g! scr:list 122
       BundleId Component Name Default State
          Component Id State      PIDs (Factory PID)
       [ 122]   com.liferay.portal.store.s3.S3FileCacheImpl  enabled
          [2410] [active      ] com.liferay.portal.store.s3.configuration.S3StoreConfiguration
       [ 122]   com.liferay.portal.store.s3.S3KeyTransformerImpl  enabled
          [ 466] [active      ] 
       [ 122]   com.liferay.portal.store.s3.S3Store  enabled
          [2409] [active      ] com.liferay.portal.store.s3.configuration.S3StoreConfiguration
      

      You can see the details of newly activated components in node 1:

      g! scr:info 2409
      *** Bundle: com.liferay.portal.store.s3 (122)
      Component Description:
        Name: com.liferay.portal.store.s3.S3Store                                                                                                                                                                                                
        ...                                                                                                                                                                                          
        Configuration Pid: [com.liferay.portal.store.s3.configuration.S3StoreConfiguration]                                                                                                                                                      
        Services:                                                                                                                                                                                                                                  
                com.liferay.document.library.kernel.store.Store                                                                                                                                                                                    
        ...                                                                                                                                                                              
        Component Configuration:                                                                                                                                                                                                                   
          ComponentId: 2409
          State: active      
          SatisfiedReference: S3FileCache
      ...
               Properties:
                  accessKey = ********************
                  bucketName = support-es
                  cacheDirCleanUpExpunge = 7
                  cacheDirCleanUpFrequency = 100
                  component.id = 2410
                  component.name = com.liferay.portal.store.s3.S3FileCacheImpl
                  httpClientMaxConnections = 50
                  objectClass = [com.liferay.portal.store.s3.S3FileCache]
                  s3Region = us-east-1
                  s3StorageClass = STANDARD
                  secretKey = *********************************************
                  service.bundleid = 122
                  service.id = 6727
                  service.pid = com.liferay.portal.store.s3.configuration.S3StoreConfiguration
                  service.scope = bundle
          SatisfiedReference: S3KeyTransformer
      ...
           Properties:
              accessKey = ********************
              bucketName = support-es
              cacheDirCleanUpExpunge = 7
              cacheDirCleanUpFrequency = 100
              component.id = 2409
              component.name = com.liferay.portal.store.s3.S3Store
              httpClientMaxConnections = 50
              s3Region = us-east-1
              s3StorageClass = STANDARD
              secretKey = *********************************************
              service.pid = com.liferay.portal.store.s3.configuration.S3StoreConfiguration
              store.type = com.liferay.portal.store.s3.S3Store
      

      Same applies for the other component in node 1

      g! scr:info 2410
      *** Bundle: com.liferay.portal.store.s3 (122)
      Component Description:
        Name: com.liferay.portal.store.s3.S3FileCacheImpl
      ...
        Configuration Pid: [com.liferay.portal.store.s3.configuration.S3StoreConfiguration]
        Services: 
                com.liferay.portal.store.s3.S3FileCache
      ...
        Component Configuration:
          ComponentId: 2410
          State: active      
      ...
            Properties:
              accessKey = ********************
              bucketName = support-es
              cacheDirCleanUpExpunge = 7
              cacheDirCleanUpFrequency = 100
              component.id = 2410
              component.name = com.liferay.portal.store.s3.S3FileCacheImpl
              httpClientMaxConnections = 50
              s3Region = us-east-1
              s3StorageClass = STANDARD
              secretKey = *********************************************
              service.pid = com.liferay.portal.store.s3.configuration.S3StoreConfiguration
      

      However, node 2 still has components not enabled:

      g! scr:list 122 
       BundleId Component Name Default State
          Component Id State      PIDs (Factory PID)
       [ 122]   com.liferay.portal.store.s3.S3FileCacheImpl  enabled
       [ 122]   com.liferay.portal.store.s3.S3KeyTransformerImpl  enabled
          [ 466] [active      ] 
       [ 122]   com.liferay.portal.store.s3.S3Store  enabled
      

      As a result, document library will use S3Store when running from node 1 and FileSystemStore when running from node 2.

      Technical notes

      This affects not only to component activation but also to any configuration change: the changes will be applied only to the "local" components for the node that dispatched the HTTP request sent from System Settings

      Root cause (I)

      The root cause of this is that clustered configuration is not listening for incoming cluster messages. ConfigurationMessageListener has an unsatisfied dependency for a ReloadablePersistenceManager, so it's not enabled.

      Let's take a look to the clustered configuration module

      g! lb | grep Cluster
         96|Active     |    1|Liferay Portal Cluster Multiple (1.0.0)
        106|Active     |    1|Liferay Portal Configuration Cluster (2.0.1)
      
      g! scr:list 106
       BundleId Component Name Default State
          Component Id State      PIDs (Factory PID)
       [ 106]   com.liferay.portal.configuration.cluster.internal.ConfigurationSynchronousConfigurationListener  enabled
          [ 398] [active      ] 
       [ 106]   com.liferay.portal.configuration.cluster.internal.messaging.ConfigurationClusterConfigurator  enabled
          [ 399] [active      ] 
       [ 106]   com.liferay.portal.configuration.cluster.internal.messaging.ConfigurationMessageListener  enabled
          [ 400] [unsatisfied reference] 
      

      If you look at the component having the unsatisfied reference:

      g! scr:info 400
      *** Bundle: com.liferay.portal.configuration.cluster (106)
      Component Description:
        Name: com.liferay.portal.configuration.cluster.internal.messaging.ConfigurationMessageListener
      ...
        Services: 
                com.liferay.portal.kernel.messaging.MessageListener
      ...
        Component Configuration:
          ComponentId: 400
          State: unsatisfied reference
      ...
          UnsatisfiedReference: ReloadablePersistenceManager
            Target: null
            (no target services)
      ...
      

      There is a ReloadablePersistenceManager component that is not being found

      Root cause (II)

      After fixing first root cause, configuration was still not being propagated to the other nodes.

      Once dependencies are satisified, there is a problem in the ConfigurationMessageListener. This component is updating the configuration only in the PersistenceManager but not in the osgi runtime.

      As a result, if you change the configuration via System Settings in node 1, node 2 UI shows the right values but node2 runtime is not using them, according to the scr:info command.

      The cause of this is that ConfigurationMessageListener.java is updating the component configuration via update() method:

      			for (Configuration configuration : configurations) {
      				if (type == ConfigurationEvent.CM_DELETED) {
      					configuration.delete();
      				}
      				else {
      					configuration.update();
      				}
      			}
      

      The Configuration interface defines different semantics for update() and update(Dictionary<String, ?> properties) methods. A more detailed description can be found in sections 104.12.3.10 and 104.12.3.11 (page 90) in the Enterprise OSGi r6.

      At the time of this writing, we are using apache felix 1.8.8. ConfigurationImpl.java shows that the 2 update() methods are a bit different. More precisely, they call ConfigurationManager.updated() in a different way.

      According to our tests and debug sessions, configuration is only being updated if we call configuration.update(properties) from the message listener. This seems due to the fact that this call fires a configuration event from the ConfigurationManager which ends up notifiyng the ConfigurationListeners. Opposedly, the call to update() is not firing that event.

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since last comment:
                3 years, 10 weeks, 5 days ago

                Packages

                Version Package
                7.0.1 CE GA2