Uploaded image for project: 'PUBLIC - Liferay Portal Community Edition'
  1. PUBLIC - Liferay Portal Community Edition
  2. LPS-114511

[CCR] DXP is unable to establish connection to the follower cluster when PKI is configured unless the native realm is also enabled in Elasticsearch

    Details

      Description

      Master/7.3 is not affected.


      Affects:

      • Liferay Connector to Elasticsearch 7 v3.0.1 and below, fixed in: N/A*
      • Liferay Connector to Elasticsearch 6 (bundled with DXP 7.2), fixed in: N/A*

      * DXP subscribers can request the fix through Support in a form of a Hotfix LPKG/Hotfix.


      Steps to reproduce

      Attached (LPS-114511-configs.zip ) are the config files for DXP, ES6 and ES7 with generic paths in the respective properties that must be changed to match your env when testing.

      When testing, you must also deploy a version of the X-Pack Security Connector (on ES6) / Elasticsearch 7 Connector (on ES7) where LPS-112251 is already fixed on both DXP nodes.

      1. Setup CCR by following the docs (TBA)
      2. Update the Elasticsearch and DXP configs to enable PKI both on the Leader and the Follower ES nodes - See LPS-112251 for the steps and the related configs
      3. Update the following config for the CCR Module on the Follower DXP node to look like this:
        LIFERAY_HOME/osgi/configs/com.liferay.portal.search.elasticsearch.cross.cluster.replication.internal.configuration.ElasticsearchConnectionConfiguration-follower.config
        connectionId = "follower"
        clusterName = "LiferayElasticsearchCluster_FOLLOWER"
        transportAddresses = ["localhost:9301"]
        networkHostAddress = "https://localhost:9201"
        sslKeyPath = "/PATH/TO/ES_FOLLOWER_1/config/certs/elastic-certificates.key"
        sslCertificatePath = "/PATH/TO/ES_FOLLOWER_1/config/certs/elastic-certificates.crt"
        certificateFormat = "PEM"
        authenticationEnabled = B"false"
        sslCertificateAuthoritiesPaths = "/PATH/TO/ES_FOLLOWER_1/config/certs/ca.crt"
        transportSSLVerificationMode = "certificate"
        transportSSLEnabled = B"true"
        
      4. Start Leader ES
      5. Start Follower ES
      6. Start Leader/Remote DXP
      7. Start Follower/Local DXP

      Result :

      1. The "follower" connection shows an error in the Connections tab on the Search admin on the follower/local DXP node:
        NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{9lyTCDDQTCCHk76zt1hfww}{localhost}{127.0.0.1:9301}]]
        
      2. The following error is thrown in the log of the Follower ES node:
        [2020-05-28T14:08:02,635][WARN ][o.e.t.TcpTransport       ] [es-follower-node-1] exception caught on transport layer [Netty4TcpChannel{localAddress=0.0.0.0/0.0.0.0:9301, remoteAddress=/127.0.0.1:43216}], closing connection
        io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 455300000061000000000000000108004d3603010f582d466f756e642d436c7573746572244c696665726179456c6173746963736561726368436c75737465725f464f4c4c4f5745520016696e7465726e616c3a7463702f68616e647368616b650004bb91f302
        	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:556) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:510) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final]
        	at java.lang.Thread.run(Thread.java:834) [?:?]
        Caused by: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 455300000061000000000000000108004d3603010f582d466f756e642d436c7573746572244c696665726179456c6173746963736561726368436c75737465725f464f4c4c4f5745520016696e7465726e616c3a7463702f68616e647368616b650004bb91f302
        	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1182) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
        	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
        	... 15 more
        

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                timothy.pak Timothy Pak
                Reporter:
                tibor.lipusz Tibor Lipusz
                Participants of an Issue:
                Recent user:
                Clarissa Velazquez
                Engineering Assignee:
                Bryan Engler
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Days since last comment:
                  7 weeks, 5 days ago

                  Packages

                  Version Package
                  7.2.10 DXP FP7
                  7.2.X