Race condition on startup with autoUpgrade

• Type: Bug
• Status: Closed
• Resolution: No Longer Reproducible
• Affects Version/s: 7.3.X, Master
• Fix Version/s: Master
• Component/s: Database Upgrade Framework
• Labels:
  • 7.4-known-issues
  • QA-AUTOMATION
  • banderillas
  • boletus
  • butifarra
  • fc_21_11
  • fc_21_12
  • liferay-ga1-ce-740
  • liferay-ga1-dxp-7413
  • liferay-ga2-ce-741
  • liferay-ga4-ce-743
  • liferay-ga5-ce-743
  • liferay-ga6-ce-743
  • liferay-ga7-ce-743
  • liferay-u1-dxp-7413
  • liferay-u2-dxp-7413
• Fix Priority: 3
• Sprint: Upgrade|19.10-2.11|BolloPreñao, Upgrade|5.11-23.11|BoqueronesV, Upgrade|23.11-7.12|Bravas, Upgrade|14.12-28.12|Callos, Upgrade|4.1-18.1|CarneEnSarsa, Upgrade|1.18-2.1|CazónEnAdobo, Upgrade|2.1-2.15|Ensaladilla, Upgrade|2.17-3.3|Espeto, Upgrade|3.3-3.17|Gallinejas, Upgrade|3.17-3.31|Gazpacho
• QA Test Name:
  • LocalFile.PortalSmoke#SmokeWithDeveloperProperties
  • LocalFile.PortalSmokeUpgrade#ViewPortalSmokeArchiveAutoUpgrade621021
  • LocalFile.PortalSmokeUpgrade#ViewPortalSmokeArchiveAutoUpgrade70106
  • LocalFile.PortalSmokeUpgrade#ViewPortalSmokeArchiveAutoUpgrade7110
  • LocalFile.PortalSmokeUpgrade#ViewPortalSmokeArchiveAutoUpgrade71103
  • LocalFile.PortalSmokeUpgrade#ViewPortalSmokeArchiveAutoUpgrade730
  • LocalFile.SearchUpgrade#ViewClassicSearchPortletConfigurationArchiveAutoUpgrade70106
  • LocalFile.SearchUpgrade#ViewClassicSearchPortletConfigurationArchiveAutoUpgrade721
  • LocalFile.SearchUpgrade#ViewSearchPortletsConfigurationArchiveAutoUpgrade7110
  • LocalFile.SearchUpgrade#ViewSearchPortletsConfigurationArchiveAutoUpgrade721
• QA Test Score: 49

Race condition on startup when auto upgrade enabled (upgrade.database.auto.run=true):

Steps to Reproduce:

1. Import attached 7.1.10.3 database (MySQL)
2. Set portal-ext properties

   upgrade.database.auto.run=true
   module.framework.properties.dependency.manager.sync.timeout=1800
   module.framework.properties.initial.system.check.enabled=true
   

   + MySQL Connection properties

3. Start master against the 7.1 database to execute an upgrade on startup

Expected Result:
Upgrade on startup always completes without unsatisfied dependencies in the osgi bundles

Actual Result:
Upgrade on startup sometimes has unsatisfied dependencies in the osgi bundles

The failure rate is pretty low for an individual test (5%). But because we run multiple upgrade test executions regularly on CI (and on different database dumps), this issue can be seen every day in automation.

Can occurs on different bundles
Example

 [parallel] 20:49:58,303 WARN [main][SystemCheckOSGiCommands:137] Declarative Service Unsatisfied Component Checker check result: 
 [parallel] Bundle \{id: 356, name: com.liferay.redirect.web, version: 1.0.28}
 [parallel] Declarative Service {id: 5389, name: com.liferay.redirect.web.internal.application.list.RedirectPanelApp, unsatisfied references: 
 [parallel] \{name: Portlet, target: (javax.portlet.name=com_liferay_redirect_web_internal_portlet_RedirectPortlet)}
 [parallel] }
 [parallel] 20:49:58,303 INFO [main][SystemCheckOSGiCommands:114] Running "Missing Component Scanner". You can run this by itself with command "ds:missing" in gogo shell.

Additional Examples from CI:
https://test-1-2.liferay.com/job/test-portal-testsuite-upstream-batch(master)/38626/AXIS_VARIABLE=0,label_exp=!master/consoleText
https://test-1-4.liferay.com/job/test-portal-testsuite-upstream-batch(master)/35366/AXIS_VARIABLE=0,label_exp=!master/consoleText
https://test-1-12.liferay.com/job/test-portal-testsuite-upstream-batch(master)/32789/AXIS_VARIABLE=0,label_exp=!master/consoleText