Uploaded image for project: 'PUBLIC - Liferay Portal Community Edition'
  1. PUBLIC - Liferay Portal Community Edition
  2. LPS-97196

As a marketer, I can review the test details before I run the test on DXP

    Details

      Description

      Motivation

      See Epic (LPS-87072) and Spike (--LPS-95354--) for general AB testing motivation.

      The purpose of an AB test is to find the page experience that maximizes a goal for a given audience, by comparing the performance of different experience variants during a time. 

      Before the test is run, the marketer must decide which percentage of the audience will view each variant, and the desired confidence level of the test. 

      Scope

      This story covers the definition of the traffic split per variant and the confidence level. 

      This story does not cover:

      • the estimated time to declare winner (will be implemented as part of LPS-96782
      • the test schedule
      • estimated visitors per variant

      Design

      • See Figma for reference

      Acceptance Criteria

      1. The Review and Start button is enabled

      • Given an AB test displayed in the AB test sidebar
      • When the test contains at least two variants (control + 1)
      • Then the Review and Start Test button is enabled

      2. The review and start test dialog is opened

      • Given the Review and Start Test button is enabled
      • When the Review and Start Test button is clicked
      • Then the Review and Start Test dialog is opened, with:
        • A traffic split slider for each variant
        • A confident level slider
        • Cancel and Run buttons (see Figma)

      3.The review and start test dialog is closed without running

      • Given the Review and Start test dialog is opened
      • When the Cancel or close button is clicked
      • Then the dialog is closed and no changes are applied to the test

      4. The traffic split adds up to 100%

      • Given the Review and Start test dialog is opened
      • When the traffic split is edited for one variant
      • Then the other traffic sliders adapt so that the traffic split always adds up 100%

      5. The traffic split and the confidence level details are shown in the sidebar

      • Given the Review and Start test dialog is opened and values have been set for the traffic split and confidence level
      • When the Run test is clicked
      • Then:
        • The test status changes to Run
        • The Confidence Level is shown in the test details (with the goal)
        • The Traffic Split % is shown close to each variant's name, replacing the edit icon

      6. The traffic split and confidence level data are sent properly to Analytics Cloud

      • Given the test has been started with certain traffic split and confidence level
      • When the test info is viewed on AC
      • Then the same traffic split and confidence level are found

      Technical Aspects

      • The confidence level is a test property, and will be stored inside the test type settings, as we do with the goal and target.
      • The traffic split is a variant property, and will be stored in the SegmentsExperimentRel entity, in the split field.
      • In all cases, values will be stored normalized to 1 (e.g. 100% --> 1, 50% --> 0.5)

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Packages

                  Version Package
                  Master