Uploaded image for project: 'PUBLIC - Liferay Commerce'
  1. PUBLIC - Liferay Commerce
  2. COMMERCE-6526 Provide minimum viable implementation of Batch process progress
  3. COMMERCE-6527

Research batch engine and document necessary requirements and updates to show it in Dispatch(deprecated)

    Details

    • Type: Sub-Task
    • Status: Selected for Development
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: Master
    • Fix Version/s: Master
    • Component/s: Data Integration
    • Labels:
      None
    • Business Value:
      4
    • Story Points:
      2

      Description

      Keep this in mind:

      • LIST YOUR Thoughts directly in Liferay CONFLUENCE - upon the review we will cut what is not good and leave good conclusions to represent REQUIREMENTS. That way we will save time to write REQUIREMENTS (some parts may even become technical documentation)
      • we would be able to rename some dispatch classes, so if you detect that some Dispatch patterns are common with Batch bring naming proposal that could unite/generalize interface to both engines
      • keep in mind that dispatch-web module should be UI core for future family assembled with dispatch-service, batch-engine-service, scheduler and UI contributors like dispatch-talend-web (and probably dispatch-batch-web)
      • Salesforce example how API client application tracks execution of batches (our batch engine already support that concept but we have no any example. Tracking progress in dispatch-batch-web may be even easier). [Here is SF link|https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/asynch_api_code_walkthrough.htm].

      What should be analysed:

      • import - how to match progress (how to know exactly how many records CSV, XML, JSONL or XLSX) - xlsx has HSSFPOI that can quickly return length, CSV number of new-line characters etc.
      • how to extend batch-service to contain source[Size|Count] and target[Size|Count](we will need to provide new field(s) and add upgrade process) - does keeping this in DB makes sense regarding to performances
      • THINK - if batch is export than sourceSize is number of records in our DB (did batch used filtering criteria which may influence this number)
      • is providing dispatch-batch-web less complex (I see decoupling as benefit + we have knowledge how to do it in dispatch-talend-web)
      • see what should be added (if any) to batch-engine-api to cover all UI needs
      • see if there is way to catch errors that BREAK batch process and if we can preserve error and cause of error like line number or node order in XML

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              matija.petanjek Matija Petanjek
              Reporter:
              igor.beslic Igor Beslic
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

                Dates

                Created:
                Updated:

                  Packages

                  Version Package
                  Master