Show Pipeline Batch Status

We were running a series of data importing jobs and wanted to show the count of items in the pipeline status dialog:

The Show Pipeline Batch Status command is available when a Pipeline Batch item (under /sitecore/System/Data Exchange/(provider)/Pipeline Batches/) is selected and is currently running.

Our pipelines use IterableDataSettings objects to pass lists of items that need to be created or updated, and this is the count we would like to display - but it doesn't automatically just count the number of items in these objects and use it as the "Entities submitted" count.

With the "Show Pipeline Batch Status" button in the Content Editor as my starting point, I tracked back from there:

• Inspecting the button shows that the onclick event fires javascript:return scForm.invoke('dataExchange:showPipelineBatchStatusCommand()', event)
• This method name is a good place to start, so opening up dotPeek and searching for "PipelineBatchStatusCommand" leads to Sitecore.DataExchange.Local.Commands.ShowPipelineBatchStatusCommand. After validating the state of the pipeline batch, this method calls GetRunningPipelineBatchContext(...) on the pipeline batch runner to retrieve a PipelineBatchSummary object.
• A PipelineBatchSummary has an int EntitySubmitedCount (sp) property, so now we know where the count is stored!
• Doing a "find usages" on the property itself reveals an example usage in Sitecore.DataExchange.Providers.XConnect.Processors.PipelineSteps.BaseSubmitXConnectBatchStepProcessor.OnBatchProcessed(...). It makes sense that this kind of foundational method could be used to update this property consistently for any of the XConnect batches that are running.
• The OnBatchProcessed(...) method ultimately calls the equivalent of: pipelineContext?.PipelineBatchContext?.CurrentPipelineBatch?.GetPipelineBatchSummary() to access the PipelineBatchSummary object, and then does a simple += to increment that int property.

For reference - GetPipelineBatchSummary() used by the previous step is an extension method found in Sitecore.DataExchange.Extensions. It calls pipelineBatch.GetPlugin<PipelineBatchSummary>() - so this object is tracked in the pipeline batch itself using the standard DEF plugin system.

Updating our import job to increment this property allowed us to show the total number of items we had processed.