Dealing with Azure Storage Queue list without any QueuMessage in ADF ForEach

Question

I am using WebActivity to get QueueMessage from Azure Blob Storage and pass the output to ForEeach to use each QueueMessage in subsequent pipeline process.The pipeline is on schedule run and this is the Dynamic Expression I use in ForEach Items:

@json(xml(activity('get_queue_message').output.Response)).QueueMessagesList.QueueMessage

It works fine when there are MessageText in the queue. But once the queue is cleared and the webactivity returns empty QueueMessageList as per below, the ForEach will start to fail

{
 "Response": "<?xml version=\"1.0\" encoding=\"utf-8\"?><QueueMessagesList />",
 "ADFWebActivityResponseHeaders": {
     "Transfer-Encoding": "chunked",
     "x-ms-request-id": "requestid",
     "x-ms-version": "2020-04-08",
     "Cache-Control": "no-cache",
     "Date": "Tue, 13 Jun 2023 07:53:30 GMT",
     "Server": "Windows-Azure-Queue/1.0;Microsoft-HTTPAPI/2.0",
     "Content-Type": "application/xml"
 },
 "effectiveIntegrationRuntime": "AutoResolveIntegrationRuntime (Southeast Asia)",
 "executionDuration": 0,
 "durationInQueue": {
     "integrationRuntimeQueue": 0
 },
 "billingReference": {
     "activityType": "ExternalActivity",
     "billableDuration": [
         {
             "meterType": "AzureIR",
             "duration": 0.016666666666666666,
             "unit": "Hours"
         }
     ]
 }
}

ForEach Failure as follows:

The expression 'length(json(xml(activity('get_queue_message').output.Response)).QueueMessagesList.QueueMessage)' cannot be evaluated because property 'QueueMessage' cannot be selected.

What is the correct way to check for the QueueMessageList before running ForEach. If the queue is empty, i would like to stop the ForEach without triggering any error.

Thank you.

Answer 1

To check if the QueueMessagesList is empty before running the ForEach activity, you can use the if function to conditionally execute the ForEach activity only if the QueueMessagesList is not empty. Since the ForEach activity cannot be used directly inside an If Condition activity, you can design a two level pipeline by using the combination of If Condition and Execute Pipeline activities to achieve the requirement. In outer pipeline take the web activity and if activity. Inside if activity take the execute pipeline activity.

Outer Pipeline:

• Take the Web activity and then if condition activity. Give the expression as,

@contains(json(xml(activity('get_queue_message').output.Response)).QueueMessagesList,'QueueMessage')

This expression will return true if the QueueMessagesList property contains one or more QueueMessage elements, and false if the QueueMessagesList property is empty. that is, it does not contain any QueueMessage elements.

• You can also give the expression as

@not(empty(json(xml(activity('get_queue_message').output.Response)).QueueMessagesList))

This expression returns true if there are messages in the QueueMessagesList property, and false if there are no messages.

• In true section of If activity you can add execute pipeline activity and in false branch, leave it as empty (no activities inside it).

Inner Pipeline:

• In the execute pipeline activity, you can invoke the pipeline that contains web activity and for-each activity.

This way, you can execute the ForEach activity only if the QueueMessagesList is not empty.