Convert Row Count to INT in Azure Data Factory

Question

I am trying to use a Lookup Activity to return a row count. I am able to do this, but once I do, I would like to run an If Statement against it and if the count returns more than 20MIL in rows, I want to execute an additional pipeline for further table manipulation. The issue, however, is that I can not compare the returned value to a static integer. Below is the current Dynamic Expression I have for this If Statement:

@greater(int(activity('COUNT_RL_WK_GRBY_LOOKUP').output),20000000)

and when fired, the following error is returned: { "errorCode": "InvalidTemplate", "message": "The function 'int' was invoked with a parameter that is not valid. The value cannot be converted to the target type", "failureType": "UserError", "target": "If Condition1", "details": "" }

Is it possible to convert this returned value to an integer in order to make the comparison? If not, is there a possible work around in order to achieve my desired result?

Answer 1

Looks like the issue is with your dynamic expression. Please correct your dynamic expression similar to below and retry.

• If firstRowOnly is set to true : @greater(int(activity('COUNT_RL_WK_GRBY_LOOKUP').output.firstRow.propertyname),20000000)

• If firstRowOnly is set to false : @greater(int(activity('COUNT_RL_WK_GRBY_LOOKUP').output.value[zero based index].propertyname),20000000)

• The lookup result is returned in the output section of the activity run result.

• When firstRowOnly is set to true (default), the output format is as shown in the following code. The lookup result is under a fixed firstRow key. To use the result in subsequent activity, use the pattern of @{activity('MyLookupActivity').output.firstRow.TableName}. Sample Output JSON code is as follows:

{
    "firstRow":
    {
        "Id": "1",
        "TableName" : "Table1"
    }
}

• When firstRowOnly is set to false, the output format is as shown in the following code. A count field indicates how many records are returned. Detailed values are displayed under a fixed value array. In such a case, the Lookup activity is followed by a Foreach activity. You pass the value array to the ForEach activity items field by using the pattern of @activity('MyLookupActivity').output.value. To access elements in the value array, use the following syntax: @{activity('lookupActivity').output.value[zero based index].propertyname}. An example is @{activity('lookupActivity').output.value[0].tablename}. Sample Output JSON Code is as follows:

{
    "count": "2",
    "value": [
        {
            "Id": "1",
            "TableName" : "Table1"
        },
        {
            "Id": "2",
            "TableName" : "Table2"
        }
    ]
}

Hope this helps.

Answer 2

Do this - when you run the debugger look at the output from your lookup. It will give a json string including the alias for the result of your query. If it's not firstrow set then you get a table. But for first you'll get output then firstRow and then your alias. So that's what you specify.

For example...if you put alias of your count as Row_Cnt then... @greater(activity('COUNT_RL_WK_GRBY_LOOKUP').output.firstRow.Row_Cnt,20000000)

You don't need the int function. You were trying to do that (just like I was!) because it was complaining about datatype. That's because you were returning a bunch of json text as the output instead of the value you were after. Totally makes sense after you realize how it works. But it is NOT intuitively obvious because it's coming back with data but its string stuff from json, not the value you're after. And functions like equals are just happy with that. It's not until you try to do something like greater where it looks for numeric value that it chokes.