Creating Azure Stream Analytics job using the python SDK gives an error

Question

Creating Azure Stream Analytics job using the python SDK gives an error azure.core.exceptions.HttpResponseError: (BadRequest). The JSON provided in the request body is invalid. The required property 'datasource type' is missing from the request.

Here is the API call I am using to create the ASA job:

response = client.streaming_jobs.begin_create_or_replace(resource_group_name, job_name123, streaming_job={
        "location": "East US", 
        "properties": { 
        "sku": {  
            "name": "standard" 
        },
        "eventsLateArrivalMaxDelayInSeconds": 1, 
        "jobType": "edge", 
        "inputs": [
            {
                "name": "input",
                "properties": {
                    "type": "stream",
                        "serialization": {
                        "type": "JSON",
                        "properties": {
                            "fieldDelimiter": ",",
                            "encoding": "UTF8"
                        }
                    },
                    "datasource": {
                        "type": "GatewayMessageBus",
                        "properties": {
                        }
                    }
                }
            }
        ],
        "transformation": {
            "name": "samplequery",
            "properties": {
                "query": "select * from input"
            }
        },
    "package": {
        "storageAccount" : {
            "accountName": "*******",
            "accountKey": "*******"
        },
        "container": "sample"
    },
        "outputs": [
            {
                "name": "output",
                "properties": {
                    "serialization": {
                        "type": "JSON",
                        "properties": {
                            "fieldDelimiter": ",",
                            "encoding": "UTF8"
                        }
                    },
                    "datasource": {
                        "type": "GatewayMessageBus",
                        "properties": {
                        }
                    }
                }
            }
        ]
    }
})

Answer 1

Creating an Azure Stream Analytics job using the Python SDK gives an error

The error shows that your code has not correctly added data source properties.

You can use the below code to create a stream analytics job using azure python SDK.

Code:

from azure.identity import DefaultAzureCredential
from azure.mgmt.streamanalytics import StreamAnalyticsManagementClient

# Replace with your Azure subscription ID and resource group name
subscription_id = "xxxxx"
resource_group_name = "xxxxxx"

# Replace with your specific Stream Analytics job name
job_name = "stream326"

# Set up credentials and Stream Analytics client
credential = DefaultAzureCredential()
client = StreamAnalyticsManagementClient(credential, subscription_id)

response = client.streaming_jobs.begin_create_or_replace(resource_group_name, job_name, streaming_job={
    "location": "East US", 
    "properties": {
    "sku": {
      "name": "Standard"
    },
    "eventsOutOfOrderPolicy": "Drop",
    "outputErrorPolicy": "Drop",
    "eventsOutOfOrderMaxDelayInSeconds": 0,
    "eventsLateArrivalMaxDelayInSeconds": 5,
    "dataLocale": "en-US",
    "compatibilityLevel": "1.0",
    "inputs": [
      {
        "properties": {
          "type": "Stream",
          "datasource": {
            "type": "Microsoft.Devices/IotHubs",
            "properties": {
                "iotHubNamespace": "iothub-name",
                "sharedAccessPolicyName": "iothubowner",
                "sharedAccessPolicyKey": "key",
                "endpoint": "messages/events",
                "consumerGroupName": "$Default"
                }
        },
          "serialization": {
            "type": "Json",
            "properties": {
              "encoding": "UTF8"
            }
          }
        },
        "name": "inputtest"
      }
    ],
    "transformation": {
      "properties": {
        "streamingUnits": 1,
        "query": "SELECT *INTO outputtest FROM inputtest WHERE Temperature > 27"
      },
      "name": "transformationtest"
    },
    "outputs": [
      {
        "properties": {
            "datasource": {
            "type": "Microsoft.Storage/Blob",
            "properties": {
                "storageAccounts": [
                    {
                      "accountName": "your-storage-account-name",
                      "accountKey": "your-account-key"
                    }
                ],
                "container": "test",
                "pathPattern": "",
                "dateFormat": "yyyy/MM/dd",
                "timeFormat": "HH"
            }
            },
            "serialization": {
            "type": "Json",
            "properties": {
              "encoding": "UTF8"
            }
          }
        },
        "name": "outputtest"
      }
    ],
    "functions": []
  },
  "tags": {
    "key1": "value1",
    "randomKey": "randomValue",
    "key3": "value3"
  }
})
print(response.status())

Output:

Succeeded

Portal:

When I start my task it reflected the data in my storage account.

Portal:

Reference:

Quickstart - Create a Stream Analytics job by using the Azure portal - Azure Stream Analytics | Microsoft Learn