I recently wrote some logic for alerting on VMs offline outside of their start-stop schedules, based on the Heartbeat table. However, that produced some unexpected results, as the _ResourceId value was blank (i.e. empty string; not null) for a number of lines.
i.e. the below query returned 22 rows covering 19 distict VMs, all at different times.
Heartbeat
| where TimeGenerated > ago(7d) and _ResourceId == ''
| project _ResourceId, Computer, TimeGenerated
| order by TimeGenerated, Computer
Is this a bug, or could there be a legitimate reason for this?
Searching around so far I've found no bug reports for this, or mentions of why this behaviour may be seen.
Additional Info
Here's the query I'm using to monitor for offline VMs outside of legitimate stop/start actions.
let lastHealthyCheckin = ago(5m);
let pollingWindow = ago(7d);
Heartbeat
| where TimeGenerated > pollingWindow
and ResourceId != '' // this line added to avoid the "bug" mentioned here
| summarize LastCheckIn = max(TimeGenerated) by _ResourceId, Computer
| where _ResourceId !in (
AzureActivity
| where OperationNameValue in~ (
'MICROSOFT.COMPUTE/VIRTUALMACHINES/START/ACTION',
'MICROSOFT.COMPUTE/VIRTUALMACHINES/DEALLOCATE/ACTION'
)
and TimeGenerated > pollingWindow
| summarize LastStop = max(case(OperationNameValue =~ 'MICROSOFT.COMPUTE/VIRTUALMACHINES/DEALLOCATE/ACTION', TimeGenerated, datetime(null)))
, LastStart = max(case(OperationNameValue =~ 'MICROSOFT.COMPUTE/VIRTUALMACHINES/START/ACTION', TimeGenerated, datetime(null)))
by _ResourceId
| where LastStop > LastStart
| project _ResourceId
)
| where LastCheckIn < lastHealthyCheckin
| project Computer, _ResourceId, LastCheckIn