Kapacitor alert using derivative value

295 Views Asked by At

I am trying to generate alert when the Cisco router's interface throughput exceeds certain value. Please have a look at my code.

I am using telegraph to collect the data. My measurement "monitor usage" contains multiple field keys and values. How to separate them and alert on each of them? Am I using "derivative" node correctly? Is it the field key I should use in the "derivative" node?

Is there a way to log each node output to help troubleshooting? In my case, how do I see "bytes_recv_per_sec" value as being calculated by the "alert" node?

What does "log" node do exactly? Where is the log file? I don't see much valuable information in "/var/log/kapacitor/kapacitor.log"

I have tried playing with different nodes with different parameter methods without much success. Thanks for any help.

dbrp "telegraf"."autogen"

stream
    |from()
        .measurement('monitor usage')
        .where(lambda: "agent_host" == '10.92.192.15')
    |window()
      .period(1m)
      .every(1m)
    |httpOut('dump')
    |stats(1m)
    |httpOut('stats')
    |derivative('Envision_Po1.891_ifHCInOctets')
        .unit(1s).nonNegative()
        .as('bytes_recv_per_sec')
    |alert()
        .warn(lambda: "bytes_recv_per_sec" > 40.0)
        .crit(lambda: "bytes_recv_per_sec" > 80.0)
         // post to logfile
        .log('/tmp/alerts.log')

At this stage I want to see "/tmp/alerts.log" to generate some alert message when the alert threshold is met. Eventually I want to send an alert email.

As per request, here is info about my database on the "monitor usage" measurement

> show series from "monitor usage";
key
---
monitor\ usage,agent_host=10.85.1.5,host=dzhao-trex,sw_name=it-na-wan-pro-01-11p.int.pason.com
monitor\ usage,agent_host=10.85.1.6,host=dzhao-trex,sw_name=it-na-wan-pro-02-11p.int.pason.com
monitor\ usage,agent_host=10.92.192.15,host=dzhao-trex,rtr_name=irt01-00rm255r2r1.int.pason.com
monitor\ usage,agent_host=10.92.192.17,host=dzhao-trex,sw_name=wsw01-00rm255r2r1.pason.com
monitor\ usage,agent_host=172.19.254.200,host=dzhao-trex,sw_name=ssw01-04rm120r7.int.pason.com
monitor\ usage,agent_host=192.168.195.1,host=dzhao-trex,sw_name=wrt01-06.int.pason.com

> show field keys from "monitor usage";
name: monitor usage
fieldKey                           fieldType
--------                           ---------
Envision_Po1.891_ifHCInOctets      integer
Envision_Po1.891_ifHCOutOctets     integer
Gi0/0/0_ifHCInOctets               integer
Gi0/0/0_ifHCOutOctets              integer
GigabitEthernet0/0/0_ifHCInOctets  integer
GigabitEthernet0/0/0_ifHCOutOctets integer
GigabitEthernet0/0/2_ifHCInOctets  integer
GigabitEthernet0/0/2_ifHCOutOctets integer
HE_Po1.892_ifHCInOctets            integer
HE_Po1.892_ifHCOutOctets           integer
Po1003_ifHCInOctets                integer
Po1003_ifHCOutOctets               integer
Rogers_Po1.894_ifHCInOctets        integer
Rogers_Po1.894_ifHCOutOctets       integer
Shaw_Po1.893_ifHCInOctets          integer
Shaw_Po1.893_ifHCOutOctets         integer
Te2/5/6_ifHCInOctets               integer
Te2/5/6_ifHCOutOctets              integer
Tunnel100_ifHCInOctets             integer
Tunnel100_ifHCOutOctets            integer
Tunnel1_ifHCInOctets               integer
Tunnel1_ifHCOutOctets              integer
Tunnel2_ifHCInOctets               integer
Tunnel2_ifHCOutOctets              integer
YYCIX_Gi0/0/1_ifHCInOctets         integer
YYCIX_Gi0/0/1_ifHCOutOctets        integer

> show tag keys from "monitor usage";
name: monitor usage
tagKey
------
agent_host
host
rtr_name
sw_name
0

There are 0 best solutions below