How can I collect custom logs (e.g. from a file of an web app) to Log Analytics Workspace [Continuously] using Azure Monitoring Agent

Question

In the Data Collection Rule I have pointed the file from where the logs should be collected.

Then it collects the existing logs from that file. However, it prevents the new logs to be appended in the log file [of my web app]. As a result, I am not receiving the new logs in the Log Analytics Workspace as well.

I am expecting to get logs (existing and future ones) of my Web Application which being generated in a file in the Log Analytics Workspace.

What I did:

• Created a log Analytics Workspace
• Tables > New Custom Log (MMA-based) > Created one
• Agent > Data Collection Rule > Created one
• Configuration: Data Source: Custom text logs > Provided the table name and file path Destination: Set the destination of the Workspace
• Resources: Set the machine name from where the logs being collected from

Answer 1

How can I collect custom logs (e.g. from a file of an web app) to Log Analytics Workspace [Continuously].

The Data Collection Rule you created is not configured correctly to collect the new logs from the log file of your web app.

To collect the new logs from the log file from web app, you should require to configure the data collection rules to monitor the log file continuously and send the new logs to the Log Analytics Workspace.

You can modify the settings of data source to Custom log files and provide the path to the log file. Then, set the destination to your Log Analytics Workspace .

Make sure that the log file is accessible by the Log Analytics agent and that the agent has the required permissions to read the log file.

You can verify that the file path specified in the data collection rule and the agent has required permission to access the file. You can check the file permissions by right-clicking on the file and selecting Properties > Security.

1. Log in to the machine where the agent is installed.

2. Open the data collection rule configuration file located at below location.

   C:\Program Files\Microsoft Monitoring Agent\Agent\Configuration\DataSources\CustomLogs.conf
   

Verify the custom log file path under data source as below.

Once you have configured the data collection rule correctly, it should collect the new logs from the log file of your web app and send them to the Log Analytics Workspace.

Reference: Collect text logs with the Log Analytics agent in Azure Monitor