I am using the provided asp.net core logging and have application insights configured. I can see unhandled exceptions in application insights but I don't see things like logger.LogInformation("testing") anywhere in application insights. I do see the logs in the log stream though.
Additionally I am not seeing logs in the Visual Studio Application Insights screen. I created a hello world app which demonstrates that you don't see the logs in app insights.
https://github.com/devlife/AppInsightsSandbox
Any thoughts?

Can you describe how did you enable ApplicationInsights for capturing ILogger? If you are in the latest beta version of the SDK, ILogger traces are captured without explicit action. But it'll only capture Warning or above level logs.
Follow this doc to configure application insights to capture different log levels. https://learn.microsoft.com/en-us/azure/azure-monitor/app/ilogger#control-logging-level
If you are not on the latest SDK, then you need to explicitly enable application insights to capture ilogger logs, as per the doc above. Here also you need to configure logging level such that LogInformation is captured.