How to print hudi logs in aws emr serverless application

25 Views Asked by At

I have created a emr serverless application to run hudi spark job but neither of driver and executor logs are having logs related to hudi.

I tried setting applicationProperties of emr serverless app to enable logs for hudi but nothing worked.

{
  "runtimeConfiguration": [
    {
      "classification": "spark-driver-log4j2",
      "configurations": null,
      "properties": {
        "rootLogger.level": "debug",
        "logger.HoodieMultiTableStreamer.level": "debug",
        "logger.HoodieMultiTableStreamer.name": "org.apache.hudi.utilities.streamer",
        "logger.hudi.level": "debug",
        "logger.hudi.name": "org.apache.hudi"
      }
    },
    {
      "classification": "spark-executor-log4j2",
      "configurations": null,
      "properties": {
        "rootLogger.level": "debug",
        "logger.HoodieMultiTableStreamer.level": "debug",
        "logger.HoodieMultiTableStreamer.name": "org.apache.hudi.utilities.streamer",
        "logger.hudi.level": "debug",
        "logger.hudi.name": "org.apache.hudi"
      }
    }
  ]
}

Can someone tell whats missing? TIA

0

There are 0 best solutions below