logging in python and kedro, how to log only DEBUG info to a file and INFO to console

Question

I'm trying to configure logging so that INFO level messages go to the console and DEBUG level messages go to a file instead. So far, I am able to get working INFO to console and DEBUG to file, the problem is that the DEBUG is also being output to the console and I'm not sure why.

In particular, I'm using kedro to organize my project and it has some other features I'm trying to figure out. The following works insofar as the console gets INFO and only DEBUG level messages are saved in the file. but I cannot figure out how to prevent DEBUG level messages sent to the console.

I have the following set up in my logging.yml:

handlers:
  ...other built-in kedro handlers...
  debug_file_handler:
    class: logging.handlers.RotatingFileHandler
    level: DEBUG
    formatter: simple
    filename: logs/debug.log
    maxBytes: 10485760 # 10MB
    backupCount: 20
    encoding: utf8
    delay: True

loggers:
  kedro:
    level: INFO

  kedro_workbench:
    level: INFO

  DataSets:
    level: DEBUG
    handlers: [debug_file_handler]

root:
  handlers: [rich, info_file_handler, error_file_handler]

in my module I'm trying to use the following:

import logging
logger = logging.getLogger('DataSets')
...
output = pp.pformat(rss_feed)
logger.debug(output)

when I run my project, the all DEBUG content is written to the console and the file. any guidance would be greatly appreciated.

Answer 1

Add level: INFO to the rich handler. EDIT: And set root to debug.

handlers:
  info_file_handler:
    class: logging.handlers.RotatingFileHandler
    level: INFO
    formatter: simple
    filename: logs/info.log
    maxBytes: 10485760 # 10MB
    backupCount: 20
    encoding: utf8
    delay: True

  debug_file_handler:
    class: logging.handlers.RotatingFileHandler
    level: DEBUG
    formatter: simple
    filename: logs/debug.log
    maxBytes: 10485760 # 10MB
    backupCount: 20
    encoding: utf8
    delay: True

  rich:
    class: kedro.logging.RichHandler
    rich_tracebacks: True
    level: INFO  <- HERE #######################################
    markup: True

loggers:
  kedro:
    level: INFO

  root:
    handlers: [rich, info_file_handler, debug_file_handler]
    level: DEBUG  <- AND HERE ##################################