How to use Theme colorScheme correctly

Question

I am building an app that supports [light/dark/system] modes. I am using BLoC to switch between different modes. Switching between light and dark mode is easy and working fine. The problem is with the system mode. I set the theme and darkTheme properties of MaterialApp with my ThemeData. I have an enum called AppMode with {light, dark, system} which I can use to identify the current mode and three buttons that updates the value of a var called themeMode with type AppMode. I have this other function called init() that basically updates the values of the colors I am using according to the themeMode whether to use fixed colors or use the colors from the Theme.of(context).colorScheme.

static void init(BuildContext context) {
    containers = Global.themeMode != AppMode.system
        ? Global.isDark!
            ? const Color.fromARGB(255, 22, 22, 22)
            : Colors.white
        : Theme.of(context).colorScheme.onPrimaryContainer;
    secondaryContainer = Global.themeMode != AppMode.system
        ? Global.isDark!
            ? Colors.grey[900]
            : const Color.fromARGB(255, 240, 240, 240)
        : Theme.of(context).colorScheme.onSecondaryContainer;
    text = Global.themeMode != AppMode.system
        ? Global.isDark!
            ? Colors.white
            : Colors.black
        : Theme.of(context).colorScheme.primary;
    shadowColor = Global.themeMode != AppMode.system
        ? Global.isDark!
            ? Colors.white
            : Colors.black54
        : Theme.of(context).colorScheme.onPrimary;
  }

the final part is to use those colors in my widgets and wrap them with BLoC so that when I fire the BLoC from the button, it updates the necessary widgets with the new colors. The problem is that if I set the color of let's say container to Theme.of(context).colorScheme.onPrimaryContainer, it gets the color but makes the widget fixed to that color and doesn't change whenever the user changes the theme mode from the his/her phone manually. To ensure I did everything correctly, I did some printing here and there to ensure that all the if conditions goes to the right place. I noticed that if I used Theme.of(context).colorScheme.onPrimaryContainer directly to a widget, it behaves the way I expect. So my theory is that my function saves the color as it is but not the behaviour of changing when the user changes the theme. Is there anyway for this to work?