iOS keyboard extension in SwiftUi with UIHostingcontroller high memory consumption and peak

Question

I am using UIHostincontroller inside of UIInputviewcontroller so I can use SwiftUI in iOS keyboard extension. Not sure if this is the root cause for my problems. When I start the keyboard, I already have around 25MB memory. With the first memory warning at 30MB I feel this is much. If I switch keyboards and reopen multiple times my keyboard memory usage is constantly growing without dropping after time. On reopening there is first a peak and then it does not go back to 25MB. After third time reopening it peaks over 70MB and the keyboard gets killed. Deployed via TestFlight I am able to repeat this multiple times and after 5 or 6 times I can use the keyboard without any issues. I have a view which displays images in 64x64 pixels. Same memory behaviour if he comment out the whole image Hstack. Furthermore interestingly after the 5 to 6 reopening the apps runs smoothly with 100+ images in LazyHStack.

The code:

class KeyboardViewController: UIInputViewController {
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        let keyboardUIHostingController = UIHostingController(rootView: KeyboardWrapper(uiInputViewController: self))
        
        keyboardUIHostingController.view.translatesAutoresizingMaskIntoConstraints = false
        view.addSubview(keyboardUIHostingController.view)
        
        keyboardUIHostingController.view.bottomAnchor.constraint(equalTo: view.safeAreaLayoutGuide.bottomAnchor).isActive = true
        keyboardUIHostingController.view.topAnchor.constraint(equalTo: view.safeAreaLayoutGuide.topAnchor).isActive = true
        keyboardUIHostingController.view.leadingAnchor.constraint(equalTo: view.safeAreaLayoutGuide.leadingAnchor).isActive = true
        keyboardUIHostingController.view.rightAnchor.constraint(equalTo: view.safeAreaLayoutGuide.rightAnchor).isActive = true
    }
}

struct KeyboardWrapper: View {
    weak var uiInputViewController: UIInputViewController?
    @State public var colorTheme: ColorTheme = ColorCandy()
    @State public var inputTextViewData = ""
    @State public var inputTextViewSelectedRange = NSRange(location: 0, length: 0)
    @State public var images: [ImageScrollViewItem] = []
    @State private var selectedTag: String = "revised"
    let outerpadding: CGFloat = 4
    
    var body: some View {
        HStack(spacing: 0) {
            VStack(spacing: 0) {
                TagHorizontalScrollView(selectedTag: $selectedTag, colorTheme: $colorTheme)
                    .padding(EdgeInsets(top: 4, leading: 0, bottom: 4, trailing: 0))
                InputTextViewWrapper(inputTextViewData: $inputTextViewData, inputTextViewSelectedRange: $inputTextViewSelectedRange)
                    .frame(height: UIFont.systemFont(ofSize: 16).lineHeight * 4)
                    .padding(EdgeInsets(top: 4, leading: 0, bottom: 12, trailing: 0))
                ImageScrollView(images: $images)
                    .padding(EdgeInsets(top: 0, leading: 0, bottom: 12, trailing: 0)).frame(height: 64)
                QWERTYKeyboard(uiInputViewController: uiInputViewController, inputTextViewData: $inputTextViewData, inputTextViewSelectedRange: $inputTextViewSelectedRange, selectedTag: $selectedTag, images: $images, colorTheme: $colorTheme)
            }.padding(outerpadding).onAppear {
                PhotoAlbumWrapper.loadImagesIntoView(images: $images)
            }
        }.background(LinearGradient(gradient: Gradient(colors: [colorTheme.backgroundColorTop, colorTheme.backgroundColorBottom]), startPoint: .top, endPoint: .bottom))
    }
}

I thought about SwiftUI views which are not deleted from memory after closing the keyboard. This function already helps in lowering the memory usage after the peak. But I never get back to the initial 25MB. It constantly grows.

override func viewDidDisappear(_ animated: Bool) {
        super.viewDidDisappear(animated)
        
        for subview in view.subviews {
            subview.removeFromSuperview()
        }
        keyboardUIHostingController?.rootView = AnyView(EmptyView())
    }

Finally I found a solution which lowers the peak by adding a delay in SwiftUI startup. This is ugly for the UX and furthermore it is not reliable.

override func viewDidLoad() {
        keyboardUIHostingController = UIHostingController(rootView: AnyView(EmptyView()))
        keyboardUIHostingController?.view.translatesAutoresizingMaskIntoConstraints = false
        view.addSubview(keyboardUIHostingController!.view)
        
        // Replace with the actual view after a delay
        DispatchQueue.main.asyncAfter(deadline: .now() + 0.5) { [self] in
            keyboardUIHostingController?.rootView = AnyView(KeyboardWrapper())
            keyboardUIHostingController?.view.bottomAnchor.constraint(equalTo: view.safeAreaLayoutGuide.bottomAnchor).isActive = true
            keyboardUIHostingController?.view.topAnchor.constraint(equalTo: view.safeAreaLayoutGuide.topAnchor).isActive = true
            keyboardUIHostingController?.view.leadingAnchor.constraint(equalTo: view.safeAreaLayoutGuide.leadingAnchor).isActive = true
            keyboardUIHostingController?.view.rightAnchor.constraint(equalTo: view.safeAreaLayoutGuide.rightAnchor).isActive = true
        }
    }

I know that there is many lines of codes behind the KeyboardWrapper and saying something in this lines blows up the memory usage, but this essentially occurs as soon as I have the UIHostingController in play. As more code I am commenting out as lower is the memory usage and peaks, but there is still the general problem. Any help or advise would be appreciated much!