Better way for controlling the loading of plugins than disabling them for certain file types?

Question

I like using the LazyVim distribution when coding. But many of the features don't work very well or get in the way of editing a vimwiki file. So I want to disable certain plugins when opening editing a vimwiki file.

I've tried everything I could think of and have pored over the lazy.nvim README files but the only way I have found to prevent the loading of certain plugins that come bundled with LazyVim is do disable them like this:

  {
    "williamboman/mason-lspconfig.nvim",
    enabled = function()
      if vim.bo.filetype == "vimwiki" then
        return false
      end
    end,
  },
  {
    "hrsh7th/cmp-nvim-lsp",
    enabled = function()
      if vim.bo.filetype == "vimwiki" then
        return false
      end
    end,
  },
  ... etc., etc.

I was surprised to find I could not just lazy load certain plugin for different file types with something simpler like this in my config:

  {
    "williamboman/mason-lspconfig.nvim",
    ft = {"lua", "perl"},
  },

I could also could find no way to completely disable LazyVim when dealing with certian file types.

Am I overlooking some obvious method for customizing LavyVim based on FileType?