Querying child node fields in Gatsby

Question

I have the following GraphQL schema, which defines 3 types: a CondaPackage which hasmany CondaVersion, which hasmany CondaExecutable. I want to be able to query a CondaVersion and ask "how many CondaExecutables do you own which succeeded my analysis". Currently I've written a succeededExeCount and allExeCount which resolve this field by loading all children and manually counting the number of children that succeeded.

exports.createSchemaCustomization = ({ actions: { createTypes }, schema }) => {
  createTypes([
    schema.buildObjectType({
      name: "CondaPackage",
      fields: {
        succeededExeCount: {
          type: "Int!",
          resolve(source, args, context){
              // TODO
          }
        },
        allExeCount: {
          type: "Int!",
          resolve(source, args, context){
              // TODO
          }
        }
      },
      interfaces: ["Node"]
    }),
    schema.buildObjectType({
      name: "CondaVersion",
      fields: {
        succeededExeCount: {
          type: "Float!",
          resolve(source, args, context){
            const children = context.nodeModel.getNodesByIds({
              ids: source.children,
              type: "CondaExecutable"
            })
            return children.reduce((acc, curr) => acc + curr.fields.succeeded, 0)
          }
        },
        allExeCount: {
          type: "Int!",
          resolve(source, args, context){
            return source.children.length;
          }
        }
      },
      interfaces: ["Node"]
    }),
    schema.buildObjectType({
      name: "CondaExecutable",
      fields: {
        succeeded: {
          type: "Boolean!",
          resolve(source, args, context, info) {
            return source.fields.succeeded || false;
          }
        },
      },
      interfaces: ["Node"]
    })
  ])
}

My first problem is that this seems incredibly inefficient. For each CondaVersion I'm running a separate query for its children, which is a classic N+1 query problem. Is there a way to tell Gatsby/GraphQL to simply "join" the two tables like I would using SQL to avoid this?

My second problem is that I now need to count the number of succeeding children from the top level type: CondaPackage. I want to ask "how many CondaExecutables do your child CondaVersions own which succeeded my analysis". Again, in SQL this would be easy because I would just JOIN the 3 types. However, the only way I can currently do this is by using getNodesByIds for each child, and then for each child's child, which is n*m*o runtime, which is terrifying. I would like to run a GraphQL query as part of the field resolution which lets me grab the succeededExeCount from each child. However, Gatsby's runQuery seems to return nodes without including derived fields, and it won't let me select additional fields to return. How can I access fields on a node's child's child in Gatsby?

Answer 1

Edit

Here's the response from a Gatsby maintainer regarding the workaround:

Gatsby has an internal mechanism to filter/sort by fields with custom resolvers. We call it materialization. [...] The problem is that this is not a public API. This is a sort of implementation detail that may change someday and that's why it is not documented.

See the full thread here.

Original Answer

Here's a little 'secret' (not mentioned anywhere in the docs at the time of writing):

When you use runQuery, Gatsby will try to resolve derived fields... but only if that field is passed to the query's options (filter, sort, group, distinct).

For example, in CondaVersion, instead of accessing children nodes and look up fields.succeeded, you can do this:

const succeededNodes = await context.nodeModel.runQuery({
  type: "CondaExecutable",
  query: { filter: { succeeded: { eq: true } } }
})

Same thing for CondaPackage. You might try to do this

const versionNodes = await context.nodeModel.runQuery({
  type: "CondaVersion",
  query: {}
})

return versionNodes.reduce((acc, nodes) => acc + node.succeededExeCount, 0) // Error

You'll probably find that succeededExeCount is undefined.

The trick is to do this:

const versionNodes = await context.nodeModel.runQuery({
  type: "CondaVersion",
- query: {}
+ query: { filter: { succeededExeCount: { gte: 0 } } }
})

It's counter intuitive, because you'd think Gatsby would just resolve all resolvable fields on a type. Instead it only resolves fields that is 'used'. So to get around this, we add a filter that supposedly does nothing.

But that's not all yet, node.succeededExeCount is still undefined.

The resolved data (succeededExeCount) is not directly stored on the node itself, but in node.__gatsby_resolved source. We'll have to access it there instead.

const versionNodes = await context.nodeModel.runQuery({
  type: "CondaVersion",
  query: { filter: { succeededExeCount: { gte: 0 } } }
})

return versionNodes.reduce((acc, node) => acc + node.__gatsby_resolved.succeededExeCount, 0)

Give it a try & let me know if that works.

PS: I notice that you probably use createNodeField (in CondaExec's node.fields.succeeded?) createTypes is also accessible in exports.sourceNodes, so you might be able to add this succeeded field directly.