Repeating an extra when using a Dragonfly CompoundRule

Question

Using dragonfly2, the voice command framework, you can make a grammar like so:

chrome_rules = MappingRule(
    name='chrome',
    mapping={
        'down [<n>]': actions.Key('space:%(n)d'),
    },
    extras=[
        IntegerRef("n", 1, 100)
    ],
    defaults={
        "n": 1
    }
)

This lets me press space n times, where n is some integer. But what do I do if I want to use the same variable (n), multiple times in the same grammar? If I repeat it in the grammar, e.g. 'down <n> <n>' and then say something like "down three four", Dragonfly will parse it correctly, but it will only execute the actions.Key('space:%(n)d') with n=3, using the first value of n. How can I get it to execute it 3 times, and then 4 times using the same variable?

Ideally I don't want to have to duplicate the variable n, in the extras and defaults, because that seems like redundant code.

Answer 1

TL;DR: Your MappingRule passes data to your Action (e.g. Key, Text) in the form of a dictionary, so it can only pass one value per extra. Your best bet right now is probably to create multiple extras.

---

This is a side-effect of the way dragonfly parses recognitions. I'll explain it first with Action objects, then we can break down why this happens at the Rule level.

When Dragonfly receives a recognition, it has to deconstruct it and extract any extras that occurred. The speech recognition engine itself has no trouble with multiple occurrances of the same extra, and it does pass that data to dragonfly, but dragonfly loses that information.

All Action objects are derived from ActionBase, and this is the method dragonfly calls when it wants to execute an Action:

    def execute(self, data=None):
        self._log_exec.debug("Executing action: %s (%s)" % (self, data))
        try:
            if self._execute(data) == False:
                raise ActionError(str(self))
        except ActionError as e:
            self._log_exec.error("Execution failed: %s" % e)
            return False
        return True

This is how Text works, same with Key. It's not documented here, but data is a dictionary of extras mapped to values. For example:

{
    "n":    "3",
    "text": "some recognized dictation",
}

See the issue? That means we can only communicate a single value per extra. Even if we combine multiple actions, we have the same problem. For example:

{
    "down <n> <n>": Key("%(n)d") + Text("%(n)d"),
}

Under the hood, these two actions are combined into an ActionSeries object - a single action. It exposes the same execute interface. One series of actions, one data dict.

Note that this doesn't happen with compound rules, even if each underlying rule shares an extra with the same name. That's because data is decoded & passed per-rule. Each rule passes a different data dict to the Action it wishes to execute.

---

If you're curious where we lose the second extra, we can navigate up the call chain.

Each rule has a process_recognition method. This is the method that's called when a recognition occurs. It takes the current rule's node and processes it. This node might be a tree of rules, or it could be something lower-level, like an Action. Let's look at the implementation in MappingRule:

    def process_recognition(self, node):
        """
            Process a recognition of this rule.

            This method is called by the containing Grammar when this
            rule is recognized.  This method collects information about
            the recognition and then calls *self._process_recognition*.

            - *node* -- The root node of the recognition parse tree.
        """
        # Prepare *extras* dict for passing to _process_recognition().
        extras = {
                  "_grammar":  self.grammar,
                  "_rule":     self,
                  "_node":     node,
                 }
        extras.update(self._defaults)
        for name, element in self._extras.items():
            extra_node = node.get_child_by_name(name, shallow=True)
            if extra_node:
                extras[name] = extra_node.value()
            elif element.has_default():
                extras[name] = element.default

        # Call the method to do the actual processing.
        self._process_recognition(node, extras)

I'm going to skip some complexity - the extras variable you see here is an early form of the data dictionary. See where we lose the value?

extra_node = node.get_child_by_name(name, shallow=True)

Which looks like:

    def get_child_by_name(self, name, shallow=False):
        """Get one node below this node with the given name."""
        for child in self.children:
            if child.name:
                if child.name == name:
                    return child
                if shallow:
                    # If shallow, don't look past named children.
                    continue
            match = child.get_child_by_name(name, shallow)
            if match:
                return match
        return None

So, you see the issue. Dragonfly tries to extract one value for each extra, and it gets the first one. Then, it stuffs that value into a dictionary and passes it down to Action. Additional occurrences are lost.