Benthos - How do you reference keywords from one processor to another processor in the same pipeline?

Question

I am new to Benthos and trying to learn how it works. I am trying to input a JSON file with some key:value pairs, add a new field and POST that new field to get a response.

pipeline:
  processors:
    - mapping: |
      root.id = this.id
      root.submitName = "testingAcc"
      root.cost = this.cost
            
      let idLen = root.id.length()
      let nameLen = root.submitName.length()
      root.send = $idLen + root.id + $nameLen + root.submitName + this.cost
                  
    - branch:
      request_map: |
        root = this.send
      processors:
        - http:
          url: "<http link>"
          verb: POST
      result_map: |
        root.response = this.content().string()

---

Example

Input = {"id": "1", "cost": "500"} Mapping = {"id": "1", "submitName": "testingAcc", "cost": "500", "send": "1110testingAcc500"}

Value that sends via POST = 1110testingAcc500

Branch = {"id": "1", "submitName": "testingAcc", "cost": "500", "send": "1110testingAcc500", "response": "OK"}

---

However, when I run this I get an error saying that the request_map line (root = this.send) did not find the expected key. I thourght this would work as I mapped the send value so it could be referenced in the branch processor. I have tried different ways to call the keyword (like root = this.send, root = root.send, etc.) but all the same error.

Hopefully I explained this well. I tried googling the error and reading documentation but couldn't find an answer or any examples to help me understand. Any help would be greatly appreciated!

Answer 1

I think you might be seeing this error: Branch error: request mapping failed: failed assignment (line 1): unable to reference message as structured (with 'this.send'): parse as json: invalid character 'b' looking for beginning of value. The reason is because the this keyword mandates that the current in-flight message is in JSON format (if it was in JSON format, but the key didn't exist, you'd get a null value in root).

In your case, I think the mapping processor is failing and then the in-flight message gets passed unchanged to the next processor in the pipeline (the branch processor). You should first ensure that your message is valid JSON by using a log processor (if it's not, you can use the content() function in a mapping processor to extract the message data and parse it somehow) and then you need to fix the data type issues in your mapping processor. Bloblang does not allow implicit conversions from integer to string, so you'll need to use $idLen.string() and $nameLen.string() (and maybe this.cost.string() if cost happens to be numeric). Try using benthos blobl server to prototype your bloblang mappings.

Answer 2

As Mihai pointed out, the issue was Here is a working version of your config:

input:
  generate:
    mapping: 'root = {"id": "1", "cost": "500"}'
pipeline:
  processors:
    - mapping: "root.id = this.id\nroot.submitName = \"testingAcc\"\nroot.cost = this.cost\n      \nlet idLen = root.id.length()\nlet nameLen = root.submitName.length()\nroot.send = $idLen.string() + root.id + $nameLen.string() + root.submitName + this.cost\n          \n"
    - branch:
        request_map: |
          root = this.send
        processors:
          - http:
              url: https://a.com
              verb: POST
        result_map: |-
          root.response = content().string()
output:
  stdout: {}