Elasticsearch: How do I reference a nested field name using Painless script in a curl command

Question

I have a painless script that works in the console but fails in curl. The script references a nested field (event.orinal).

In the Kibana console, this statement works fine: ctx._source.event.remove('original')

In a curl statement, I get this result:

...
"ctx._source.event.remove(original)",
"                         ^---- HERE"
]
...
"caused_by":{"type":"illegal_argument_exception","reason":"cannot resolve symbol [original]"}},"status":400}

We have an elastic subscription, but support has repeatedly stated that help with this is 'out of scope' because curl is not an elastic product.

Any help would be greatly appreciated.

Full cURL request and response:

curl -XPOST "https://<server>:9200/<index>/_update_by_query?conflicts=proceed" -u <creds> --cacert <certpath> -H "Content-Type: application/json" -d '
{
    "script" : "ctx._source.event.remove(original)",
    "query" : {
        "exists": { "field": "event.original" }
    }
}'

{
  "error":
    {
      "root_cause":[
        {
          "type":"script_exception",
          "reason":"compile error",
          "script_stack":
            [
              "ctx._source.event.remove(original)",
              "                         ^---- HERE"
            ],
          "script":"ctx._source.event.remove(original)",
          "lang":"painless",
          "position":
              {
                "offset":25,
                "start":0,
                "end":34
              }
        }
      ],
      "type":"script_exception",
      "reason":"compile error",
      "script_stack":
        [
          "ctx._source.event.remove(original)",
          "                         ^---- HERE"
        ],
      "script":"ctx._source.event.remove(original)",
      "lang":"painless",
      "position":
        {
          "offset":25,
          "start":0,
          "end":34
        },
      "caused_by":
        {
          "type":"illegal_argument_exception",
          "reason":"cannot resolve symbol [original]"
        }
    },
  "status":400
}

Answer 1

EDIT : Your query is running NOT working.

You will always get the following result.

{
  "took": 0,
  "timed_out": false,
  "total": 0,     <---- "0" results.
  "updated": 0,
  "deleted": 0,
  "batches": 0,
  "version_conflicts": 0,
  "noops": 0,
  "retries": {
    "bulk": 0,
    "search": 0
  },
  "throttled_millis": 0,
  "requests_per_second": -1,
  "throttled_until_millis": 0,
  "failures": []
}

Try with ingest pipeline. Here is the documentation for foreach processor of ingest pipeline.

PUT test_nested
{
  "mappings": {
    "properties": {
      "event": {
        "type": "nested"
      }
    }
  }
}

---

POST test_nested/_doc/1
{
  "event": [
    {
      "original": "nested_value_1",
      "other_field": "first_value"
    },
    {
      "original": "nested_value_2",
      "other_field": "second_value"
    }
  ]
}

---

PUT _ingest/pipeline/remove_nested_field
{
  "processors": [
    {
      "foreach": {
        "field": "event",
        "processor": {
          "remove": {
            "field": "_ingest._value.original"
          }
        },
        "ignore_failure": true
      }
    }
  ]
}

---

POST test_nested/_update_by_query?pipeline=remove_nested_field

---

GET test_nested/_search

Reference: https://discuss.elastic.co/t/ingest-processor-failure-when-accessing-nested-field/144194/2

Answer 2

an ingest pipeline might have worked. I originally tried an ingest pipeline, but it failed due to the record size. Long story on how this happened, but it was not possible.

I did figure out how to access a nested field in painless using curl, however. it requires escaped double quotes (\") around the nested field name.

curl -XPOST "https://<server>:9200/<index>/_update_by_query?conflicts=proceed" -u <creds> --cacert <certpath> -H "Content-Type: application/json" -d '
{
    "script" : "ctx._source.event.remove(\"original\")",
    "query" : {
        "exists": { "field": "event.original" }
    }
}'