Elasticsearch query syntax update 1.0.14 to 7.5.2

Question

I am attempting to update a rather sizable Elasticsearch query. I am very new to Elasticsearch and am having a hard time wrapping my head around everything that is happening here.

This is the original query:

elasticsearch_query search_models, {
  query: {
    filtered: {
      query: {
        function_score: {
          query: {
            bool: {
              must: [
                {
                  multi_match: {
                    operator: "and",
                    type: "cross_fields",
                    query: params[:term],
                    fuzziness: (params[:fuzzy] || 0),
                    fields: [
                      "name^2", "address", "email", "email2",
                      "primary_contact", "id", "lotname^7",
                      "lotname_keyword^9", "corp_name^4",
                      "vin^4", "serial_number^3", "dba_names",
                      "title_number", "title_tracking",
                      "full_name^3", "username", "reference", "user_name",
                      "phone", "text_number", "name_keyword^9"
                    ],
                  },
                }
              ],
              must_not: [
                { term: { access: { value: 2 } } },
                { term: { ledger: { value: "payroll" } } },
                { term: { ledger: { value: "credit card" } } }

              ],
              should: [
                { term: { active: { value: 1, boost: 100 } } },
                { term: { active: { value: 2, boost: 50 } } },
                { term: { active: { value: 3, boost: 0.05 } } },
                { term: { active: { value: 4, boost: 50 } } },
                { term: { active: { value: 5, boost: 50 } } },
                { term: { active: { value: 6, boost: 50 } } },
                { term: { branch_id: { value: current_branch.id, boost: 100 } } }
              ]
            }
          },
          functions: [
            {
              filter: { term: { auto_declined: 1 } },
              boost_factor: 0.3
            },
            {
              filter: { term: { auto_declined: 0 } },
              boost_factor: 0.0001
            },
            {
              filter: { term: { access: 1 } }, # current employee
              boost_factor: 10
            },
            {
              filter: { term: { access: 0 } },
              boost_factor: 0.2
            },
            {
              filter: { term: { unit_status: 1 } }, # current unit
              boost_factor: 2
            },
            {
              filter: { type: {value: 'txn'} },
              boost_factor: 0.4
            }
          ]
        }
      },
      filter: {
        "or" =>
        { filters: [
          { term: { "branch_id" => current_branch.id }},
          { type: { "value" => "auction" }},
          { type: { "value" => "fee_schedule"}},
          { type: { "value" => "unit"}},
          {
            "and" => [
              { type: { "value" => "user" }},
              { "or" => [
                { term: { "access" => 1 }}
              ]}
            ]
          }
        ]}
      }
    }
  }
}

This is where I have gotten to thus far:

elasticsearch_query search_models, {
  query: {
    bool: {
      must: {
        function_score: {
          query: {
            bool: [
              must: {
                multi_match: {
                  query: params[:term],
                  type: "cross_fields",
                  operator: "and",
                  fields: [
                    "name^2", "address", "email", "email2",
                    "primary_contact", "id", "lotname^7",
                    "lotname_keyword^9", "corp_name^4",
                    "vin^4", "serial_number^3", "dba_names",
                    "title_number", "title_tracking",
                    "full_name^3", "username", "reference", "user_name",
                    "phone", "text_number", "name_keyword^9"
                  ]
                }
              },
              must_not: [
                { term: { access: { value: 2 } } },
                { term: { ledger: { value: "payroll" } } },
                { term: { ledger: { value: "credit card" } } }
              ],
              should: [
                { term: { active: { value: 1, boost: 100 } } },
                { term: { active: { value: 2, boost: 50 } } },
                { term: { active: { value: 3, boost: 0.05 } } },
                { term: { active: { value: 4, boost: 50 } } },
                { term: { active: { value: 5, boost: 50 } } },
                { term: { active: { value: 6, boost: 50 } } },
                { term: { branch_id: { value: current_branch.id, boost: 100 } } }
              ]
            ]
          },
          functions: [
            {
              filter: { term: { auto_declined: 1 } },
              weight: 0.3
            },
            {
              filter: { term: { auto_declined: 0 } },
              weight: 0.0001
            },
            {
              filter: { term: { access: 1 } }, # current employee
              weight: 10
            },
            {
              filter: { term: { access: 0 } },
              weight: 0.2
            },
            {
              filter: { term: { unit_status: 1 } }, # current unit
              weight: 2
            },
            {
              filter: { type: {value: 'txn'} },
              weight: 0.4
            }
          ]
        }
      },
      filter: [
        { term: { branch_id: current_branch.id } },
        { type: { value: "auction" } },
        { type: { value: "fee_schedule"} },
        { type: { value: "unit"} },
        bool: {
          must: {
            bool: {
              should: [
                { type: { value: "user" } },
                { term: { access: 1 } }
              ]
            }
          }
        }
      ]
    }
  }
}

I have:

• replaced 'filtered' with 'bool' and 'must'
• replaced 'boost_factor' with 'weight'
• removed 'fuzziness' from the 'cross_fields' type 'multi_match'
• attempted to update the 'or' and 'and' logic with newer 'bool' syntax.

The first three actions seemed to have done the trick with their respective errors, but I am getting hung up on this filter with the 'or' and 'and' logic. I would greatly appreciate some guidance!

This is the error I am receiving:

[400] {"error":{"root_cause":[{"type":"parsing_exception","reason":"[bool] query malformed, no start_object after query name","line":1,"col":61}],"type":"parsing_exception","reason":"[bool] query malformed, no start_object after query name","line":1,"col":61},"status":400}

If any further information would be helpful, please let me know.

Answer 1

Your query is forming invalid Json, that why the exception, see below query with no Json error.

{
  "query": {
    "bool": {
      "must": {
        "function_score": {
          "query": {
            "bool": {
              "must": [{
                "multi_match": {
                  "query": "params[:term]",
                  "type": "cross_fields",
                  "operator": "and",
                  "fields": [
                    "name^2", "address", "email", "email2",
                    "primary_contact", "id", "lotname^7",
                    "lotname_keyword^9", "corp_name^4",
                    "vin^4", "serial_number^3", "dba_names",
                    "title_number", "title_tracking",
                    "full_name^3", "username", "reference", "user_name",
                    "phone", "text_number", "name_keyword^9"
                  ]
                }
              }],
              "must_not": [
                { "term": { "access": { "value": 2 } } },
                { "term": { "ledger": { "value": "payroll" } } },
                { "term": { "ledger": { "value": "credit card" } } }
              ],
              "should": [
                { "term": { "active": { "value": 1, "boost": 100 } } }
              ]
            }
          },
          "functions": [
            {
              "filter": { "term": { "auto_declined": 1 } },
              "weight": 0.3
            },
            {
              "filter": { "term": { "auto_declined": 0 } },
               "weight": 0.0001
            },
            {
              "filter": { "term": { "access": 1 } },
               "weight": 10
            },
            {
              "filter": { "term": { "access": 0 } },
               "weight": 0.2
            },
            {
              "filter": { "term": { "unit_status": 1 } },
               "weight": 2
            },
            {
              "filter": { "type": {"value": "txn"} },
               "weight": 0.4
            }
          ]
        }
      },
      "filter": [
        { "term": { "branch_id": "current_branch.id" } },
        { "type": { "value": "auction" } },
        { "type": { "value": "fee_schedule"} },
        { "type": { "value": "unit"} },
        {"bool": {
          "must": {
            "bool": {
              "should": [
                { "type": { "value": "user" } },
                { "term": { "access": 1 } }
              ]
            }
          }
        }
        }
      ]
    }
  }
}

You can compare and modify the curly braces and brackets.