I am trying to accomplish a migration from MySQL to OpenSearch. Because, it is a kind of migration from relational data to NoSQL, I firstly created view to combine data to a json string (DMS is not supporting json migration).
After I run the migration from DMS, here is the result of a data from the index:
"hits": [
{
"_index": “orders”,
"_id": "096D70F8D93531CB03900EDED9FF0E32625C4E25A349E4BC3F8395C7B40008B2",
"_score": 1,
"_source": {
"orderId": “12345678”,
“json_str: """{"store": “IKEA”, "isTest": 0, "orderId": "12345678", "orderLines": [{"id": "d743142d-c106-11ee-afdf-06d5a72a9bcf", "ean": “1111111”, "vat": 0.25, "quantity": 1, "listPrice": 359.95, "lineNumber": 1, "productName": “Television”}], “orderDate": "2024-01-16 09:31:53.000000", "customerInformation": {"email": “aaa”@bbb.com, "customerId": “999999”, "isEmployee": 0}, "shippingInformation": {"parcelLocker": “Migros”, "shippingAddress": {"city": “ISTANBUL”, "state": null, "country": “TR”, "zipcode": “34000”, "houseNumber": "11"}}}"""
}
} ]
Now, I need to convert this json string(json_str) to json data as format of the OpenSearch like this:
"hits": [
{
"_index": "orders",
"_id": "12345678",
"_score": 1,
"_source": {
"orderId": "12345678",
“orderDate": "2024-01-16 09:31:53.000000",
"store": "IKEA",
"shippingInformation": {
"shippingAddress": {
"city": “ISTANBUL”,
"country": “TR”,
"houseNumber": “11”,
"phoneNumber": "12345678",
"zipcode": “34000”
},
"parcelLocker": "Migros"
},
"customerInformation": {
"email": "aaa”@bbb.com",
"customerId": "999999",
"isEmployee": false
},
"orderLines": [
{
"id": "d743142d-c106-11ee-afdf-06d5a72a9bcf",
"ean": "1111111",
"lineNumber": 1,
"productName": "Television",
"quantity": 1,
"listPrice": 359.95,
"vat": 0.25,
"brand": "JJXX"
}
]
}
}]
How can I create a OpenSearch pipeline to accomplish this? Is it possible to run it after migration (post processor)?
I tried "json" processor, but couldn't success.