How to amend policy associated with Azure APIM APIs or operations with additional policy in automated way

Question

I am trying to have an automated way to amend or apply additional policy on APIs or operations scope of Azure APIM and in my script i am able to read a variable $inbound or $outbound or $backend or $onerror as per user inputs. Now i am trying to apply this newly given policy to the API or Operations, where some global policies/ existing policies already applied. How to add or amend these newly adding policies to the right session of the API or operations, without duplication them if the policy is already existing?

for f in $policy_ ; do
  if [[ $(eval echo \$${f}_name) == "ipfilter" ]]; then
    echo " given policy name is ipfilter "

    if [[ $(eval echo \$${f}_scope) == "api" ]]; then
      echo "decided the scope of Ipfilter policy as api "
    fi

    if [[ "$(eval echo \$${f}_apiname)" ]]; then
       echo "export the policy for the api $(eval echo \$${f}_apiname)"
       curl -H "Content-Type: application/json" -H "Authorization: Bearer $accessToken" "https://management.azure.com/subscriptions/xxxxx/resourceGroups/xx-rg/providers/Microsoft.ApiManagement/service/xxx-apim/apis/myapi/policies/policy?effective=true&format=xml&api-version=2022-08-01" > effectivepolicy.xml
    fi

    if [ -z "$(eval echo \$${f}_inboundsession)" ]; then
        echo 'the inbound session is not present'
    fi
    if [[ "$(eval echo \$${f}_inboundsession)" ]]; then
      echo 'the inbound session is present and append the policy settings to inbound'
      inbound=$(printf "$(eval echo \$${f}_inboundsession)")
       echo "$inbound"
       Add the $inbound to the inbound session of policy.xml if its not existing and apply back 
    fi

For example here $inbound variable value is

<ip-filter action="allow">
 <address-range from="xxxxx" to="yyy" />
</ip-filter> 

Below is the extracted effective policy sample from the api scope and need to amend and apply the same api with above newly added policy without duplication.

<policies>
    <inbound>
            <!--base: Begin Global scope-->
            <cors xxxxxxxxxx="true">
            ****************************
            ****************************
            ****************************
            </cors>
            <!--base: End Global scope-->
    </inbound>
    <backend>
            <!--base: Begin Global scope-->
             ****************************
             ****************************
             ****************************
            <!--base: End Global scope-->
    </backend>
    <outbound>
            <!--base: Begin Global scope-->
            ****************************
            ****************************
            ****************************
            ****************************
            <!--base: End Global scope-->
    </outbound>
    <on-error>
            <!--base: Begin Global scope-->
            ****************************
            ****************************
            ****************************
            <!--base: End Global scope-->
    </on-error>

Output of the custom policy file after trying the @khtesam Afrin Tried solution

parser script tried.

input of policy file in yaml

---

- name: ipfilter
  scope: api
  apiname: xxxxx
  inboundsession: |
                <ip-filter action="allow">
                    <address-range from="xxxxx" to="yyyy" />
                </ip-filter>
  outboundsession: |
                <ip-filter action="allow">
                    <address-range from="xxxxx" to="vvvv" />
                </ip-filter>

policy Script Tried

---

source parse_yaml.sh
eval $(parse_yaml input.yaml policy)
echo ".............Eval Result..............................."
for f in $policy_ ; do eval echo \$f \$${f}_ ; done
echo "............Eval Result................................"

for f in $policy_ ; do
  if [[ $(eval echo \$${f}_name) == "ipfilter" ]]; then
    echo " given policy name is ipfilter "

    if [[ $(eval echo \$${f}_scope) == "api" ]]; then
      echo "decided the scope of Ipfilter policy as api "
    fi

    if [[ "$(eval echo \$${f}_apiname)" ]]; then
       echo "export the policy for the api $(eval echo \$${f}_apiname)"
       curl -H "Content-Type: application/json" -H "Authorization: Bearer $accessToken" "https://management.azure.com/xxxx/providers/Microsoft.ApiManagement/service/ssssssss/apis/xxxxxxxxx/policies/policy?effective=true&format=xml&api-version=2022-08-01" > policy.xml
    fi

    if [ -z "$(eval echo \$${f}_inboundsession)" ]; then
        echo 'the inbound session is not present'
    fi
    if [[ "$(eval echo \$${f}_inboundsession)" ]]; then
      echo 'the inbound session is present and append the policy settings to inbound'
      inboundPolicy=$(printf "$(eval echo \$${f}_inboundsession)")
      echo "$inboundPolicy"
      if grep -qF "$inboundPolicy" policy.xml; then
        echo "Policy already exists in the inbound session."
      else
        # Insert the new policy into the existing policy XML
        echo "updating existing policy with new policy content"
        awk -v policy="$inboundPolicy" '/<\/inbound>/ && !p {print policy; p=1} 1' policy.xml > temp.xml
        mv temp.xml policy.xml
      fi
    fi

The parser script output for the variable inboundPolicy

---

<ip-filter action="allow">
 <address-range from="xxxxx" to="yyy" />
</ip-filter> 

OutPut Of the Script

---

.............Eval Result...............................
policy1 policy1_name policy1_scope policy1_apiname policy1_inboundsession policy1_outboundsession
............Eval Result................................
 given policy name is ipfilter
decided the scope of Ipfilter policy as api
export the policy for the api xxxxxxx
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
the inbound session is present and append the below policy settings to inbound
<ip-filter action="allow">
 <address-range from="xxxxx" to="yyyyyyy" />
 </ip-filter>
updating existing policy with new policy content
the outbound session is present
the backend session is not present

Output of the custom policy file

---

<policies>^M
        <inbound>^M
                <!--base: Begin Global scope-->^M
                <cors axxxxxxx="true">^M
                        <xxxxxxx>^M
                                <origin>aaaaaaaaaaa</origin>^M
                                <origin>bbbbbbbbbbb</origin>^M
                                <origin>cccccccccccc</origin>^M
                        </xxxxxxx>^M
                </cors>^M
                <!--base: End Global scope-->^M
<ip-filter action="allow">
 <address-range from="xxxx" to="yyyy" />
 </ip-filter>
        </inbound>^M
        <backend>^M
                <!--base: Begin Global scope-->^M
                <forward-request />^M
                <!--base: End Global scope-->^M

Answer 1

I am using the below script to check for duplication of the policy in effective policy before adding it.

#!/bin/bash

# Set variables
accessToken="eyJ0eXAi******qaoJinw"
subscriptionId="{subscriptionId}"
resourceGroupName="{resourceGroupName}"
apimServiceName="{apimServiceName}"
apiName="echo-api"
inboundPolicy='<set-header name="Test" exists-action="override">
            <value>Hi, Ikhtesam</value>
        </set-header>'

# Fetch existing effective policy XML
curl -H "Authorization: Bearer $accessToken" \
     "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.ApiManagement/service/$apimServiceName/apis/$apiName/policies/policy?effective=true&format=xml&api-version=2022-08-01" \
     > policy.xml

if grep -qF "$inboundPolicy" policy.xml; then
    echo "Policy already exists in the inbound session."
else
    # Insert the new policy into the existing policy XML
    awk -v policy="$inboundPolicy" '/<\/inbound>/ && !p {print policy; p=1} 1' policy.xml > temp.xml
    mv temp.xml policy.xml

    # Update the API's policy with the modified XML
    curl -X PUT \
  "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.ApiManagement/service/$apimServiceName/apis/$apiName/policies/policy?api-version=2022-08-01" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $accessToken" \
  -d '{
    "properties": {
      "format": "xml",
      "value": "'"$(sed -e 's/\\/\\\\/g' -e 's/"/\\"/g' policy.xml)"'"
    }
  }'
    echo "Policy added to the inbound session."
fi

By executing this script, I am able to add the set-header policy successfully.

When I ran my script for set-header policy again, I got below output.

Script for taking the inputs from yaml file.

#!/bin/bash

# Set variables
accessToken="eyJ0*****yXG6craOOQG3g"
subscriptionId="b823f"
resourceGroupName="RG_Name"
apimServiceName="Service_Name"
apiName="echo-api"

source parse_yaml.sh
eval $(parse_yaml input.yaml policy)
echo ".............Eval Result..............................."
for f in $policy_ ; do eval echo \$f \$${f}_ ; done
echo "............Eval Result................................"

for f in $policy_ ; do
  if [[ $(eval echo \$${f}_name) == "setheader" ]]; then
    echo " given policy name is setheader "

    if [[ $(eval echo \$${f}_scope) == "api" ]]; then
      echo "decided the scope of setheader policy as api "
    fi

    if [[ "$(eval echo \$${f}_apiname)" ]]; then
       echo "export the policy for the api $(eval echo \$${f}_apiname)"
       curl -H "Content-Type: application/json" -H "Authorization: Bearer $accessToken" "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.ApiManagement/service/$apimServiceName/apis/$apiName/policies/policy?effective=true&format=xml&api-version=2022-08-01" > policy.xml
    fi

    if [ -z "$(eval echo \$${f}_inboundsession)" ]; then
        echo 'the inbound session is not present'
    fi
    if [[ "$(eval echo \$${f}_inboundsession)" ]]; then
      echo 'the inbound session is present and append the policy settings to inbound'
      inboundPolicy=$(printf "$(eval echo \$${f}_inboundsession)")
      echo "$inboundPolicy"
      if grep -qF "$inboundPolicy" policy.xml; then
        echo "Policy already exists in the inbound session."
      else
        # Insert the new policy into the existing policy XML
        awk -v policy="$inboundPolicy" '/<\/inbound>/ && !p {print policy; p=1} 1' policy.xml > temp.xml
        mv temp.xml policy.xml

        # Update the API's policy with the modified XML
        curl -X PUT \
        "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.ApiManagement/service/$apimServiceName/apis/$apiName/policies/policy?api-version=2022-08-01" \
        -H "Content-Type: application/json" \
        -H "Authorization: Bearer $accessToken" \
        -d '{
            "properties": {
                "format": "xml",
                "value": "'"$(sed -e 's/\\/\\\\/g' -e 's/"/\\"/g' policy.xml)"'"
            }
        }'
      fi
    fi
  fi
done

Output-

In this way, you can add the policy without duplication.