AzureDevops task to create document in azure wiki

Question

In azuredevops server pipeline task, i have a task to create .csv file from the kubernetes get command output and which working as expected. Looking for a way to publish this .csv table format to Azure- wiki and this pipeline need to scheduled every 5 minutes so that the wiki will be updated dynamically with the current details.

- bash: |
     kubectl get vpa -A -o json | jq -r '["Namespace", "AppName", "ContainerName", "LowerBoundCPU","LowerBoundMemmory", "upperBoundCPU", "upperBoundmemmory", "TargetCPU", "TargetMem"], (.items[] | [.metadata.namespace, .metadata.name] +(.status.recommendation.containerRecommendations[] | [.containerName, .lowerBound.cpu, .lowerBound.memory, .upperBound.cpu, .upperBound.memory, .target.cpu, .target.memory])) | @csv' >output.csv
  displayName: 'VPA Report Creation'

Tried the Provided solution as below, but getting error.

My Subpage path copied from wiki and the Wiki name of my project is: - MyProject.wiki

pagepath :- VPA-Status

- bash: |
    # Convert CSV to Markdown Table
    awk -F, 'BEGIN { OFS = "|" } { print "| " $0 " |" }' output.csv > temp.md
    awk 'NR==1 {print; print "|---|---|---|---|---|---|---|---|---|"} NR>1 {print}' temp.md > output.md
    rm temp.md
  displayName: 'Convert CSV to Markdown'

- bash: |
    PAT=$(System.AccessToken)
    URI="https://<myadoserver url>/devops/<>Collectionname>/My-Project/_apis/wiki/wikis/MyProject.wiki/pages?path=/My-Project/Operations/Infra-Optimisation/VPA%2DStatus&api-version=6.0"
    CONTENT=$(<output.md)
    curl -u ":$PAT" -X PUT "$URI" -H "Content-Type: application/json" --data "{'content': '$CONTENT'}"
  displayName: 'Update Azure Wiki'

Error as below

  {"$id":"1","innerException":null,"message":"One or more ancestor pages of the page '/My-Project/Operations/Infra-Optimisation/VPA-Status' does not exist.","typeName":"Microsoft.TeamFoundation.Wiki.Server.WikiAncestorPageNotFoundException, Microsoft.TeamFoundation.Wiki.Server","typeKey":"WikiAncestorPageNotFoundException","errorCode":0,"eventId":3000}

Answer 1

Looking for a way to publish this .csv table format to Azure-wiki

That should be possible with the REST API endpoint Pages - Update

After your VPA Report Creation task, add a script task to convert your output.csv into a Markdown table format that you can push to Azure Wiki.
Then use curl or another HTTP client to call the Azure DevOps REST API to update the Wiki page.

schedules:
- cron: "*/5 * * * *"
  displayName: Every 5 minutes
  branches:
    include:
    - main  # Replace with your target branch
  always: true

jobs:
- job: UpdateWiki
  steps:
  - bash: |
      kubectl get vpa -A -o json | jq -r '["Namespace", "AppName", "ContainerName", "LowerBoundCPU","LowerBoundMemmory", "upperBoundCPU", "upperBoundmemmory", "TargetCPU", "TargetMem"], (.items[] | [.metadata.namespace, .metadata.name] +(.status.recommendation.containerRecommendations[] | [.containerName, .lowerBound.cpu, .lowerBound.memory, .upperBound.cpu, .upperBound.memory, .target.cpu, .target.memory])) | @csv' >output.csv
    displayName: 'VPA Report Creation'

  - bash: |
      # Your script here to convert output.csv into a Markdown table and save it as output.md
    displayName: 'Convert CSV to Markdown'

  - bash: |
      PAT=$(System.AccessToken) 
      URI="https://dev.azure.com/{organization}/{project}/_apis/wiki/wikis/{wikiIdentifier}/pages?path={path}&api-version=6.0"
      CONTENT=$(<output.md)

      curl -u ":$PAT" -X PUT "$URI" -H "Content-Type: application/json" --data "{'content': '$CONTENT'}"
    displayName: 'Update Azure Wiki'

Do replace {organization}, {project}, and {wikiIdentifier} with the appropriate values for your Azure DevOps setup.
The variable System.AccessToken contains the Azure DevOps PAT (Personal Access Token) that is automatically injected into the pipeline run. Ensure that this token has permissions to update the Wiki.

---

But the challenge i am facing to convert the .csv file to markdown

Converting a .csv file to Markdown table format can be achieved using a variety of scripting languages. Since you are already using a bash script in your pipeline, we can keep bash for consistency.

For instance, here is a script snippet that reads a .csv file and converts it to a Markdown table. You can add this script as a step in your pipeline after generating the output.csv file.

# Convert CSV to Markdown Table
awk -F, 'BEGIN { OFS = "|" } { print "| " $0 " |" }' output.csv > temp.md

# Add header separator
awk 'NR==1 {print; print "|---|---|---|---|---|---|---|---|---|"} NR>1 {print}' temp.md > output.md

# Remove temporary file
rm temp.md

To integrate this into your existing pipeline, you can simply add it as a separate bash step.

schedules:
- cron: "*/5 * * * *"
  displayName: Every 5 minutes
  branches:
    include:
    - main  # Replace with your target branch
  always: true

jobs:
- job: UpdateWiki
  steps:
  - bash: |
      kubectl get vpa -A -o json | jq -r '["Namespace", "AppName", "ContainerName", "LowerBoundCPU","LowerBoundMemmory", "upperBoundCPU", "upperBoundmemmory", "TargetCPU", "TargetMem"], (.items[] | [.metadata.namespace, .metadata.name] +(.status.recommendation.containerRecommendations[] | [.containerName, .lowerBound.cpu, .lowerBound.memory, .upperBound.cpu, .upperBound.memory, .target.cpu, .target.memory])) | @csv' >output.csv
    displayName: 'VPA Report Creation'

  - bash: |
      # Convert CSV to Markdown Table
      awk -F, 'BEGIN { OFS = "|" } { print "| " $0 " |" }' output.csv > temp.md
      awk 'NR==1 {print; print "|---|---|---|---|---|---|---|---|---|"} NR>1 {print}' temp.md > output.md
      rm temp.md
    displayName: 'Convert CSV to Markdown'

  - bash: |
      PAT=$(System.AccessToken)
      URI="https://dev.azure.com/{organization}/{project}/_apis/wiki/wikis/{wikiIdentifier}/pages?path={path}&api-version=6.0"
      CONTENT=$(<output.md)
      
      curl -u ":$PAT" -X PUT "$URI" -H "Content-Type: application/json" --data "{'content': '$CONTENT'}"
    displayName: 'Update Azure Wiki'

That modification should handle the conversion of your .csv file to a Markdown table, which is then updated to the Azure Wiki every 5 minutes.

Tried solution and unexpected behavior

The given solution didnt work for subpages, and the published page is not in the table format.

"Namespace","AppName","ContainerName","LowerBoundCPU","LowerBoundMemmory","upperBoundCPU","upperBoundmemmory","TargetCPU","TargetMem"
"xxxxx","yyyyyy","tttttt","10m","109809547","11m","112416782","11m","109814751"
"xxxxxxx","xxxxxxx-dddddd","xxxxxxx-ffffff","10m","628665161","11m","643591466","11m","628694953"