How to use Ansible python API to parse remote file and export json format

Question

I would like to integrate Ansbile 2.9.9 python API with python 3.6.8 to sequentially parse each file (pt-mysql-summary.txt) on each host and export json format.

Here is my ansible and python code.

ansible-playbook -i hosts sum.yml

sum.yml : generated summary file for each host

- hosts: staging
  tasks:
    - name: pt_mysql_sum
      shell: PTDEST=/tmp/collected;mkdir -p $PTDEST;cd /tmp;wget percona.com/get/pt-mysql-summary;chmod +x pt*;./pt-mysql-summary -- --user=adm --password=***** > $PTDEST/pt-mysql-summary.txt;
      register: result
    - name: ansible_result
      debug: var=result.stderr_lines
    - name: fetch_log
      fetch:
        src: /tmp/collected/pt-mysql-summary.txt
        dest: /tmp/collected/pt-mysql-summary-{{ inventory_hostname }}.txt
        flat: yes

hosts file

[staging]
vm1-kanpai ansible_ssh_host=10.41.219.11 ansible_ssh_user=testuser ansible_ssh_pass=*****

Here is pt-mysql-summary.txt

# Summary Report #######################

System time | 2020-05-27 16:35:00 UTC (local TZ: UTC +0000)
# Instances ##################################################
  Port  Data Directory             Nice OOM Socket
  ===== ========================== ==== === ======
                                   0    0
# Configuration File #########################################
              Config File | /etc/my.cnf
[mysqld]
server_id            = 1
port                                = 3307
tmpdir                              = /tmp
performance_schema_instrument       = '%=on'
innodb_monitor_enable               = 'module_adaptive_hash'
innodb_monitor_enable               = 'module_buffer'

[client]
port                                = 3307

# management library ##################################
jemalloc is not enabled in mysql config for process with id 2425
# The End ####################################################

[conf2json.py]: parse file (pt-mysql-summary.txt) and export json format

import json
import re
import collections
from datetime import datetime
import sys
# reads all the lines from the text file

conf_file = 'pt-mysql-summary.txt'
all_lines = open(conf_file, 'r').readlines()

# skip lines, look for patterns here []
final_dict = {}
#final_dict = collections.defaultdict(list)
regex = r"^([a-zA-Z]+)(.)+="

config = 0 # not yet found config
for line in all_lines:
    if '[mysqld]' in line:
        final_dict['mysqld'] = {}
        config = 1
        continue
    if '[client]' in line:
        final_dict['client'] = {}
        config = 2
        continue
    if config == 1 and re.search(regex, line):
        try:
            clean_line = line.strip() # get rid of empty space
            k = clean_line.split('=')[0].rstrip() # get the key
            v = clean_line.split('=')[1].lstrip()
            # when value include '='
            if len(clean_line.split('=')) > 2 :
                count = 1
                while count < len(clean_line.split('='))-1:
                    count = count + 1
                    v = v + "=" + clean_line.split('=')[count].lstrip()

            # put multiple values in the same variable
            if k in final_dict['mysqld']:
                v = final_dict['mysqld'][k]+", "+v

            final_dict['mysqld'][k] = v
        except Exception as e:
            print(clean_line, e)
    if config == 2 and re.search(regex, line):
        try:
            clean_line = line.strip() # get rid of empty space
            k = clean_line.split('=')[0].rstrip() # get the key
            v = clean_line.split('=')[1].lstrip()
            final_dict['client'][k] = v
        except Exception as e:
            print(clean_line, e)

print(json.dumps(final_dict, sort_keys=True))

with open('my.json', 'w') as f:
    json.dump(final_dict, f, sort_keys=True, indent=4)

However, I hope to only deploy my python code on one client, not all hosts. How to integrate my python code with Ansbile python API to make it?

Answer 1

from what you showd this should work. It's not tested but it should show the process...get the file name as parameter and use the script add in. You could also use the shell add in.

- hosts: staging
  tasks:
  - name: Ansible find files 
    find:
      paths: "/tmp/collected/"
      patterns: "pt-mysql-summary.*"
      recurse: yes
    register: found_files

  - name: show files
    script: conf2json.py
    with_items: "{{ found_files.files }}"