Ansible read files line by line and delete lines on condition

Question

Am trying to read a file line by line and delete the line if certain conditions are met

sample input file:
abcd Sat Oct 25 04:30:35 EDT 2036
defg Tue Dec 01 18:59:59 EST 2037
ghij Fri Jun 17 06:15:06 EDT 2022

Expected output file:
abcd Sat Oct 25 04:30:35 EDT 2036
defg Tue Dec 01 18:59:59 EST 2037

The logic needed is read the file line by line and read the year at the last field. If it's less than 2023 then delete that line in the file. if greater than 2023 keep the line. in this example, the last line should be deleted , since the year 2022 is lesser than 2023.

sample code:

- name: read file   
  set_fact:
    root_certificate_content: "{{ lookup('file', /home/user/cert_cleanup/{{ output_file }}) }}"
  
- name: display   debug:
  year: "{{ root_certificate_content.split('\n') [-1]}}" 

- name: remove line
  << delete that line >>
  when:
   - year.value < 2023

I tried multiple options, nothing see to work. Error in 1st step itself

Answer 1

Given the file

shell> cat /tmp/test.txt 
abcd Sat Oct 25 04:30:35 EDT 2036
defg Tue Dec 01 18:59:59 EST 2037
ghij Fri Jun 17 06:15:06 EDT 2022

Get the content of the file

  content: "{{ lookup('file', output_file) }}"

Split lines, create a list of the years, and create a dictionary from the years and lines

  lines: "{{ content.splitlines()|map('split') }}"
  years: "{{ lines|map('last')|map('int') }}"
  lines_year: "{{ dict(years|zip(lines)) }}"

gives

  lines_year:
    2022: [ghij, Fri, Jun, '17', '06:15:06', EDT, '2022']
    2036: [abcd, Sat, Oct, '25', '04:30:35', EDT, '2036']
    2037: [defg, Tue, Dec, '01', '18:59:59', EST, '2037']

Reject the lines older than 2023 and join the lines

  lines_2023: "{{ lines_year|dict2items|
                  rejectattr('key', 'lt', 2023)|
                  map(attribute='value')|map('join', ' ') }}"

gives

  lines_2023:
  - abcd Sat Oct 25 04:30:35 EDT 2036
  - defg Tue Dec 01 18:59:59 EST 2037

Write the file

     - copy:
         dest: "{{ output_file }}"
         content: |
           {% for line in lines_2023 %}
           {{ line }}
           {% endfor %}

---

Example of a complete playbook for testing

shell> cat pb.yml
- hosts: all

  vars:

    lines: "{{ content.splitlines()|map('split') }}"
    years: "{{ lines|map('last')|map('int') }}"
    lines_year: "{{ dict(years|zip(lines)) }}"
    lines_2023: "{{ lines_year|dict2items|
                    rejectattr('key', 'lt', 2023)|
                    map(attribute='value')|map('join', ' ') }}"

  tasks:

     - set_fact:
         content: "{{ lookup('file', output_file) }}"
       run_once: true

     - debug:
         var: lines_year|to_yaml
     - debug:
         var: lines_2023

     - copy:
         dest: "{{ output_file }}"
         content: |
           {% for line in lines_2023 %}
           {{ line }}
           {% endfor %}
       run_once: true
       delegate_to: localhost

gives running with options --check --diff

shell> ansible-playbook -e output_file=/tmp/test.txt pb.yml -CD

PLAY [all] ************************************************************************************

TASK [set_fact] *******************************************************************************
ok: [localhost]

TASK [debug] **********************************************************************************
ok: [localhost] => 
  lines_year|to_yaml: |-
    2022: [ghij, Fri, Jun, '17', '06:15:06', EDT, '2022']
    2036: [abcd, Sat, Oct, '25', '04:30:35', EDT, '2036']
    2037: [defg, Tue, Dec, '01', '18:59:59', EST, '2037']

TASK [debug] **********************************************************************************
ok: [localhost] => 
  lines_2023:
  - abcd Sat Oct 25 04:30:35 EDT 2036
  - defg Tue Dec 01 18:59:59 EST 2037

TASK [copy] ***********************************************************************************
--- before: /tmp/test.txt
+++ after: /home/admin/.ansible/tmp/ansible-local-16289938umqjl9j/tmpbav5odkq
@@ -1,3 +1,2 @@
 abcd Sat Oct 25 04:30:35 EDT 2036
 defg Tue Dec 01 18:59:59 EST 2037
-ghij Fri Jun 17 06:15:06 EDT 2022

changed: [localhost]

PLAY RECAP ************************************************************************************
localhost: ok=4    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

---

Notes:

• The lookup plugins always execute on the localhost
• Read and write the file once. Set run_once: true
• Write the file back to localhost. Set delegate_to: localhost