Finding Files in Found Directories with Ansible

If you spend some time around shell scripts, you've likely come across one that looks something like this:

#!/bin/bash
LOG_DIR=/my/log/dir

cd ${LOG_DIR}

for h in `ls -d web*`
do
  cd ${LOG_DIR}/${h};
  for f in `ls access.log-20??????.gz`
  do
    # move these to cold storage
    ...
  done
done

For those whose BASH is a little rusty, this script goes into a log directory, finds all of the subdirectories that begin with 'web', then finds files matching a glob pattern inside of those and does something with them.

Those final two parts (finding files matching the glob pattern and doing something with them) are trivial in Ansible. However, when it comes to finding files in directories you don't know the name of yet, things get a bit trickier.

Ansible's find module does a great job determining the list of directories we want (such as all directories that begin with web).

- name: Get web log directories
  find:
    paths: /my/log/dir
    file_type: directory
    patterns: 'web*'
  register: dir_paths

The problem is that dir_paths.files isn't an acceptable input for find's paths parameter, as it is really a list of dictionaries containing lots of information about those directories and not just their paths. So, we need to extract the paths from those dictionaries and we're going to use the json_query filter to do it.

The json_query filter will allow us to iterate through those dictionaries and build a new list from a value extracted from each, in this case path.

The syntax for many json_query uses can get tricky, but ours will be straight-forward:

'files[*].path'

This goes through every member of the list files and extracts the value of that member's path item.

So, we can now chain these together, to look like this:

- name: Get web log directories
  find:
    paths: /my/log/dir
    file_type: directory
    patterns: 'web*'
  register: dir_paths

- name: Get log files
  find:
    paths: "{{ dir_paths | json_query('files[*].path') }}"
    file_type: file
    patterns: 'access.log-20??????.gz'
  register: log_files

From here, you can do whatever you need to with the files, iterating over log_files.files via with_items.

Hopefully, this helped you as you move from those old shell scripts to Ansible.

Comments

Tags