Ansible blockinfile TypeError: startswith first arg must be bytes or a tuple of bytes, not str

Written by Claudio Kuenzler - 0 comments

Published on November 24th 2021 - Listed in Ansible

---

An interesting error was seen today in our Ansible configuration management. A playbook, which uses the blockinfile module to write a "block" of content into a file, and was in use for many years, suddenly stopped working.

The playbook

To reproduce the error, a very simple playbook, based on the original playbook was created:

ck@ansible:~$ cat /tmp/blockinfile.yaml
---
- name: ANSIBLE - Blockinfile test - Infiniroot LLC  
  hosts: '{{ target }}'
  roles:
    - yaegashi.blockinfile
  tasks:

  ##########################
  # Set facts
  ##########################

  - name: Write a simple block into testfile
    blockinfile:
      dest: /tmp/blockinfile.test
      create: yes
      insertafter: "EOF"
      marker: "# {mark} -- configured by Ansible"
      block: |
        Write some text
        Write some more text
        Write even more text
        And again write some text

Running the playbook -> error

When the plugin was run against an Ubuntu 20.04 machine with Python 3.8, the blockinfile task failed with the following error:

ck@ansible:~$ ansible-playbook /tmp/blockinfile.yaml --extra-vars "target=target.example.com"

PLAY [ANSIBLE - Blockinfile test - Infiniroot LLC] **************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************
ok: [target.example.com]

TASK [Write a simple block into testfile] **************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: startswith first arg must be bytes or a tuple of bytes, not str
fatal: [target.example.com]: FAILED! => {"changed": false, "module_stderr": "Shared connection to 10.10.2.54 closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1637765329.6180422-5446-104584190794430/AnsiballZ_blockinfile.py\", line 100, in <module>\r\n    _ansiballz_main()\r\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1637765329.6180422-5446-104584190794430/AnsiballZ_blockinfile.py\", line 92, in _ansiballz_main\r\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1637765329.6180422-5446-104584190794430/AnsiballZ_blockinfile.py\", line 40, in invoke_module\r\n    runpy.run_module(mod_name='ansible.modules.blockinfile', init_globals=dict(_module_fqn='ansible.modules.blockinfile', _modlib_path=modlib_path),\r\n  File \"/usr/lib/python3.8/runpy.py\", line 207, in run_module\r\n    return _run_module_code(code, init_globals, run_name, mod_spec)\r\n  File \"/usr/lib/python3.8/runpy.py\", line 97, in _run_module_code\r\n    _run_code(code, mod_globals, init_globals,\r\n  File \"/usr/lib/python3.8/runpy.py\", line 87, in _run_code\r\n    exec(code, run_globals)\r\n  File \"/tmp/ansible_blockinfile_payload_390rv7rl/ansible_blockinfile_payload.zip/ansible/modules/blockinfile.py\", line 268, in <module>\r\n  File \"/tmp/ansible_blockinfile_payload_390rv7rl/ansible_blockinfile_payload.zip/ansible/modules/blockinfile.py\", line 215, in main\r\nTypeError: startswith first arg must be bytes or a tuple of bytes, not str\r\n", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

PLAY RECAP **************************************************************************************************
target.example.com : ok=1    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0 

However when the target's Python version was changed to use Python 2.7 instead of 3.8 (by setting the ansible_python_interpreter in the Ansible inventory), the task was successfully executed:

ck@ansible:~$ grep target inventory/*
inventory/hosts:target.example.com ansible_ssh_host=10.10.2.54 ansible_python_interpreter=/usr/bin/python2.7

ck@ansible:~$ ansible-playbook /tmp/blockinfile.yaml --extra-vars "target=target.example.com"

PLAY [ANSIBLE - Blockinfile test - Infiniroot LLC] **************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************
ok: [target.example.com]

TASK [Write a simple block into testfile] **************************************************************************************************
changed: [target.example.com]

PLAY RECAP **************************************************************************************************
target.example.com : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Is it a Python bug?

The blockinfile task worked fine under Python 2.7, but not under the newer Python 3.8 on the target Ubuntu 20.04 machine. So it must be a Python bug, right?

That was our first thought but no! Although we created a new bug report #76358 in the Ansible repository, we continued the analysis. Another playbook was created in another customer environment with Debian 11 target systems (which have Python 3.9) and on these targets the playbook ran successfully!

ck@ansible2:~$ cat /tmp/simpleblockinfile.yaml
---
- name: ANSIBLE - minimal blockinfile test - Infiniroot LLC  
  hosts: '{{ target }}'
  tasks:

  # Write a block into testfile
  - name: Write a block into testfile
    blockinfile:
      dest: /tmp/blockinfile.test
      insertafter: "EOF"
      marker: "# {mark} -- configured by Ansible"
      block: |
        Write some text
        Write some more text
        Write even more text
        And again write some text
    when: testfile.stat.exists == true

Maybe the problem only hits under a specific Python 3 version (before 3.9)? But the simplified playbook also ran on Ubuntu 18.04 targets with Python 3.6.

The simplified playbook was then copied to the original customer environment where the blockinfile problems occurred - and it worked!

Legacy yaegashi.blockinfile

Finally we realized what was causing the errors on Python 3: The original playbook was still using the yaegashi.blockinfile role. This was an old method to implement the blockinfile module before it became part of the "builtin" modules. That happened a while ago, but the playbook still contained the role include.

As long as the target hosts still linked to Python 2.7 under /usr/bin/python, the original blockinfile module continued to work. But recent distribution releases have now ditched Python 2.7 and /usr/bin/python points to a Python 3.x version. And bam! The playbooks stopped working on these targets.

By simply removing the yaegashi.blockinfile role in the playbook "headers", the blockinfile module is used from the Ansible builtin modules - which works correctly under both Python 2.7 and Python 3.x.

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.