Cannot open file for writing error while trying to edit a file in /tmp (even as root user)

To my understanding, the fs.protected_regular description matches the permission problems when trying to edit the file with vim. Besides this, the file in question is located inside /tmp, which is a world writeable sticky directoy.

Let's check the current Kernel settings using sysctl:

[root@rhel9 ~]# sysctl -a | grep fs.protected
fs.protected_fifos = 1
fs.protected_hardlinks = 1
fs.protected_regular = 1
fs.protected_symlinks = 1

They are all enabled.

Let's try a vim modification after disabling fs.protected_regular:

[root@rhel9 ~]# sysctl -w fs.protected_regular=0
fs.protected_regular = 0

[root@rhel9 ~]# vi /tmp/test1
-> Editing the file and adding [adding some additional text] at the end of the file
:wq

No error showed up this time and the added text was indeed saved in /tmp/test1:

[root@rhel9 ~]# tail -n 1 /tmp/test1
adding some additional text

After enabling fs.protected_regular again (set to 1), the vim edit permission error showed up again.

Where does this come from?

These additional file protections were added in the Linux Kernel 4.19 in commit namei: allow restricted O_CREAT of FIFOs and regular files. The goal is obviously to tackle a security issue, where an attacker could spoof on "shared" files in world writeable directories (e.g. PHP sys_temp_dir). That even root can't edit files anymore seems to be a side-effect.

This can also be reproduced in other Linux distributions. On an Ubuntu 22.04 and Debian 11 with Kernel 5.15 and on Debian 12 with Kernel 6.1.0, the fs.protected Kernel settings are by default set to these values:

claudio@ubuntu:~$ sudo sysctl -a | grep fs.protected
fs.protected_fifos = 1
fs.protected_hardlinks = 1
fs.protected_regular = 2
fs.protected_symlinks = 1

It turns out this is not a RHEL9 related "problem" but a newer Kernel security standard which I happened to never come across - until now. :-)

Note that these additional security settings are only applied on world writeable (1) or group writeable (2) directories with the sticky bit set (chmod 1777). On most Linux systems this is by default /tmp and /var/tmp:

root@debian ~ # stat /tmp/
  File: /tmp/
  Size: 4096          Blocks: 8          IO Block: 4096   directory
Device: fe00h/65024d    Inode: 2           Links: 9
Access: (1777/drwxrwxrwt)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2023-09-05 19:51:26.708920185 +0200
Modify: 2023-09-06 11:49:08.931388484 +0200
Change: 2023-09-06 11:49:08.931388484 +0200
 Birth: 2018-07-04 08:18:47.000000000 +0200

root@debian ~ # stat /var/tmp/
  File: /var/tmp/
  Size: 4096          Blocks: 8          IO Block: 4096   directory
Device: fe02h/65026d    Inode: 198         Links: 4
Access: (1777/drwxrwxrwt)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2023-09-06 02:15:10.973909196 +0200
Modify: 2023-09-06 00:00:02.193560882 +0200
Change: 2023-09-06 00:00:02.193560882 +0200
 Birth: 2018-07-04 08:18:59.675347253 +0200

Outside such directories, the normal permission handling applies and root can open, edit and save any regular file.