防篡改系统开发之一：用fsnotify监控文件变动

先来看下fsnotif的官方介绍：

fsnotify is a Go library to provide cross-platform filesystem notifications on Windows, Linux, macOS, BSD, and illumos.

fsnotify的github地址是：https://github.com/fsnotify/fsnotify

概括来说，fsnotify是一个Go语言的库，用于提供跨平台的文件系统通知。提供的文件系统变动事件包含：Rename, Create, Write, Remove, Chmod。

fsnotify开发要点

1. 当一个监控目录下的文件（或目录，下同）被重命名时，fsnotify会产生Rename + Create两个事件，在linux系统上通常先触发Rename事件然后再触发Create事件，所以用户执行Rename产生的事件须开发者自行合并；
2. fsnotify事件只包含当前发生事件的文件路径。因此在用户执行Rename时，须自行合并Rename和Create事件，以便获得Rename所需的src和dst文件路径；
3. 用户须自行处理Rename事件被移出监控目录的情况。fsnotify只提供监控目录下的文件变动事件，因此当Rename操作的dst（目标文件）不在监控目录时不会触发后续的Create事件，因此其路径无法获知；
4. 向fsnotify添加监控目录时，fsnotify并不会自动监控其子目录，须自行决定是否添加子目录的监控；
5. 在服务器上用各种编辑器编辑文件时，在编辑过程中会产生许多临时文件，这些临时文件建议做过滤处理；

用我们的监控系统模块gwatch举例一些处理要点

1、 合并Rename + Create事件，看下在服务器上执行rename mv a.txt c.txt命令产生的事件：

2024/02/03 18:35:49.090512 [D] gwatch - Get rename event, path=/tests/a.txt
2024/02/03 18:35:49.090600 [D] gwatch - rcChan receive data, type=rename, path=/tests/a.txt
2024/02/03 18:35:49.090650 [D] gwatch - Get create event, path=/tests/c.txt
2024/02/03 18:35:49.090664 [D] gwatch - rcChan receive data, type=create, path=/tests/c.txt
2024/02/03 18:35:49.090930 [D] gwatch - rename handle, fileType=file, src=/tests/a.txt, dst=/tests/c.txt
3 callback on rename /a.txt -> /c.txt file

由上面的log可以看出，当执行mv操作产生了rename+create事件，系统处理后最终执行的回调是3 callback on rename /a.txt -> /c.txt file

2、 编辑文件vi a.txt时，过滤不必要的临时文件产生的事件：

2024/02/03 18:44:07.867862 [D] gwatch - 文件在忽略规则中, file=/tests/.a.txt.swp
2024/02/03 18:44:07.867898 [D] gwatch - 文件在忽略规则中, file=/tests/.a.txt.swx
2024/02/03 18:44:07.867922 [D] gwatch - 文件在忽略规则中, file=/tests/.a.txt.swx
2024/02/03 18:44:07.867932 [D] gwatch - 文件在忽略规则中, file=/tests/.a.txt.swp
2024/02/03 18:44:07.868058 [D] gwatch - 文件在忽略规则中, file=/tests/.a.txt.swp
2024/02/03 18:44:07.868090 [D] gwatch - 文件在忽略规则中, file=/tests/.a.txt.swp
2024/02/03 18:44:09.025145 [D] gwatch - 文件在忽略规则中, file=/tests/.a.txt.swp
2024/02/03 18:44:11.033781 [D] gwatch - Get create event, path=/tests/a.txt
2024/02/03 18:44:11.033817 [D] gwatch - Get write event, path=/tests/a.txt
2024/02/03 18:44:11.033848 [D] gwatch - rcChan receive data, type=create, path=/tests/a.txt
2024/02/03 18:44:11.033874 [D] gwatch - Get write event, path=/tests/a.txt
2024/02/03 18:44:11.035107 [D] gwatch - create file handle, file=/tests/a.txt
6 callback on create file name: /a.txt hash: b426a48b8501c1d8
2024/02/03 18:44:11.067247 [D] gwatch - 文件在忽略规则中, file=/tests/.a.txt.swp
2024/02/03 18:44:11.068355 [D] gwatch - 文件在忽略规则中, file=/tests/.a.txt.swp

由上面的log看到用vi编辑文件最终保存时，gwatch只触发了6 callback on create file name: /a.txt hash: b426a48b8501c1d8 这么一次回调事件

3、 从监控目录外拷贝一个文件到监控目录cp ../../readme.md ./：

2024/02/03 18:49:46.104228 [D] gwatch - Get create event, path=/tests/readme.md
2024/02/03 18:49:46.104250 [D] gwatch - Get write event, path=/tests/readme.md
2024/02/03 18:49:46.104265 [D] gwatch - rcChan receive data, type=create, path=/tests/readme.md
2024/02/03 18:49:46.105479 [D] gwatch - create file handle, file=/tests/readme.md
9 callback on create file name: /readme.md hash: 8cfaaca233720be0

从上面的log可以看到，gwatch最终触发的回调是9 callback on create file name: /readme.md hash: 8cfaaca233720be0

4、 将文件移出监控目录mv a.txt ../：

2024/02/03 18:58:22.641719 [D] gwatch - Get rename event, path=/tests/a.txt
2024/02/03 18:58:22.641752 [D] gwatch - rcChan receive data, type=rename, path=/tests/a.txt
2024/02/03 18:58:22 callback on remove: id=10, name=/a.txt, type=file
2024/02/03 18:58:22.643029 [D] gwatch - remove handle, fileType=file, path=/tests/a.txt

文件移出监控区后，fsnotify并不会提供后续的Create事件，因此对于监控系统来说，实际就是Remove事件。所以gwatch最终执行的回调是OnRemove：callback on remove: id=10, name=/a.txt, type=file

5、 看下执行综合操作产生的记录：
操作命令如下：

mkdir -p a/b/c
vi a/b/c/d.txt
rm -rf a

操作产生的事件如下：

2024/02/03 19:05:40.349900 [D] gwatch - Get create event, path=/tests/a
2024/02/03 19:05:40.349929 [D] gwatch - rcChan receive data, type=create, path=/tests/a
2024/02/03 19:05:40.351141 [D] gwatch - create dir handle, dir=/tests/a
2024/02/03 19:05:40.351178 [I] gwatch - 开始添加监控目录, dir=/tests/a
2024/02/03 19:05:40.351591 [I] gwatch - 监控目录添加完成, dir=/tests/a, cost=406.804µs
19 callback on create dir /a
2024/02/03 19:05:48.743503 [D] gwatch - 文件在忽略规则中, file=/tests/a/b/c/.d.txt.swp
2024/02/03 19:05:48.743540 [D] gwatch - 文件在忽略规则中, file=/tests/a/b/c/.d.txt.swx
2024/02/03 19:05:48.743565 [D] gwatch - 文件在忽略规则中, file=/tests/a/b/c/.d.txt.swx
2024/02/03 19:05:48.743576 [D] gwatch - 文件在忽略规则中, file=/tests/a/b/c/.d.txt.swp
2024/02/03 19:05:48.743716 [D] gwatch - 文件在忽略规则中, file=/tests/a/b/c/.d.txt.swp
2024/02/03 19:05:48.743736 [D] gwatch - 文件在忽略规则中, file=/tests/a/b/c/.d.txt.swp
2024/02/03 19:05:49.511789 [D] gwatch - 文件在忽略规则中, file=/tests/a/b/c/.d.txt.swp
2024/02/03 19:05:51.767396 [D] gwatch - Get create event, path=/tests/a/b/c/d.txt
2024/02/03 19:05:51.767415 [D] gwatch - Get write event, path=/tests/a/b/c/d.txt
2024/02/03 19:05:51.767438 [D] gwatch - rcChan receive data, type=create, path=/tests/a/b/c/d.txt
2024/02/03 19:05:51.768643 [D] gwatch - create file handle, file=/tests/a/b/c/d.txt
20 callback on create file name: /a/b/c/d.txt hash: e151e288881a1e42
2024/02/03 19:05:51.794393 [D] gwatch - 文件在忽略规则中, file=/tests/a/b/c/.d.txt.swp
2024/02/03 19:05:51.795030 [D] gwatch - 文件在忽略规则中, file=/tests/a/b/c/.d.txt.swp
2024/02/03 19:05:55.384754 [D] gwatch - Get remove event, path=/tests/a/b/c/d.txt
2024/02/03 19:05:55.384781 [D] gwatch - Get remove event, path=/tests/a/b/c
2024/02/03 19:05:55.384802 [D] gwatch - Get remove event, path=/tests/a/b
2024/02/03 19:05:55.384814 [D] gwatch - Get remove event, path=/tests/a
2024/02/03 19:05:55.384872 [D] gwatch - remove handle, fileType=file, path=/tests/a/b/c/d.txt
2024/02/03 19:05:55.384880 [I] gwatch - 开始移除监控目录, dir=/tests/a/b/c
2024/02/03 19:05:55.384914 [I] gwatch - 移除监控目录操作完成, dir=/tests/a/b/c, cost=30.749µs
2024/02/03 19:05:55.384921 [D] gwatch - remove handle, fileType=directory, path=/tests/a/b/c
2024/02/03 19:05:55.384927 [I] gwatch - 开始移除监控目录, dir=/tests/a/b
2024/02/03 19:05:55.384948 [I] gwatch - 移除监控目录操作完成, dir=/tests/a/b, cost=20.161µs
2024/02/03 19:05:55.384954 [D] gwatch - remove handle, fileType=directory, path=/tests/a/b
2024/02/03 19:05:55.384960 [I] gwatch - 开始移除监控目录, dir=/tests/a
2024/02/03 19:05:55.385030 [I] gwatch - 移除监控目录操作完成, dir=/tests/a, cost=68.506µs
2024/02/03 19:05:55.385036 [D] gwatch - remove handle, fileType=directory, path=/tests/a
2024/02/03 19:05:55 callback on remove: id=21, name=/a/b/c/d.txt, type=file
2024/02/03 19:05:55 callback on remove: id=22, name=/a/b/c, type=directory
2024/02/03 19:05:55 callback on remove: id=23, name=/a/b, type=directory
2024/02/03 19:05:55 callback on remove: id=24, name=/a, type=directory

从上面的log记录可以看出，3条命令总共产生的回调事件如下：

19 callback on create dir /a
20 callback on create file name: /a/b/c/d.txt hash: e151e288881a1e42
callback on remove: id=21, name=/a/b/c/d.txt, type=file
callback on remove: id=22, name=/a/b/c, type=directory
callback on remove: id=23, name=/a/b, type=directory
callback on remove: id=24, name=/a, type=directory