Monitor Filesystem Events with Pyinotify

File system monitoring through inotify can be interfaced through Python using pyinotify. This guide will demonstrate how to use a Python script to monitor a directory then explore practical uses by incorporating async modules or running additional threads.

1. Download and install Miniconda:

   curl -OL https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
   bash Miniconda3-latest-Linux-x86_64.sh
   

2. You will be prompted several times during the installation process. Review the terms and conditions and select “yes” for each prompt.

3. Restart your shell session for the changes to your PATH to take effect.

4. Check your Python version:

   python --version
   

Installing pyinotify within a virtual environment is highly recommended. This guide will use Miniconda, but virtualenv can also be used.

1. Create a virtual environment in Anaconda:

   conda create -n myenv python=3
   

2. Activate the new environment:

   source activate myenv
   

3. Install pyinotify within the virtual environment:

   pip install pyinotify
   

Similar to events in inotify, the Python implementation will be through an EventProcessor object with method names containing “process_” that is appended before the event name. For example, IN_CREATE in pyinotify though the EventProcessor will be process_IN_CREATE. The table below lists the inotify events used in this guide. In depth descriptions can be found in the man pages of inotify.

Below is the full script used in this guide. The EventProcessor class contain methods that print out the monitored file or directory will along with the corresponding inotify event. This guide will breakdown the code into smaller bits.

File: notify_me.py

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

import os import pyinotify class EventProcessor(pyinotify.ProcessEvent): _methods = ["IN_CREATE", "IN_OPEN", "IN_ACCESS", "IN_ATTRIB", "IN_CLOSE_NOWRITE", "IN_CLOSE_WRITE", "IN_DELETE", "IN_DELETE_SELF", "IN_IGNORED", "IN_MODIFY", "IN_MOVE_SELF", "IN_MOVED_FROM", "IN_MOVED_TO", "IN_Q_OVERFLOW", "IN_UNMOUNT", "default"] def process_generator(cls, method): def _method_name(self, event): print("Method name: process_{}()\n" "Path name: {}\n" "Event Name: {}\n".format(method, event.pathname, event.maskname)) _method_name.__name__ = "process_{}".format(method) setattr(cls, _method_name.__name__, _method_name) for method in EventProcessor._methods: process_generator(EventProcessor, method) watch_manager = pyinotify.WatchManager() event_notifier = pyinotify.Notifier(watch_manager, EventProcessor()) watch_this = os.path.abspath("notification_dir") watch_manager.add_watch(watch_this, pyinotify.ALL_EVENTS) event_notifier.loop()

Create notify_me.py in a text editor.

File: ~/notify_me.py

1 2 3

import pyinotify watch_manager = pyinotify.WatchManager()

Instantiate the Notifier class with an instance of WatchManager as the first argument and a ProcessEvent subclass instance as the second argument.

File: notify_me.py

1	event_notifier = pyinotify.Notifier(watch_manager, EventProcessor())

1. A watch is a file or directory to be monitored by pyinotify. Create a sample directory called notification_dir in your home directory:

   mkdir ~/notification_dir

2. Add this directory to our file system notification system. Call add_watch() on the watch manager instance watch_manager.

   File: notify_me.py

   1 2 3 4

   import os watch_this = os.path.abspath("notification_dir") watch_manager.add_watch(watch_this, pyinotify.ALL_EVENTS)

By looping the Notifier object, the directory can be monitored continuously. This loop method takes additional parameters, callback and daemonize, which calls a function after each iteration and daemonizes the thread respectively.

File: notify_me.py

1	event_notifier.loop()

Run the completed script and trigger the notifications.

1. Run the script:

   python notify_me.py
   

2. Open another terminal session and use ls to view the contents of the notification_dir folder:

   ls notification_dir
   

   This should trigger the pyinotify script in the original terminal session, and display the following output:

   Method name: process_IN_OPEN()
   Path name: /home/linode/linode_pyinotify/notification_dir
   Event Name: IN_OPEN|IN_ISDIR
   
   Method name: process_IN_ACCESS()
   Path name: /home/linode/linode_pyinotify/notification_dir
   Event Name: IN_ACCESS|IN_ISDIR
   
   Method name: process_IN_CLOSE_NOWRITE()
   Path name: /home/linode/linode_pyinotify/notification_dir
   Event Name: IN_CLOSE_NOWRITE|IN_ISDIR
   

   This output shows that the ls command involves three filesystem events. The notification_dir was opened, accessed, and then closed in non-writable mode.

   Note

   Observe that not only files are opened but also directories are opened too.

3. Change the current working directory to notification_dir with cd:

   cd notification_dir
   

4. Use different shell commands to manipulate files within the watched directory to fire other events:

   touch test_file.txt
   mv test_file.txt test_file2.txt
   rm test_file.txt
   

   Observe the output in the terminal as commands are executed in the monitored directory.

The call to loop() is blocking the current process in this example. Anything after the loop will not be executed because monitoring is expected to happen continuously. There are three options to workaround this problem:

• Notifier with a timeout

  When constructing the notifier, the timeout argument tells the Notifier to get notifications at certain intervals.

  event_notifier = pyinotify.Notifier(watch_manager, EventProcessor(), timeout=10)
  

  When using timeout, the application will not get file system change notification automatically. You need to explicitly call event_notifier.process_events() and event_notifier.read_events() at different times. Optionally call event_notifier.check_events() to check if there are any events waiting for processing.

• ThreadedNotifier

  We can deploy our file system notifier in a different thread. It is not necessary to create a new thread explicitly. Use the ThreadedNotifier class instead of Notifier and call event_notifier.start() to start event processing:

  File: notify_me.py

  1 2 3 4 5

  event_notifier = pyinotify.ThreadedNotifier(watch_manager, EventProcessor()) watch_this = os.path.abspath("notification_dir") watch_manager.add_watch(watch_this, pyinotify.ALL_EVENTS) event_notifier.start()

• AsyncNotifier

  If using Python’s asynchronous feature, AsyncNotifier can be used in place of Notifier.

  event_notifier = pyinotify.AsyncNotifier(watch_manager, EventProcessor())
  

  Then call the loop() function of the asyncore module.

  import asyncore
  asyncore.loop()
  

The limitations of inotify also apply when using pyinotify. For example, recursive directories are not monitored; another instance of inotify must be running to track subdirectories. Although Python provides a convenient interface to inotify, this also results in reduced performance compared to the C implementation of inotify.

More Information

You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

• Pyinotify on Github
• Pyinotify API documentation
• Inotify manpage