mentat-enricher.py

This Mentat module is a real-time message processing daemon capable of enriching IDEA messages with additional data.

This daemon is implemented using the pyzenkit.zendaemon framework and so it provides all of its core features. See the documentation for in-depth details.

It is further based on mentat.daemon.piper module, which provides pipe-like message processing features. See the documentation for in-depth details.

Usage examples

# Display help message and exit.
mentat-enricher.py --help

# Run in debug mode and stay in foreground (enable output of debugging
# information to terminal and do not daemonize).
mentat-enricher.py --no-daemon --debug

# Run with increased logging level.
mentat-enricher.py --log-level debug

Configuration

The application supports multiple means for adjusting the internal configurations. When appropriate the default values for each configuration is hardcoded in module source code. However there are several options to change the value:

  • Override the internal default value when instantinating the application object by passing different value to object constructor.
  • Pass the different value by configuration file.
  • Pass the different value by command line option.

The configuration values are assigned from the sources mentioned above in that particular order, so the value given by command line option overwrites the value written in configuration file.

Command line options

Configuration can be passed down to application by command line options. These options have the highest priority and will overwrite any other configuration values. Depending on the base object of the application different set of options is available.

Common application options

Following configuration options are available for all applications based on pyzenkit.baseapp:

--help
Display help and usage description and exit (flag)
--name alternative-name

Alternative name for application instead of default $0, using which names for log, runlog, pid, status and other files will be generated.

Type: string, default: $0

--quiet

Run in quiet mode (flag).

Do not write anything to stdout or stderr.

Type: boolean, default: False

--verbose

Increase application output verbosity (flag, repeatable).

Type: boolean, default: False

--log-file file-name

Name of the log file.

Type: string, default: autodetected

--log-level level

Logging level [debug, info, warning, error, critical].

Type: string, default: info

--runlog-dir dir-name

Name of the runlog directory.

Type: string, default: autodetected

--runlog-dump

Dump runlog to stdout when done processing (flag).

Type: boolean, default: False

--runlog_log

Write runlog to logging service when done processing (flag)

Type: boolean, default: False

--pstate_file file-name

Name of the persistent state file.

Type: string, default: autodetected

--pstate_dump

Dump persistent state to stdout when done processing (flag).

Type: boolean, default: False

--pstate_log

Write persistent state to logging service when done processing (flag).

Type: boolean, default: False

--action action

Execute given quick action and exit. List of available actions can be displayed with --help option.

Type: string, default: None

--user name-or-id

Name/gid of the system user for process permissions.

Type: string, default: None

--group name-or-id

Name/gid of the system group for process permissions.

Type: string, default: None

Common daemon options

Following configuration options are available on top of common application options for all applications based on pyzenkit.zendaemon:

--no-daemon

Do not daemonize and stay in foreground (flag).

Type: boolean, default: False

--chroot-dir dir-name

Name of the chroot directory.

Type: string, default: None

--work_dir dir-name

Name of the process work directory.

Type: string, default: /

--pid_file file-name

Name of the pid file.

Type: string, default: autodetected

--state_file file-name

Name of the state file.

Type: string, default: autodetected

--umask mask

Default file umask.

Type: string, default: 0o002

--stats-interval interval

Processing statistics display interval in seconds.

Type: integer, default: 300

--runlog-interval interval

Application runlog dump interval in seconds.

Type: integer, default: 300

--paralel

Run in paralel mode (flag).

Type: boolean, default: False

Common piper daemon options

Following configuration options are available on top of common daemon options for all applications based on mentat.daemon.piper:

--queue-in-dir

Name of the input queue directory.

Type: string

--queue-out-dir

Name of the output queue directory.

Type: string, default: None

--queue-out-limit

Limit on the number of the files for the output queue directory.

Type: integer, default: 10000

--queue-out-wait

Waiting time when the output queue limit is reached in seconds.

Type: integer, default: 30

Configuration files and directories

Configuration can be passed down to application using a combination of configuration file or configuration directory. The configuration file

The available configuration keys are very similar to command line options and the names differ only in the use of _ character instead of -. However there is a certain set of configuration keys that is available only through command line options and not through configuration file and vice versa.