mentat.system module¶

Mentat system management and inspection library.

class mentat.system.MentatCronjob(params)[source]¶

Bases: object

Class representing Mentat cronjob module configuration for control utility.

class mentat.system.MentatModule(params)[source]¶

Bases: object

Class representing Mentat real-time module configuration for control utility.

mentat.system.REGEXP_MENTAT_CACHEF = re.compile('(.+)\\.json$')¶: Regular expression for selecting Mentat related cache files.

mentat.system.REGEXP_MENTAT_CRONF = re.compile('(.+)\\.cron$')¶: Regular expression for selecting Mentat related log files.

mentat.system.REGEXP_MENTAT_LOGF = re.compile('(.+)\\.log$')¶: Regular expression for selecting Mentat related log files.

mentat.system.REGEXP_MENTAT_PIDF = re.compile('(.+?)(?:\\.([0-9a-fA-F]+))?\\.pid$')¶: Regular expression for selecting Mentat related PID files.

mentat.system.REGEXP_MENTAT_PS = re.compile('\\s*(\\d+)\\s+([^\\s]+)\\s+([^\\s]*(?:python(?:\\d+(?:\\.\\d+)?)?))\\s+[^\\s]+(mentat-[^\\s]+)(?:\\s+(.*))?')¶: Regular expression for selecting Mentat related processes.

mentat.system.REGEXP_MENTAT_RLOGF = re.compile('(.+)(?:\\.([0-9a-fA-F]+))?\\.runlog$')¶: Regular expression for selecting Mentat related runlog files.

mentat.system.analyze_cache_file(cache_file, cache_file_path)[source]¶

Analyze given cache file.

Example of returned structure:

{
    'atime': datetime.datetime(2018, 1, 22, 15, 11, 15, 562358),
    'file': 'itemset-stat-detectors.json',
    'mtime': datetime.datetime(2018, 1, 23, 9, 8, 43, 403734),
    'name': 'itemset-stat-detectors',
    'path': '/var/mentat/cache/itemset-stat-detectors.json',
    'size': 5831678
}

Parameters

cache_file (str) – Basename of the cache file.
cache_file_path (str) – Full path to the cache file.

Returns

Structure containing parsed cache file information.

Return type

dict

mentat.system.analyze_cache_files(cache_dir_path)[source]¶

Analyze all cache files in cache directory.

Example of returned structure:

{
    'itemset-stat-detectors': {'atime': datetime.datetime(2018, 1, 9, 12, 25, 26, 338720),
                               'file': 'itemset-stat-detectors.json',
                               'mtime': datetime.datetime(2018, 1, 23, 9, 8, 38, 163710),
                               'name': 'mentat-enricher.py',
                               'path': '/var/mentat/cache/itemset-stat-detectors.json',
                               'size': 6057586,
                               'data': {...}}
}

Parameters: cache_dir_path (str) – Path to directory containing cache files.
Returns: Structure containing parsed cache file information.
Return type: dict

mentat.system.analyze_cron_file(cron_file, cron_file_path, cron_links)[source]¶

Analyze given cron file.

Example of returned structure:

{
    'atime': datetime.datetime(2018, 1, 21, 9, 13, 48, 34648),
    'file': 'mentat-statistician-py.cron',
    'link': None,
    'mtime': datetime.datetime(2017, 7, 19, 10, 25, 30),
    'name': 'mentat-statistician-py',
    'path': '/etc/mentat/cron/mentat-statistician-py.cron',
    'size': 429
}

Parameters

cron_file (str) – Basename of the cron file.
cron_file_path (str) – Full path to the cron file.
cron_links (dict) – Dictionary containing existing links in cron directory.

Returns

Structure containing parsed cron file information.

Return type

dict

mentat.system.analyze_cron_files(cfg_dir_path, cron_dir_path)[source]¶

Analyze all cron files in config and cron directory.

Example of returned structure:

{
    'mentat-precache-py': {'atime': datetime.datetime(2018, 1, 21, 9, 13, 45),
                           'file': 'mentat-precache-py.cron',
                           'link': None,
                           'mtime': datetime.datetime(2017, 9, 1, 11, 10, 17),
                           'name': 'mentat-precache-py',
                           'path': '/etc/mentat/cron/mentat-precache-py.cron',
                           'size': 417},
    'mentat-statistician-py': {'atime': datetime.datetime(2018, 1, 21, 9, 13, 48, 34648),
                               'file': 'mentat-statistician-py.cron',
                               'link': None,
                               'mtime': datetime.datetime(2017, 7, 19, 10, 25, 30),
                               'name': 'mentat-statistician-py.cron',
                               'path': '/etc/mentat/cron/mentat-statistician-py.cron',
                               'size': 429}
}

Parameters

cfg_dir_path (str) – Path to configuration directory containing cron scripts.
cron_dir_path (str) – Path to system cron directory.

Returns

Structure containing parsed cron file information.

Return type

dict

mentat.system.analyze_log_file(log_file, log_file_path)[source]¶

Analyze given log file.

Example of returned structure:

{
    'atime': datetime.datetime(2018, 1, 22, 15, 11, 15, 562358),
    'file': 'mentat-storage.py.log',
    'mtime': datetime.datetime(2018, 1, 23, 9, 8, 43, 403734),
    'name': 'mentat-storage.py',
    'path': '/var/mentat/log/mentat-storage.py.log',
    'size': 5831678
}

Parameters

log_file (str) – Basename of the log file.
log_file_path (str) – Full path to the log file.

Returns

Structure containing parsed log file information.

Return type

dict

mentat.system.analyze_log_files(log_dir_path)[source]¶

Analyze all PID files in run directory.

Example of returned structure:

{
    'mentat-enricher.py': {'atime': datetime.datetime(2018, 1, 9, 12, 25, 26, 338720),
                           'file': 'mentat-enricher.py.log',
                           'mtime': datetime.datetime(2018, 1, 23, 9, 8, 38, 163710),
                           'name': 'mentat-enricher.py',
                           'path': '/var/mentat/log/mentat-enricher.py.log',
                           'size': 6057586},
    'mentat-inspector.py': {'atime': datetime.datetime(2018, 1, 18, 17, 5, 22, 734164),
                            'file': 'mentat-inspector.py.log',
                            'mtime': datetime.datetime(2018, 1, 23, 9, 8, 35, 291697),
                            'name': 'mentat-inspector.py',
                            'path': '/var/mentat/log/mentat-inspector.py.log',
                            'size': 8156449},
    'mentat-storage.py': {'atime': datetime.datetime(2018, 1, 22, 15, 11, 15, 562358),
                          'file': 'mentat-storage.py.log',
                          'mtime': datetime.datetime(2018, 1, 23, 9, 8, 43, 403734),
                          'name': 'mentat-storage.py',
                          'path': '/var/mentat/log/mentat-storage.py.log',
                          'size': 5831678}
}

Parameters: log_dir_path (str) – Path to directory containing log files.
Returns: Structure containing parsed log file information.
Return type: dict

mentat.system.analyze_pid_file(pid_file, pid_file_path)[source]¶

Analyze given PID file.

Example of returned structure:

{
    'atime': datetime.datetime(2018, 1, 23, 8, 25, 16, 439742),
    'file': 'mentat-storage.py.pid',
    'mtime': datetime.datetime(2018, 1, 23, 8, 25, 12, 943727),
    'name': 'mentat-storage.py',
    'paralel': False,
    'path': '/var/mentat/run/mentat-storage.py.pid',
    'pid': 9349,
    'size': 5
}

Parameters

pid_file (str) – Basename of the PID file.
pid_file_path (str) – Full path to the PID file.

Returns

Structure containing parsed PID file information.

Return type

dict

mentat.system.analyze_pid_files(pid_dir_path)[source]¶

Analyze all PID files in given run directory.

Example of returned structure:

{
    'mentat-enricher.py': {9356: {'atime': datetime.datetime(2018, 1, 23, 8, 25, 16, 439742),
                                  'file': 'mentat-enricher.py.pid',
                                  'mtime': datetime.datetime(2018, 1, 23, 8, 25, 14, 319733),
                                  'name': 'mentat-enricher.py',
                                  'paralel': False,
                                  'path': '/var/mentat/run/mentat-enricher.py.pid',
                                  'pid': 9356,
                                  'size': 5}},
    'mentat-inspector.py': {9362: {'atime': datetime.datetime(2018, 1, 23, 8, 25, 16, 439742),
                                   'file': 'mentat-inspector.py.pid',
                                   'mtime': datetime.datetime(2018, 1, 23, 8, 25, 14, 947736),
                                   'name': 'mentat-inspector.py',
                                   'paralel': False,
                                   'path': '/var/mentat/run/mentat-inspector.py.pid',
                                   'pid': 9362,
                                   'size': 5}},
    'mentat-storage.py': {9349: {'atime': datetime.datetime(2018, 1, 23, 8, 25, 16, 439742),
                                 'file': 'mentat-storage.py.pid',
                                 'mtime': datetime.datetime(2018, 1, 23, 8, 25, 12, 943727),
                                 'name': 'mentat-storage.py',
                                 'paralel': False,
                                 'path': '/var/mentat/run/mentat-storage.py.pid',
                                 'pid': 9349,
                                 'size': 5}}
}

Parameters: pid_dir_path (str) – path to directory containing PID files.
Returns: Structure containing parsed information for all PID files.
Return type: dict

mentat.system.analyze_process_list_ps()[source]¶

Analyze all running processes using Linux`s ps utility.

Processes are listed with following exact command:

/bin/ps axo pid,comm,args | grep mentat | grep -v grep

Only processes that match the internal regular expression are selected.

Example of returned structure:

{
    'mentat-enricher.py': {9356: {'args': None,
                           'exec': 'mentat-enricher.py',
                           'name': 'mentat-enricher.py',
                           'paralel': False,
                           'pid': 9356,
                           'process': 'python3',
                           'psline': '9356 python3 python3 /usr/local/bin/mentat-enricher.py'}},
    'mentat-inspector.py': {9362: {'args': None,
                            'exec': 'mentat-inspector.py',
                            'name': 'mentat-inspector.py',
                            'paralel': False,
                            'pid': 9362,
                            'process': 'python3',
                            'psline': '9362 python3 python3 /usr/local/bin/mentat-inspector.py'}},
    'mentat-storage.py': {9349: {'args': None,
                          'exec': 'mentat-storage.py',
                          'name': 'mentat-storage.py',
                          'paralel': False,
                          'pid': 9349,
                          'process': 'python3',
                          'psline': '9349 python3 python3 /usr/local/bin/mentat-storage.py'}}
}

Returns: Structured dictionary containing information about selected processess.
Return type: dict

mentat.system.analyze_process_ps(cmdline)[source]¶

Analyze given process command line received from ps utility.

Example of returned structure:

{
    'args': None,
    'exec': 'mentat-storage.py',
    'name': 'mentat-storage.py',
    'paralel': False,
    'pid': 9349,
    'process': 'python3',
    'psline': '9349 python3 python3 /usr/local/bin/mentat-storage.py'
}

Parameters: cmdline (str) – Command line as received from ps utility).
Returns: Structure containing parsed process information.
Return type: dict

mentat.system.analyze_runlog_file(runlog_file, runlog_file_path)[source]¶

Analyze given runlog file.

Example of returned structure:

{
    'atime': datetime.datetime(2018, 1, 22, 16, 33, 56, 766819),
    'data': {...},
    'file': '201801221611.runlog',
    'mtime': datetime.datetime(2018, 1, 22, 15, 30, 0, 427902),
    'name': '201801221611',
    'path': '/var/mentat/run/mentat-storage.py/201801221611.runlog',
    'size': 1209
}

Parameters

runlog_file (str) – Basename of the runlog file.
runlog_file_path (str) – Full path to the runlog file.

Returns

Structure containing parsed runlog file information.

Return type

dict

mentat.system.analyze_runlog_files(run_dir_path, limit=None)[source]¶

Analyze all runlog files in run directory.

Parameters: run_dir_path (str) – Path to directory containing run files.
Returns: Structure containing parsed runlog file information.
Return type: dict

mentat.system.analyze_versions()[source]¶: Analyze versions of various relevant dependencies (like PostgreSQL, …).

mentat.system.cronjob_status(cronj_data, cronf_data, overall_status)[source]¶: Analyze status of given cron module.

mentat.system.make_cronjob_list(cronjobs)[source]¶: Helper function

mentat.system.make_module_list(modules)[source]¶: Helper function

mentat.system.module_status(mod_data, pidf_data, proc_data, overall_status)[source]¶: Analyze status of given module.

mentat.system.system_status(modules, cronjobs, cfg_dir_path, cron_dir_path, log_dir_path, run_dir_path)[source]¶: Analyze status of all modules and cronjobs and detect additional problems.