pyzenkit.baseapp module¶
This module provides base implementation of generic console application represented
by the pyzenkit.baseapp.BaseApp
class with many usefull features including
(but not limited to) the following:
- Application configuration service
The base application provides tools for loading configurations from multiple sources and merging them all into single dictionary. This is then available to developers as public class attribute
config
.Currently the following configuration sources are available:
- Optional configuration via configuration directory. All JSON files in given directory are loaded and then merged together.
- Optional configuration via single JSON configuration file.
- Optional configuration via command line arguments and options.
- Command line argument parsing service
- The base application preconfigures an instance of standard
argparse.ArgumentParser
class for parsing command line arguments and prepopulates in with a set of built-in options and arguments. This instance can be then further modified and enhanced in subclass by the user. - Logging service
The base application is capable of automated setup of
logging.Logger
logging service. Logger parameters like threshold level, or target file name are fully configurable. Currently following destinations are supported:- Optional logging to console.
- Optional logging to text file.
- Persistent state service
- The base application contains optional persistent state feature, which is intended for storing or passing data between multiple executions of the same application. The feature is implemented as a simple dictionary, that is populated from simple JSON file on startup and written back on teardown.
- Application runlog service
- The base application provides optional runlog service, which is a intended to provide storage for relevant data and results during the processing and enable further analysis later. The feature is implemented as simple dictionary, that is written into JSON file on teardown.
- Plugin system
- The base application provides tools for writing and using plugins, that can be used to further enhance the functionality of application and improve code reusability by composing the application from smaller building blocks.
- Application actions
- The base application provides tools for quick actions. These actions are intended to be used for global application management tasks such as vieving or validating configuration without executing the application itself, listing or evaluating runlogs and so on. There is a number of built-in actions and more can be implemented very easily.
The application is designed to provide all of these usefull features by default and with as less as possible work, while also maintaining high customizability. This goal is achived by following measures:
- Most of the hardcoded features are somehow customizable by configuration file keys or command line options.
- There are many callback hooks prepared for subclasses, that can be used to add the desired functionality.
- If you are more familiar with the code, you may override the default implementation of almost any method and provide your own functionality, or call the parent implementation at some point in the new method.
Please browse the source code, there is an example implementation in
pyzenkit.baseapp.DemoBaseApp
class.
Application usage modes¶
Applications created using this framework can be utilized in two work modes:
- run
- plugin
In the run mode all application features are initialized and configured and desired action or application processing code is immediatelly executed.
In the plugin mode the application is only initialized and configured and any other interactions must be performed manually by calling appropriate methods. This approach enables users to plug one apllication into another one on a wider scope. One example use case of this feature may be the implementation of an user command line interface that controls multiple applications (for example like git executable controls almost everything in Git universe).
Application actions¶
Applications created based on this framework come with built-in support for actions. The action is a designation for some kind of support task, that does not relate to actual purpose of the application. The actions however are executed within the same environment and have access to the whole application, so they are perfect for simple tasks like configuration validation. The actions are also much more simple and when executing an action the application does not go through all life-cycle stages (see below).
Currently following actions are built-in and supported:
- config-view
- Display current configuration tree.
- runlog-dump
- Simple dump of chosen application runlog.
- runlog-view
- View chosen application runlog (nicer display with calculated statistics).
- runlogs-dump
- Simple dump of all application runlogs.
- runlogs-list
- View list of all application runlogs.
- runlogs-evaluate
- View evaluated statistics of all application runlogs.
Actions may be executed by selecting desired action by command line argument:
path/to/baseapp.py --action config-view
path/to/baseapp.py --action=config-view
Action may be selected permanently inside configuration file under the dictionary
key action
.
Application life-cycle¶
Depending on the mode of operation (run or plugin) the application code goes
through a different set of stages during its life span. See subsections below for
more details on each life-cycle stage. The life-cycle is implemented partly in the
pyzenkit.baseapp.BaseApp.__init__()
method and mostly in
pyzenkit.baseapp.BaseApp.run()
method.
In the plugin mode the following stages are performed:
- __init__
- setup
In the run mode the following stages are performed in case some action is being handled:
- __init__
- setup
- action
In the run mode the following stages are performed in case of normal processing:
- __init__
- setup
- process
- evaluate
- teardown
Stage __init__¶
The __init__ stage is responsible for creating and basic default initialization of application object. No exception should occur or be raised during the initialization stage and the code should always work regardles of environment setup, so no files should be opened, etc. Any exception during this stage will intentionally not get handled in any way and will result in full traceback dump and immediate application termination.
All more advanced setup tasks should be performed during the setup stage, which is capable of intelligent catching and displaying/logging of any exceptions.
This stage is implemented in the BaseApp.__init__()
method and there are
following substages in this stage:
- init command line argument parser:
BaseApp._init_argparser()
- parse command line arguments:
BaseApp._parse_cli_arguments()
- initialize application name:
BaseApp._init_name()
- initialize filesystem paths:
BaseApp._init_paths()
- initialize application runlog:
BaseApp._init_runlog()
- initialize default configurations:
BaseApp._init_config()
- subclass hook for additional initializations:
BaseApp._sub_stage_init()
Any of the previous substages may be overriden in a subclass to enhance or alter the functionality, but always be sure of what you are doing.
Stage setup¶
The setup stage is responsible for bootstrapping and configuring of the whole
application. Any exception, that is the instance of ZenAppSetupException
will be catched, only simple message will be displayed to the user and application
will terminate. This use case represents the case when the error is on the user`s side,
for example non-existent configuration file, etc. In any other case the application will
terminate with full traceback print.
This stage is implemented in the BaseApp._stage_setup()
method and there
are following substages in this stage:
- setup configuration:
BaseApp._stage_setup_configuration()
- setup user and group privileges:
BaseApp._stage_setup_privileges()
- setup logging:
BaseApp._stage_setup_logging()
- setup persistent state:
BaseApp._stage_setup_pstate()
- setup plugins:
BaseApp._stage_setup_plugins()
- subclass hook for additional setup:
BaseApp._sub_stage_setup()
- setup dump:
BaseApp._stage_setup_dump()
Any of the previous substages may be overriden in a subclass to enhance or alter the functionality, but always be sure of what you are doing.
Stage action¶
The action stage takes care of executing built-in actions. See appropriate section above for list of all available built-in actions.
More actions can be implemented very easily. The action callback methods just have to follow these requirements to be autodetected by the application engine:
- Action callback must be method without any mandatory arguments.
- Action callback method name must begin with
cbk_action_
prefix. - Action name in the method name after the prefix must also be snake_cased`.
- Action name will be calculated by replacing
_
with-
.
Following are examples of valid action callbacks:
cbk_action_test(self): # Will be mapped to 'test' action.
cbk_action_another_test(self): # Will be mapped to 'another-test' action.
When a method is implemented according to these guidelines, it will be automatically recognized and executable.
This stage is implemented in the BaseApp._stage_action()
method.
Please see the implementation of existing built-in actions for examples.
Stage process¶
The process stage is supposed to perform the required task(s). This stage is
implemented in the BaseApp._stage_process()
method. It is a wrapper
method, that takes care of exception catching and the application must provide
its own implementation of template method BaseApp._sub_stage_process()
,
which is being called from the wrapper and should contain actual processing code.
Stage evaluate¶
The evaluate stage is supposed to perform any required evaluations of current runlog. Currently, there are no built-in evaluations.
Stage teardown¶
The teardown stage is supposed to perform any cleanup tasks before the application exits.
This stage is implemented in the BaseApp._stage_teardown()
method and there
are following substages in this stage:
- subclass hook for additional teardown actions:
BaseApp._sub_stage_teardown()
- save persistent state:
BaseApp._stage_teardown_pstate()
- save runlog:
BaseApp._stage_teardown_runlog()
Module contents¶
Programming API¶
The application features mentioned in this document are available to the user/developer
either as public attributes of the pyzenkit.baseapp.BaseApp
class,
or as public methods:
public attributes:
BaseApp.name
- Name of the application.BaseApp.paths
- Paths to various locations, like config or temp directories.BaseApp.runlog
- Writable dictionary containing the application runlog.BaseApp.config
- Dictionary containing final merged application configuration.BaseApp.logger
- Configured instance oflogging.Logger
logger.BaseApp.pstate
- Writable dictionary containing the application persistent state.BaseApp.retc
- Exit status code as integer.
public methods:
BaseApp.c()
- Shortcut method for accessing the self.config.get(key, default)BaseApp.cc()
- Shortcut method for accessing the self.config[self.CORE].get(key, default)BaseApp.p()
- Method for printing to terminal honoring theverbose
setting.BaseApp.dbgout()
- Method for printing additional debug messages honoring thedebug
setting.BaseApp.excout()
- Method for printing exception without traceback and terminating the application.BaseApp.error()
- Register given error into application.BaseApp.json_dump()
- Utility method for dumping data structure into JSON string.BaseApp.json_load()
- Utility method for loading JSON file.BaseApp.json_save()
- Utility method for writing data structure into JSON file.
Subclass extension hooks¶
The base application provides following extension hooks, that can be used in subclasses to enhance the base functionality. There is one hook for each application life-cycle stage:
BaseApp._sub_stage_init()
BaseApp._sub_stage_setup()
BaseApp._sub_stage_process()
BaseApp._sub_stage_teardown()
There are also couple of methods for runlog analysis and evaluation, that can be
used to enhance the default output of runlog-*
built-in actions:
BaseApp._sub_runlog_analyze()
BaseApp._sub_runlog_format_analysis()
BaseApp._sub_runlogs_evaluate()
BaseApp._sub_runlogs_format_evaluation()
-
class
pyzenkit.baseapp.
BaseApp
(**kwargs)[source]¶ Bases:
object
Base implementation of generic executable application. This class attempts to provide robust and stable framework, which can be used to writing all kinds of scripts or daemons. Although is is usable, this is however a low level framework and should not be used directly, use the
pyzenkit.zenscript
orpyzenkit.zendaemon
modules for writing custom scripts or daemons respectively. That being said, thepyzenkit.baseapp.DemoBaseApp
class is an example implementation of using this class directly without any additional overhead.-
CONFIG_ACTION
= 'action'¶
-
CONFIG_CFG_DIR
= 'config_dir'¶
-
CONFIG_CFG_DIR_S
= 'config_dir_silent'¶
-
CONFIG_CFG_FILE
= 'config_file'¶
-
CONFIG_CFG_FILE_S
= 'config_file_silent'¶
-
CONFIG_DEBUG
= 'debug'¶
-
CONFIG_GROUP
= 'group'¶
-
CONFIG_INPUT
= 'input'¶
-
CONFIG_LIMIT
= 'limit'¶
-
CONFIG_LOG_FILE
= 'log_file'¶
-
CONFIG_LOG_LEVEL
= 'log_level'¶
-
CONFIG_NAME
= 'name'¶
-
CONFIG_PLUGINS
= 'plugins'¶
-
CONFIG_PSTATE_DUMP
= 'pstate_dump'¶
-
CONFIG_PSTATE_FILE
= 'pstate_file'¶
-
CONFIG_PSTATE_LOG
= 'pstate_log'¶
-
CONFIG_QUIET
= 'quiet'¶
-
CONFIG_RUNLOG_DIR
= 'runlog_dir'¶
-
CONFIG_RUNLOG_DUMP
= 'runlog_dump'¶
-
CONFIG_RUNLOG_LOG
= 'runlog_log'¶
-
CONFIG_USER
= 'user'¶
-
CONFIG_VERBOSITY
= 'verbosity'¶
-
CORE
= '__core__'¶
-
CORE_LOGGING
= 'logging'¶
-
CORE_LOGGING_LEVEL
= 'level'¶
-
CORE_LOGGING_LEVELC
= 'level_console'¶
-
CORE_LOGGING_LEVELF
= 'level_file'¶
-
CORE_LOGGING_TOCONS
= 'to_console'¶
-
CORE_LOGGING_TOFILE
= 'to_file'¶
-
CORE_PSTATE
= 'pstate'¶
-
CORE_PSTATE_SAVE
= 'save'¶
-
CORE_RUNLOG
= 'runlog'¶
-
CORE_RUNLOG_SAVE
= 'save'¶
-
FLAG_DEBUG
= False¶
-
PATH_BIN
= 'bin'¶
-
PATH_CFG
= 'cfg'¶
-
PATH_LOG
= 'log'¶
-
PATH_RUN
= 'run'¶
-
PATH_TMP
= 'tmp'¶
-
PATH_VAR
= 'var'¶
-
PTRN_ACTION_CBK
= 'cbk_action_'¶
-
PTRN_APP_NAME
= '^[_a-zA-Z][-_a-zA-Z0-9.]*$'¶
-
RC_FAILURE
= 1¶
-
RC_SUCCESS
= 0¶
-
RESULT_FAILURE
= 'failure'¶
-
RESULT_SUCCESS
= 'success'¶
-
RLANKEY_AGE
= 'age'¶
-
RLANKEY_COMMAND
= 'command'¶
-
RLANKEY_DURATIONS
= 'durations'¶
-
RLANKEY_DURPOST
= 'dur_post'¶
-
RLANKEY_DURPRE
= 'dur_pre'¶
-
RLANKEY_DURPROC
= 'dur_proc'¶
-
RLANKEY_DURRUN
= 'dur_run'¶
-
RLANKEY_EFFECTIVITY
= 'effectivity'¶
-
RLANKEY_LABEL
= 'label'¶
-
RLANKEY_RESULT
= 'result'¶
-
RLANKEY_RUNLOG
= 'runlog'¶
-
RLEVKEY_ANALYSES
= 'analyses'¶
-
RLEVKEY_AVGDURPROC
= 'avg_dur_proc'¶
-
RLEVKEY_AVGDURRUN
= 'avg_dur_run'¶
-
RLEVKEY_AVGEFFECT
= 'avg_effectivity'¶
-
RLEVKEY_MAXDURPROC
= 'max_dur_proc'¶
-
RLEVKEY_MAXDURRUN
= 'max_dur_run'¶
-
RLEVKEY_MAXEFFECT
= 'max_effectivity'¶
-
RLEVKEY_MINDURPROC
= 'min_dur_proc'¶
-
RLEVKEY_MINDURRUN
= 'min_dur_run'¶
-
RLEVKEY_MINEFFECT
= 'min_effectivity'¶
-
RLKEY_ARGV
= 'argv'¶
-
RLKEY_ERRORS
= 'errors'¶
-
RLKEY_NAME
= 'name'¶
-
RLKEY_PID
= 'pid'¶
-
RLKEY_RC
= 'rc'¶
-
RLKEY_RESULT
= 'result'¶
-
RLKEY_TMARKS
= 'time_marks'¶
-
RLKEY_TS
= 'ts'¶
-
RLKEY_TSFSF
= 'ts_fsf'¶
-
RLKEY_TSSTR
= 'ts_str'¶
-
argparser
= None¶ [PUBLIC] Initialize command line argument parser.
-
c
(key, default=None)[source]¶ Shortcut method: Get given configuration value, shortcut for:
self.config.get(key, default)Parameters: - key (str) – Name of the configuration value.
- default – Default value to be returned when key is not set.
Returns: Configuration value fogr given key.
-
cc
(key, default=None)[source]¶ Shortcut method: Get given core configuration value, shortcut for:
self.config[self.CORE].get(key, default)Core configurations are special configurations under configuration key
__CORE__
, which may only either be hardcoded, or calculated from other configurations.Parameters: - key (str) – Name of the core configuration value.
- default – Default value to be returned when key is not set.
Returns: Core configuration value fogr given key.
-
config
= None¶ [PUBLIC] Application configuration dictionary.
-
static
dbgout
(message)[source]¶ Routine for printing additional debug messages. The given message will be printed only in case the static class variable
FLAG_DEBUG
flag is set toTrue
. This can be done either by explicit assignment in code, or using command line argument--debug
, which is evaluated ASAP and sets the variable toTrue
. The message will be printed tosys.stderr
.Parameters: message (str) – Message do be written.
-
description
= None¶ [PUBLIC] Default application help description.
-
static
draw_progress_bar
(percent, bar_len=50)[source]¶ Draw progress bar on standard output terminal.
-
error
(error, retc=None, trcb=None)[source]¶ Register given error, that occured during application run. Registering in the case of this method means printing the error message to logging facility, storing the message within the appropriate runlog data structure, generating the traceback when required and altering the runlog result and return code attributes accordingly.
Parameters:
-
static
excout
(exception, retc=None)[source]¶ Routine for displaying the exception message to the user without traceback and terminating the application. This routine is intended to display information about application errors, that are not caused by the application code itself (like missing configuration file, non-writable directories, etc.) and that can not be logged because of the fact, that the logging service was not yet initialized. For that reason this method is used to handle exceptions during the __init__ and setup stages.
Parameters:
-
classmethod
get_resource_path
(fs_path, *more_chunks)[source]¶ Return filesystem path to application resource with
APP_ROOT_PATH
taken into consideration. Iffs_path
is absolute theAPP_ROOT_PATH
will be ignored as usual.
-
classmethod
get_resource_path_fr
(fs_path, *more_chunks)[source]¶ Force given application filesystem path to be relative to
APP_ROOT_PATH
.
-
static
json_dump
(data, **kwargs)[source]¶ Dump given data structure into JSON string.
Parameters: - data (dict) – Data to be dumped to JSON.
- kwargs – Optional arguments to pass to
pyzenkit.jsonconf.json_dump()
method.
Returns: Data structure as JSON string.
Return type:
-
static
json_load
(json_file)[source]¶ Load data structure from given JSON file.
Parameters: json_file (str) – Name of the JSON file to read from. Returns: Loaded data structure. Return type: dict
-
static
json_save
(json_file, data, **kwargs)[source]¶ Save given data structure into given JSON file.
Parameters: - json_file (str) – Name of the JSON file to write to.
- data (dict) – Data to be dumped to JSON.
- kwargs – Optional arguments to pass to
pyzenkit.jsonconf.json_save()
method.
Returns: Always returns
True
.Return type:
-
logger
= None¶ [PUBLIC] Preconfigured
logging.Logger
object.
-
name
= None¶ [PUBLIC] Name of the application, autodetected, or forced by object constructor arguments.
-
p
(string, level=0)[source]¶ Print given string to
sys.stdout
with respect toquiet
andverbosity
settings.Parameters:
-
paths
= None¶ [PUBLIC] Application paths that will be used to construct various absolute file paths.
-
plugin
()[source]¶ APPLICATION MODE: Plugin mode - Main processing method.
This method allows the object to be used as plugin within larger framework. Only the necessary setup is performed.
-
pstate
= None¶ [PUBLIC] Persistent state dictionary.
-
retc
= None¶ [PUBLIC] Final return code as integer.
-
run
()[source]¶ APPLICATION MODE: Standalone application mode - Main processing method.
- Run as standalone application, performs all stages of object life-cycle:
- 1. setup stage 2.1 action stage 2.2.1 processing stage 2.2.2 evaluation stage 2.2.3 teardown stage
-
runlog
= None¶ [PUBLIC] Application processing runlog.
-
-
class
pyzenkit.baseapp.
DemoBaseApp
(name=None, description=None)[source]¶ Bases:
pyzenkit.baseapp.BaseApp
Minimalistic class for demonstration purposes. Study implementation of this class for tutorial on how to use this framework.
-
exception
pyzenkit.baseapp.
ZenAppEvaluateException
(description, **params)[source]¶ Bases:
pyzenkit.baseapp.ZenAppException
Describes problems or errors that occur during the evaluate phase.
-
exception
pyzenkit.baseapp.
ZenAppException
(description, **params)[source]¶ Bases:
Exception
Base class for all ZenApp custom exceptions.
When appropriate, these exceptions will be catched, error will be displayed to the user and the application will attempt to gracefully terminate without dumping the traceback visibly to the user. These exceptions should be used for anticipated errors, which can occur during normal application execution and do not mean there is anything wrong with the code itself, for example missing configuration file, etc…
-
class
pyzenkit.baseapp.
ZenAppPlugin
[source]¶ Bases:
object
Base class for all ZenApp application plugins. Plugins can be used to further enhance the code reusability by composing the application from smaller building blocks.
-
init_argparser
(app, argparser, **kwargs)[source]¶ Callback to be called during argparser initialization phase.
-
init_config
(app, config, **kwargs)[source]¶ Callback to be called during default configuration initialization phase.
-
-
exception
pyzenkit.baseapp.
ZenAppProcessException
(description, **params)[source]¶ Bases:
pyzenkit.baseapp.ZenAppException
Describes problems or errors that occur during the process phase.
-
exception
pyzenkit.baseapp.
ZenAppSetupException
(description, **params)[source]¶ Bases:
pyzenkit.baseapp.ZenAppException
Describes problems or errors that occur during the setup phase.
-
exception
pyzenkit.baseapp.
ZenAppTeardownException
(description, **params)[source]¶ Bases:
pyzenkit.baseapp.ZenAppException
Describes problems or errors that occur during the teardown phase.