The Mentat system is available as a package for Debian based systems via our custom internal repository. We are trying hard to keep the installation process as simple as possible, however at the same time we try not to force too many decisions on the users of the system. For that reason the Debian package for Mentat system does not pull and automatically install all required dependencies, but you will have to install couple of additional Debian system packages and/or Python libraries manually according to your setup and preferences.
As an example try to compare version of PostgreSQL database that is available via official Debian repositories with version available via repository operated by PostgreSQL development team. That is the reason why we did not add is directly to the list of Debian package dependencies.
Currently you have following installation options:
We are using HTTPS to serve our internal Debian package repository. You need
to have the
apt-transport-https package installed on your target host to
be able to access it.
Following is a list of most important system dependencies:
The OpenSource industry standard, high performance data logging and graphing system for time series data.
In Mentat system it is used for storing and displaying continuous system processing performance statistics.
This dependency will be istalled for you automatically by Debian packaging system.
PostgreSQL is a powerful, open source object-relational database system. It has more than 15 years of active development and a proven architecture that has earned it a strong reputation for reliability, data integrity, and correctness. It runs on all major operating systems, including Linux, UNIX (AIX, BSD, HP-UX, macOS, Solaris), and Windows. It is fully ACID compliant, has full support for foreign keys, joins, views, triggers, and stored procedures (in multiple languages). It includes most SQL:2008 data types, including INTEGER, NUMERIC, BOOLEAN, CHAR, VARCHAR, DATE, INTERVAL, and TIMESTAMP. It also supports storage of binary large objects, including pictures, sounds, or video. It has native programming interfaces for C/C++, Java, .Net, Perl, Python, Ruby, Tcl, ODBC, among others, and exceptional documentation.
In Mentat system it is used as backend service for persistent data storage.
You have to install this dependency manually according to your preferences. Alternativelly you can use prepared minimalistic Ansible role postgresql (see section Installation guide - Ansible roles below for more details).
Following is a list of most important Python dependencies:
- Python binding for MaxMind GeoIP2 API.
- A small but fast and easy to use stand-alone template engine written in pure Python.
- A collection of tools for internationalizing Python applications.
- WTForms is a flexible forms validation and rendering library for Python web development.
- Python Lex & Yacc.
- Python driver for PostgreSQL.
- Python universal DBAL and ORM toolkit.
- Database migration tool.
- Python bindings for RRDTool.
- Awesome Python microframework for web development.
- Python library for working with IP addressess.
- Python library providing typed collections.
- Python library for working with IDEA messages.
- Python3 command line widget library.
- Collection of usefull tools and utilities for creating console applications, scripts and system services (daemons) in Python 3.
- Python library for filtering, querying or inspecting almost arbitrary data structures.
All these dependencies can be quickly installed using
pip3 and prepared Python
pip3 install -r /etc/mentat/requirements.pip
Installation guide - Manual installation¶
The installation of base system is pretty straightforward, please follow these simple steps:
# Step 1: Install and configure PostgreSQL database. See official documentation or use our Ansible role. # Step 2: Download apt list file for mentat production package repository: wget -O /etc/apt/sources.list.d/mentat-production.list https://alchemist.cesnet.cz/mentat/deb/mentat-production.list # Step 3: Download and install GnuPG signing key for mentat production package repository: wget -O - https://alchemist.cesnet.cz/mentat/deb/mentat-production.gpg | apt-key add - # Step 4: Refresh apt cache: apt-get update # Step 5: List all packages available in newly added repository (optional): grep "Package:" /var/lib/apt/lists/alchemist.cesnet.cz_*mentat*_Packages | uniq # Step 6: Install the Mentat system: apt-get install mentat-ng # Step 7: Install additional Python requirements: pip3 install -r /etc/mentat/requirements.pip --upgrade # Step 8: Download IP geolocation databases: /etc/mentat/scripts/fetch-geoipdb.sh # Step 9: Create necessary databases and database user accounts: /etc/mentat/scripts/sqldb-init.sh # Step 10: Initialize PostgreSQL database schema: mentat-dbmngr.py --command init # Step 11: Stamp both metadata end event databases with latest migration revisions: /etc/mentat/scripts/sqldb-migrate.py db stamp head --directory /etc/mentat/migrations /etc/mentat/scripts/sqldb-migrate-e.sh stamp head # Step 12: Precache various event search form select option dictionaries: mentat-precache.py --allow-empty
If you wish to use the web interface Hawat, you wil have to install and configure a web server. You may use any server you like as long as it supports execution of WSGI Python3 scripts. Do not forget to setup some initial user accounts, either use the demonstration fixtures, or setup real user accounts, otherwise you will not be able to log in.
# Step 12: Install Apache web server: apt-get install apache2 libapache2-mod-wsgi-py3 # Step 13: Copy example Apache site configuration file and adjust it according to your needs: cp /etc/mentat/apache/site_mentat.conf.htpasswd.example /etc/apache2/sites-available/site_mentat.conf # Step 14: Configure Apache2 to run as mentat user in '/etc/apache2/envvars' file: sed -i.bak s/APACHE_RUN_USER=www-data/APACHE_RUN_USER=mentat/g /etc/apache2/envvars sed -i.bak s/APACHE_RUN_GROUP=www-data/APACHE_RUN_GROUP=mentat/g /etc/apache2/envvars rm /etc/apache2/envvars.bak # Step 15: Enable/disable appropriate Apache sites and modules: a2enmod rewrite a2enmod ssl a2enmod wsgi a2dissite 000-default.conf a2dissite default-ssl.conf a2ensite site_mentat.conf # !!!!! THIS STEP IS SUPER IMPORTANT !!!!! # Step 16: Change default SECRET_KEY in /etc/mentat/mentat-hawat.py.conf: # !!!!! THIS STEP IS SUPER IMPORTANT !!!!! vim /etc/mentat/mentat-hawat.py.conf # Step 17: Restart Apache service: systemctl restart apache2.service # Step 18: OPTION A: Insert demonstration data into Mentat`s metadata database # (this will insert some demo accounts, groups, networks and filters into db # and can be used in conjuction with htpasswd file above): mentat-dbmngr.py --command fixtures-add # Step 19: OPTION B: Create initial user account/s in Mentat`s metadata database # (please adjust the attributes, do not simply copy and paste): mentat-dbmngr.py --command user-add login=superman "fullname=Clark Kent" email@example.com "organization=Daily Planet, inc." roles=user,admin
For demonstration purposes Mentat package ships with preconfigured
file containing following user accounts (login - password):
- user - user
- manager - manager
- developer - developer
- admin - admin
This can be used in conjunction with database fixtures above (step 17, option A) to immediately start exploring the Mentat system. Please make sure NOT to use it in production environment!!!
# Step 12: Install Apache web server: apt-get install apache2 libapache2-mod-wsgi-py3 libapache2-mod-shib2 # Step 13: Copy example Apache site configuration file and adjust it according to your needs: cp /etc/mentat/apache/site_mentat.conf.shibboleth.example /etc/apache2/sites-available/site_mentat.conf # Step 14: Configure Apache2 to run as mentat user in '/etc/apache2/envvars' file: sed -i.bak s/APACHE_RUN_USER=www-data/APACHE_RUN_USER=mentat/g /etc/apache2/envvars sed -i.bak s/APACHE_RUN_GROUP=www-data/APACHE_RUN_GROUP=mentat/g /etc/apache2/envvars rm /etc/apache2/envvars.bak # Step 15: Enable/disable appropriate Apache sites and modules: a2enmod rewrite a2enmod ssl a2enmod shib2 a2enmod wsgi a2dissite 000-default.conf a2dissite default-ssl.conf a2ensite site_mentat.conf # !!!!! THIS STEP IS SUPER IMPORTANT !!!!! # Step 16: Change default SECRET_KEY in /etc/mentat/mentat-hawat.py.conf: # !!!!! THIS STEP IS SUPER IMPORTANT !!!!! vim /etc/mentat/mentat-hawat.py.conf # Step 17: Restart Apache service: systemctl restart apache2.service # Step 18: Create initial user account/s in Mentat`s metadata database # (please adjust the attributes, do not simply copy and paste): mentat-dbmngr.py --command user-add login=superman "fullname=Clark Kent" firstname.lastname@example.org "organization=Daily Planet, inc." roles=user,admin
Installation guide - Ansible roles¶
Please do not use Ansible roles for installation during migration from previous stable version of Mentat based on Perl and MongoDB. Some of the configuration files changed and you may encounter weird errors in case original configuration stays active. Part of the migration process is assessing changes in configuration files and this process is not automated and needs to be done by hand.
If you are fond of Ansible, there is great news for you: we have prepared Ansible roles to make the installation as simple as possible. These roles are available via official Ansible Galaxy site. They can be found easily using the galaxy search form. For the sake of completeness this is a short guide how to quickly use them, but you can use them as usual in your own role ecosystem.
First there is a basic role called mentat. This role will perform only basic Mentat installation. It covers steps 1 through 11 of manual installation with some additional actions like starting Mentat system and enabling it at system boot. Additionally it will install the PostgreSQL database for you through native dependency on postgresql role:
# Step 1: Create working directory: mkdir ansible-mentat && cd ansible-mentat && mkdir roles # Step 2: Download the roles: ansible-galaxy install honzamach.mentat --roles-path=./roles # Step 3: Pull the Mentat playbook to local directory: cp roles/honzamach.mentat/role_mentat.yml . # Step 4: Create inventory file 'inventory' with content similar to this: [servers-production] your.server.hostname [servers-mentat] your.server.hostname # Step 5: Execute the playbook: ansible-playbook -i inventory role_mentat.yml
And that is it. After a lot of crunching Mentat system should be installed on your target device. Again, this playbook installs only basic system without configuration of Apache webserver or creating any demonstration accounts.
If you would like to quickly bootstrap the demonstration installation of Mentat system including Hawat web interface and some preconfigured user accounts, groups etc., you may use another prepared role called mentat_demo. This role uses dependency on mentat role described above to perform basic installation and then builds on top of that:
# Step 1: Create working directory: mkdir ansible-mentat && cd ansible-mentat && mkdir roles # Step 2: Download the roles: ansible-galaxy install honzamach.mentat_demo --roles-path=./roles # Step 3: Pull the Mentat demo playbook to local directory: cp roles/honzamach.mentat_demo/role_mentat_demo.yml . # Step 4: Create inventory file 'inventory' with content similar to this: [servers-mentat-demo] your.server.hostname # Step 5: Execute the playbook: ansible-playbook -i inventory role_mentat_demo.yml
And voilà! Your Mentat demo server is up and running!
Please browse through the tasks of both playbooks to known what actions are actually being done on taget host. Ansible is designed to be human readable and easily understandable, so it does not make much sense to describe set of actions for each playbook. That would only duplicate the information.
|/etc/mentat||Configuration files for Mentat modules.|
|/etc/cron.d||Some Mentat modules are launched periodically via cron. When enabled, you may find them here prefixed with
|/var/mentat||Working directory for all Mentat modules. Pid files, runlogs, log files and other runtime data is located inside this directory.|
What is next?¶
OK. You have your Mentat system successfully installed, so what is next?
- If you are upgrading from previous version of Mentat system, you might wish to read and follow the Upgrading documentation page.
- If you are migrating from previous version of Mentat system, you might wish to read and follow the Migration documentation page.
- If you want to take quick tour of the Mentat system, you might wish to read and follow the Quickstart documentation page.
- If you want to read our recommendations and tips about administration and management of the Mentat system, you might wish to read and follow the Administration documentation page.
Installing HTTPS driver for apt¶
You might encounter following error when trying to install mentat-ng Debian package:
E: The method driver /usr/lib/apt/methods/https could not be found.
The problem is, that our Debian are server over HTTPS and you have to install appropriate driver for apt:
apt-get install apt-transport-https
Installing pip3 manually using get-pip.py¶
If you run into issues with pip3, because your native packaging system contains older versions and you are attempting to install a package requiring newer version, you might opt to uninstall the Debian version of pip3 and install latest manually using get-pip.py script: