Installation

The Mentat system is available as a package for Debian based systems via our custom internal repository. We are trying hard to keep the installation process as simple as possible, however at the same time we try not to force too many decisions on the users of the system. For that reason the Debian package for Mentat system does not pull and automatically install all required dependencies, but you will have to install couple of additional Debian system packages and/or Python libraries manually according to your setup and preferences.

Note

As an example try to compare version of PostgreSQL database that is available via official Debian repositories with version available via repository operated by PostgreSQL development team. That is the reason why we did not add is directly to the list of Debian package dependencies.

Currently you have following installation options:

  1. Installation guide - Manual installation
  2. Installation guide - Ansible roles

Note

We are using HTTPS to serve our internal Debian package repository. You need to have the apt-transport-https package installed on your target host to be able to access it.

Dependencies

System dependencies

Following is a list of most important system dependencies:

RRDTool

The OpenSource industry standard, high performance data logging and graphing system for time series data.

In Mentat system it is used for storing and displaying continuous system processing performance statistics.

This dependency will be istalled for you automatically by Debian packaging system.

PostgreSQL

PostgreSQL is a powerful, open source object-relational database system. It has more than 15 years of active development and a proven architecture that has earned it a strong reputation for reliability, data integrity, and correctness. It runs on all major operating systems, including Linux, UNIX (AIX, BSD, HP-UX, macOS, Solaris), and Windows. It is fully ACID compliant, has full support for foreign keys, joins, views, triggers, and stored procedures (in multiple languages). It includes most SQL:2008 data types, including INTEGER, NUMERIC, BOOLEAN, CHAR, VARCHAR, DATE, INTERVAL, and TIMESTAMP. It also supports storage of binary large objects, including pictures, sounds, or video. It has native programming interfaces for C/C++, Java, .Net, Perl, Python, Ruby, Tcl, ODBC, among others, and exceptional documentation.

In Mentat system it is used as backend service for persistent data storage.

You have to install this dependency manually according to your preferences. Alternativelly you can use prepared minimalistic Ansible role postgresql (see section Installation guide - Ansible roles below for more details).

Python dependencies

Following is a list of most important Python dependencies:

geoip2
Python binding for MaxMind GeoIP2 API.
Jinja2
A small but fast and easy to use stand-alone template engine written in pure Python.
Babel
A collection of tools for internationalizing Python applications.
WTForms
WTForms is a flexible forms validation and rendering library for Python web development.
ply
Python Lex & Yacc.
psycopg2
Python driver for PostgreSQL.
SQLAlchemy
Python universal DBAL and ORM toolkit.
rrdtool
Python bindings for RRDTool.
Flask
Awesome Python microframework for web development.
ipranges
Python library for working with IP addressess.
typedcols
Python library providing typed collections.
idea-format
Python library for working with IDEA messages.
Pydgets
Python3 command line widget library.
PyZenKit
Collection of usefull tools and utilities for creating console applications, scripts and system services (daemons) in Python 3.
Pynspect
Python library for filtering, querying or inspecting almost arbitrary data structures.

All these dependencies can be quickly installed using pip3 and prepared Python requirements file /etc/mentat/requirements.pip:

pip3 install -r /etc/mentat/requirements.pip

Installation guide - Manual installation

The installation of base system is pretty straightforward, please follow these simple steps:

# Step 1: Install and configure PostgreSQL database. See official documentation or use our Ansible role.

# Step 2: Download apt list file for mentat production package repository:
wget -O /etc/apt/sources.list.d/mentat-production.list https://alchemist.cesnet.cz/mentat/deb/mentat-production.list

# Step 3: Download and install GnuPG signing key for mentat production package repository:
wget -O - https://alchemist.cesnet.cz/mentat/deb/mentat-production.gpg | apt-key add -

# Step 4: Refresh apt cache:
apt-get update

# Step 5: List all packages available in newly added repository (optional):
grep "Package:" /var/lib/apt/lists/alchemist.cesnet.cz_*mentat*_Packages | uniq

# Step 6: Install the Mentat system:
apt-get install mentat-ng

# Step 7: Install additional Python requirements:
pip3 install -r /etc/mentat/requirements.pip --upgrade

# Step 8: Download IP geolocation databases:
/etc/mentat/scripts/fetch-geoipdb.sh

# Step 9: Create necessary databases and database accounts:
/etc/mentat/scripts/sqldb-init.sh

# Step 10: Initialize PostgreSQL database schema:
mentat-dbmngr.py --command init

# Step 11: Precache various event search form select option dictionaries:
mentat-precache.py --allow-empty

If you wish to use the web interface Hawat, you wil have to install and configure a web server. You may use any server you like as long as it supports execution of WSGI Python3 scripts. Do not forget to setup some initial user accounts, either use the demonstration fixtures, or setup real user accounts, otherwise you will not be able to log in.

# Step 12: Install Apache web server:
apt-get install apache2 libapache2-mod-wsgi-py3

# Step 13: Copy example Apache site configuration file and adjust it according to your needs:
cp /etc/mentat/apache/site_mentat.conf.htpasswd.example /etc/apache2/sites-available/site_mentat.conf

# Step 14: Configure Apache2 to run as mentat user in '/etc/apache2/envvars' file:
sed -i.bak s/APACHE_RUN_USER=www-data/APACHE_RUN_USER=mentat/g /etc/apache2/envvars
sed -i.bak s/APACHE_RUN_GROUP=www-data/APACHE_RUN_GROUP=mentat/g /etc/apache2/envvars
rm /etc/apache2/envvars.bak

# Step 15: Enable/disable appropriate Apache sites and modules:
a2enmod rewrite
a2enmod ssl
a2enmod wsgi
a2dissite 000-default.conf
a2dissite default-ssl.conf
a2ensite site_mentat.conf

# !!!!! THIS STEP IS SUPER IMPORTANT !!!!!
# Step 16: Change default SECRET_KEY in /etc/mentat/mentat-hawat.py.conf:
# !!!!! THIS STEP IS SUPER IMPORTANT !!!!!
vim /etc/mentat/mentat-hawat.py.conf

# Step 17: Restart Apache service:
systemctl restart apache2.service

# Step 18: OPTION A: Insert demonstration data into Mentat`s metadata database
# (this will insert some demo accounts, groups, networks and filters into db
# and can be used in conjuction with htpasswd file above):
mentat-dbmngr.py --command fixtures-add

# Step 19: OPTION B: Create initial user account/s in Mentat`s metadata database
# (please adjust the attributes, do not simply copy and paste):
mentat-dbmngr.py --command user-add login=superman "fullname=Clark Kent" email=kent@dailyplanet.com "organization=Daily Planet, inc." roles=user,admin

Warning

For demonstration purposes Mentat package ships with preconfigured htpasswd file containing following user accounts (login - password):

  • user - user
  • manager - manager
  • developer - developer
  • admin - admin

This can be used in conjunction with database fixtures above (step 17, option A) to immediately start exploring the Mentat system. Please make sure NOT to use it in production environment!!!

Following recipe is another example installation of Apache web server, this time with Shibboleth SSO login service (this SSO login service is used for example in Czech eduID identity federation):

# Step 12: Install Apache web server:
apt-get install apache2 libapache2-mod-wsgi-py3 libapache2-mod-shib2

# Step 13: Copy example Apache site configuration file and adjust it according to your needs:
cp /etc/mentat/apache/site_mentat.conf.shibboleth.example /etc/apache2/sites-available/site_mentat.conf

# Step 14: Configure Apache2 to run as mentat user in '/etc/apache2/envvars' file:
sed -i.bak s/APACHE_RUN_USER=www-data/APACHE_RUN_USER=mentat/g /etc/apache2/envvars
sed -i.bak s/APACHE_RUN_GROUP=www-data/APACHE_RUN_GROUP=mentat/g /etc/apache2/envvars
rm /etc/apache2/envvars.bak

# Step 15: Enable/disable appropriate Apache sites and modules:
a2enmod rewrite
a2enmod ssl
a2enmod shib2
a2enmod wsgi
a2dissite 000-default.conf
a2dissite default-ssl.conf
a2ensite site_mentat.conf

# !!!!! THIS STEP IS SUPER IMPORTANT !!!!!
# Step 16: Change default SECRET_KEY in /etc/mentat/mentat-hawat.py.conf:
# !!!!! THIS STEP IS SUPER IMPORTANT !!!!!
vim /etc/mentat/mentat-hawat.py.conf

# Step 17: Restart Apache service:
systemctl restart apache2.service

# Step 18: Create initial user account/s in Mentat`s metadata database
# (please adjust the attributes, do not simply copy and paste):
mentat-dbmngr.py --command user-add login=superman "fullname=Clark Kent" email=kent@dailyplanet.com "organization=Daily Planet, inc." roles=user,admin

Installation guide - Ansible roles

Note

Please do not use Ansible roles for installation during migration from previous stable version of Mentat based on Perl and MongoDB. Some of the configuration files changed and you may encounter weird errors in case original configuration stays active. Part of the migration process is assessing changes in configuration files and this process is not automated and needs to be done by hand.

If you are fond of Ansible, there is great news for you: we have prepared Ansible roles to make the installation as simple as possible. These roles are available via official Ansible Galaxy site. They can be found easily using the galaxy search form. For the sake of completeness this is a short guide how to quickly use them, but you can use them as usual in your own role ecosystem.

First there is a basic role called mentat. This role will perform only basic Mentat installation. It covers steps 1 through 11 of manual installation with some additional actions like starting Mentat system and enabling it at system boot. Additionally it will install the PostgreSQL database for you through native dependency on postgresql role:

# Step 1: Create working directory:
mkdir ansible-mentat && cd ansible-mentat && mkdir roles

# Step 2: Download the roles:
ansible-galaxy install honzamach.mentat --roles-path=./roles

# Step 3: Pull the Mentat playbook to local directory:
cp roles/honzamach.mentat/role_mentat.yml .

# Step 4: Create inventory file 'inventory' with content similar to this:
[servers-production]
your.server.hostname

[servers-mentat]
your.server.hostname

# Step 5: Execute the playbook:
ansible-playbook -i inventory role_mentat.yml

And that is it. After a lot of crunching Mentat system should be installed on your target device. Again, this playbook installs only basic system without configuration of Apache webserver or creating any demonstration accounts.

If you would like to quickly bootstrap the demonstration installation of Mentat system including Hawat web interface and some preconfigured user accounts, groups etc., you may use another prepared role called mentat_demo. This role uses dependency on mentat role described above to perform basic installation and then builds on top of that:

# Step 1: Create working directory:
mkdir ansible-mentat && cd ansible-mentat && mkdir roles

# Step 2: Download the roles:
ansible-galaxy install honzamach.mentat_demo --roles-path=./roles

# Step 3: Pull the Mentat demo playbook to local directory:
cp roles/honzamach.mentat_demo/role_mentat_demo.yml .

# Step 4: Create inventory file 'inventory' with content similar to this:
[servers-mentat-demo]
your.server.hostname

# Step 5: Execute the playbook:
ansible-playbook -i inventory role_mentat_demo.yml

And voilà! Your Mentat demo server is up and running!

Note

Please browse through the tasks of both playbooks to known what actions are actually being done on taget host. Ansible is designed to be human readable and easily understandable, so it does not make much sense to describe set of actions for each playbook. That would only duplicate the information.

Directory layout

Location Content
/etc/mentat Configuration files for Mentat modules.
/etc/cron.d Some Mentat modules are launched periodically via cron. When enabled, you may find them here prefixed with mentat-*.
/var/mentat Working directory for all Mentat modules. Pid files, runlogs, log files and other runtime data is located inside this directory.
/usr/local/bin Executable files.

What is next?

OK. You have your Mentat system successfully installed, so what is next?

  • If you are upgrading from previous version of Mentat system, you might wish to read and follow the Upgrading documentation page.
  • If you are migrating from previous version of Mentat system, you might wish to read and follow the Migration documentation page.
  • If you want to take quick tour of the Mentat system, you might wish to read and follow the Quickstart documentation page.
  • If you want to read our recommendations and tips about administration and management of the Mentat system, you might wish to read and follow the Administration documentation page.

Troubleshooting

Installing HTTPS driver for apt

You might encounter following error when trying to install mentat-ng Debian package:

E: The method driver /usr/lib/apt/methods/https could not be found.

The problem is, that our Debian are server over HTTPS and you have to install appropriate driver for apt:

apt-get install apt-transport-https

Installing pip3 manually using get-pip.py

If you run into issues with get-pip.py, because your native packaging system contains older versions and you are attempting to install a package requiring newer version, you might opt to uninstall the Debian version of pip3 and install latest manually.