mentat.module.netmngr module

This Mentat module is a script providing functions for abuse group network management for Mentat system database.

This script is implemented using the pyzenkit.zenscript framework and so it provides all of its core features. See the documentation for more in-depth details.

Note

Still work in progress, use with caution.

Usage examples

# Display help message and exit.
mentat-netmngr.py --help

# Run in debug mode (enable output of debugging information to terminal).
mentat-netmngr.py --debug

# Run with increased logging level.
mentat-netmngr.py --log-level debug

Available script commands

status (default)

Detect and display the state of internal whois database contents according to the data in given reference whois file.

update

Attempt to update the state of internal whois database contents according to the data in given reference whois file.

convert-exceptions

Attempt to convert given list of exception files into valid whois file.

Custom configuration

Custom command line options

--whois-file file-path

Path to reference whois file containing network data.

Type: string, default: None

--source

Origin of the whois file.

Type: string, default: whois

Custom config file options

exception_files

List of paths to exception files and their appropriate abuse groups.

Example configuration:

"exception_files": [
    {
        "path":  "/path/to/file",
        "abuse": "abuse_group_id"
    }
],

Type: list of dicts, default: None

Exception file format

The exception file is an ordinary text file containing single IPv(4|6) address|network|range on each line. Blank lines and lines beginning with # are ignored. Whois exception files are very easy to be generated and they are meant for specifying whois resolving exceptions. For example you may use it to describe hosts with addresses from the domain of one particular abuse group, but actually belonging to different group. This might be the case of routers belonging to service provider but residing within the network address space of the customer. Another example may be nodes of some cloud computing service that have addresses from address space of the cloud computing organization member.

Whois file format

Whois file is an ordinary text file containg whois information in specific structured way. It is recognized by the mentat.services.whois.FileWhoisModule and can be used for whois resolving.

The structure of the data comes from the export format of CESNET’s Negistry tool, which is an internal custom copy of relevant RIPE whois data. It is JSON based format. Following example content describes multiple valid syntaxes for describing network records:

[
    # Option 1: Pass IP4 start and end addresses
    {
        "primary_key": "78.128.128.0 - 78.128.255.255",
        "ip4_start": "78.128.128.0",
        "ip4_end": "78.128.255.255",
        "netnames": ["CZ-TEN-34-20070410"],
        "resolved_abuses": {
          "low": [
            "abuse@cesnet.cz"
          ]
        }
    },

    # Option 2: Pass network CIDR or range and type
    {
        "primary_key": "78.128.212.64 - 78.128.212.127",
        "network": "78.128.212.64/26",
        "type": "ipv4",
        "netnames": ["CESNET-HSM4"],
        "descr": [
          "CESNET, z.s.p.o.",
          "Ostrava"
        ],
        "resolved_abuses": {
          "low": [
            "abuse@cesnet.cz"
          ]
        }
    },

    # Option 3: Pass IP6 address and prefix
    {
        "primary_key": "2001:718::/29",
        "ip6_addr": "2001:718::",
        "ip6_prefix": 29,
        "netnames": ["CZ-TEN-34-20010521"],
        "descr": ["Extensive network description"],
        "resolved_abuses": {
          "low": [
            "abuse@cesnet.cz"
          ]
        }
    },

    # Option 4: Pass only IPv(4|6) network|range without type for autodetection (slower)
    {
        "primary_key": "2001:718::/29",
        "network": "2001:718::/29",
        "netnames": ["CZ-TEN-34-20010521"],
        "resolved_abuses": {
          "low": [
            "abuse@cesnet.cz"
          ]
        }
    },
    ...
]

The netname, descr and description attributes are optional and will be used/stored into database, if present.

The resolved_abuses attribute is mandatory and must contain list of abuse groups (abuse contacts) for that particular network record.

class mentat.module.netmngr.MentatNetmngrScript

Bases: mentat.script.fetcher.FetcherScript

Implementation of Mentat module (script) providing functions for abuse group network management for Mentat database.

CONFIG_EXCEPTION_FILES = 'exception_files'

CONFIG_WHOIS_FILE = 'whois_file'

CONFIG_WHOIS_SOURCE = 'source'

cbk_command_convert_exceptions()

Implementation of the convert-exceptions command.

Attempt to convert given list of exception files into a valid whois file.

cbk_command_status()

Implementation of the status command (default).

Detect and display the status of abuse group collection with respect to network configurations.

cbk_command_update()

Implementation of the update command.

Attempt to update the state of internal whois database contents according to the data in given reference whois file.

get_default_command()

Return the name of the default script command. This command will be executed in case it is not explicitly selected either by command line option, or by configuration file directive.

Returns

Name of the default command.

Return type

str