pynspect.gparser module¶

This module contains object encapsulation of PLY parser for universal filtering and query language grammar. It is designed for working with almost arbitrary data structures and can be used in wide range of projects.

Grammar features¶

Logical operations: and or xor not exists

All logical operations support upper case and lower case name variants. Additionally, there are also symbolic variants || ^^ && ! ? with higher priority and which can be used in some cases instead of parentheses and thus improve parsing performance.
Comparison operations: like in is eq ne gt ge lt le

All comparison operations support upper case and lower case name variants. Additionally, there are also symbolic variants =~ ~~ == != <> >= > <= <.
Mathematical operations: + - * / %
JPath variables: Source[0].IP4[1]
Directly recognized constants:
- IPv4: 127.0.0.1 127.0.0.1/32 127.0.0.1-127.0.0.5 127.0.0.1..127.0.0.5
- IPv6: ::1 ::1/64 ::1-::5 ::1..::5
- Datetime: 2017-01-01T12:00:00Z 2017-01-01t12:00:00.123-02:00
- Timedelta: 12:01:15 15D00:00:00 21d11:11:00
- Integer: 0 1 42
- Float: 3.14159
Quoted literal constants: "double quoted" or 'single quoted'
Functions: time() size(Source.IP4)

Grammar supports calling arbitrary functions with optional arguments. Argument may be any valid expression, multiple arguments must be passed down as list. Function support in grammar is only one part of the whole picture, it must also be implemented in tree traversers to fully work. Each traverser may provide certain set of available functions and define required and optional arguments.

For more details on supported grammar token syntax please see the documentation of pynspect.lexer module.

Example expressions¶

utcnow() > (CreateTime + 3600)
CreateTime > 2017-01-01T12:00:00Z and Source.IP4 in [127.0.0.1, 127.0.0.2]
Category in ['Attempt.Login'] and (Target.Proto in ['telnet'] or Source.Proto in ['telnet'] or Target.Port in [23])

Currently implemented grammar¶

expression : xor_expression OP_OR expression
           | xor_expression

xor_expression : and_expression OP_XOR xor_expression
               | and_expression

and_expression : or_p_expression OP_AND and_expression
               | or_p_expression

or_p_expression : xor_p_expression OP_OR_P or_p_expression
                | xor_p_expression

xor_p_expression : and_p_expression OP_XOR_P xor_p_expression
                 | and_p_expression

and_p_expression : not_expression OP_AND_P and_p_expression
                 | not_expression

not_expression : OP_NOT ex_expression
               | ex_expression

ex_expression : OP_EXISTS cmp_expression
              | cmp_expression

cmp_expression : term OP_LIKE cmp_expression
               | term OP_IN cmp_expression
               | term OP_IS cmp_expression
               | term OP_EQ cmp_expression
               | term OP_NE cmp_expression
               | term OP_GT cmp_expression
               | term OP_GE cmp_expression
               | term OP_LT cmp_expression
               | term OP_LE cmp_expression
               | term

term : factor OP_PLUS term
     | factor OP_MINUS term
     | factor OP_TIMES term
     | factor OP_DIVIDE term
     | factor OP_MODULO term
     | factor

factor : IPV4
       | IPV6
       | DATETIME
       | TIMEDELTA
       | INTEGER
       | FLOAT
       | VARIABLE
       | CONSTANT
       | FUNCTION RPAREN
       | FUNCTION expression RPAREN
       | LBRACK list RBRACK
       | LPAREN expression RPAREN

list : IPV4
     | IPV6
     | DATETIME
     | TIMEDELTA
     | INTEGER
     | FLOAT
     | VARIABLE
     | CONSTANT
     | IPV4 COMMA list
     | IPV6 COMMA list
     | DATETIME COMMA list
     | TIMEDELTA COMMA list
     | INTEGER COMMA list
     | FLOAT COMMA list
     | VARIABLE COMMA list
     | CONSTANT COMMA list

Note

Implementation of this module is very PLY specific, please read the appropriate documentation to understand it. For the same reason the pylint tool comlains a lot about code style in this module, but that is a feature.