Testing

TESTING

Actual desing of GRIP testing system contains same components as production system with help of PostgreSQL database.

PostgreSQL database helps with grouping algorithm. Stable solution lists rows from big NERD database, which takes long time, and after that find some similarities. New testing solution creates own smaller tables in database in real time only with needed data (ex. only last 2 hours).

In realtime solution is creating table 'idea1' of last 2 hours of records and will insert only records (IDEA reports) with enough information. Table contains columns as IDEA report: ID of report, detecttime of report, array of categories, array of sourceIP, array of sourcePort, array of sourceProtocol, array of targetIP, array of targetPort, array of targetProtocol and attachment. Next table 'history' contains data of last day (time period will be probably changed according to usage). This table will be used for finding group with same bad behavior in time. Table has same columns as 'idea1' table.

Another table 'uniq_ip' will be created once in minute (time period will be probably changed according to usage). This table has index on column 'sourceIP' of uniq IP addresses which occure in 'Source' attribute in IDEA report. To make lighter table we decided turn big array of non-uniq IP addresses in targetIP into number of uniq IP addresses. Rest of data are stored in arrays (array of IDs, array of detecttimes, ...).

Last table 'features' helps to quickly find information. Table has columns that indicates if IP address is/was part of group with specific behavior, for example if IP address attack to port 23 or 2323, if IP address send spam, or if IP address was part of botnet - if IP address was tag as C&C machine or bot.

Grouping algorithm execute SQL query on tables with less rows so the result will come sooner.

We consider that graph database with results would be switch to relation database.