Bug in configextractor handling venv

[GuillaumeCSekoia]

This issue follows a conversation on the Discord.

It seems there are some issues with configextractor-py handling venvs : either the parsing is not done properly or the imports after the parsing are not done properly.

The following will be a summup of the Discord conversation.

Context

• Using configextractor-py with AssemblyLine.
• Put my own GitHub repo in the "updater source".
• My repo tree looks like this :

ConfigExtractorSekoia
     requirements.txt
     README.md
     modules/
ConfigExtractorSekoia/modules/:
    DummyExtractor.py
    __init__.py

• My pattern for the updater is .*/modules/$

The bug(s)

In AssemblyLine
So when doing this, it does not find the parser because it can't find the appropriate requirements.txt for the parser.

Tried several things :

• __init__.py file in the root dir and pattern to .*/ConfigExtractorSekoia/$ (also tried several other patterns like ConfigExtractorSekoia/* etc. didn't work). Didn't fix the issue.
• requirements.txt not in the root folder but in the parsers' folder instead which makes more sense, and pattern to .*/modules/$. This time it found my parser. However I had the following error : "/updates/tmpsetcir3m/ConfigExtractorSekoia/venv/bin/python: Relative module names not supported\n" .

Locally

I tried locally as well with the following steps:

requirements.txt in the ConfigExtractorSekoia/modules/ folder.
./create_venv.sh /home/user/ConfigExtractorSekoia/modules
source /home/user/ConfigExtractorSekoia/modules/venv/bin/activate
pip install configextractor-py

Then ran the following code :

from configextractor.main import ConfigExtractor
import logging

log = logging.getLogger()
log.setLevel(logging.WARNING)
log.handlers = [logging.StreamHandler()]
cx = ConfigExtractor(['/home/user/ConfigExtractorSekoia/modules/'], logger=log)
results = cx.run_parsers('/home/user/Downloads/sample.bin')
print(results)

And I had the following error :

   from .DummyExtractor import DummyExtractor 
ImportError: attempted relative import with no known parent package

Did some debugger to find out where exactly and I had the exception exactly here (that then propagated)

configextractor-py/configextractor/frameworks/base.py

Line 94 in 2d44805

except Exception:

How to make it work locally

Locally I managed to make it work by doing these steps instead :

• put requirements.txt in the root folder
• ./create_venv.sh /home/user/ConfigExtractorSekoia
• source /home/user/ConfigExtractorSekoia/venv/bin/activate
• pip install configextractor-py
• Same snippet of code.

[cccs-rs]

This should be resolved in next release (v1.1.2) coming soon! Thanks for bug!