various performances improvements (30-50% in certain workflows, 15-25% in average)

[mat-gas]

various performances improvements, especially when parsing import table to compute imphash

as python 2 support has been dropped, we can safely used python3 only features (in count_zeroes for instance)

gain is when parsing only import table and compyuting imphash
gain full is when performing a "full" load of PE file (without fast_load)

binary	old	new	gain	old full	new full	gain full
AgentService	0.011420	0.006963	39.0%	0.274231	0.233094	15.0%
Taskmgr.exe	0.051962	0.027470	47.1%	0.353381	0.241182	31.8%
VSSVC.exe	0.016385	0.010664	34.9%	0.353499	0.289626	18.1%
certutil.exe	0.033767	0.019646	41.8%	0.397970	0.313657	21.2%
cmd.exe	0.014613	0.009378	35.8%	0.088915	0.071951	19.1%
conhost.exe	0.013193	0.008659	34.4%	0.265667	0.224680	15.4%
consent.exe	0.013805	0.008923	35.4%	0.044017	0.034052	22.6%
mavinject.exe	0.008297	0.005268	36.5%	0.060388	0.053025	12.2%
mshta.exe	0.002762	0.001819	34.1%	0.005971	0.004286	28.2%
msiexec.exe	0.007087	0.004125	41.8%	0.019160	0.013540	29.3%
notepad.exe	0.014233	0.009075	36.2%	0.042831	0.034543	19.3%
rdpclip.exe	0.015521	0.009676	37.7%	0.097005	0.080027	17.5%
regsvr32.exe	0.005074	0.003636	28.3%	0.010174	0.007732	24.0%
rundll32.exe	0.007997	0.005495	31.3%	0.023463	0.016805	28.4%
schtasks.exe	0.009094	0.006030	33.7%	0.045278	0.036827	18.7%
services.exe	0.025681	0.013355	48.0%	0.198105	0.163667	17.4%
svchost.exe	0.008043	0.004687	41.7%	0.019350	0.015881	17.9%
winlogon.exe	0.021102	0.013716	35.0%	0.201703	0.167614	16.9%

raw results :

using AgentService.exe (length : 1180672 / sha256 : b3d7aab8ad319dcf7050fe8ce67d69c8c59adc0d90c19a144c2d5c1f66c1babf
  OLD took 1.142029 (0.011420 / iter)
  NEW took 0.696284 (0.006963 / iter)  --> 39.0% gain
  OLD full took 2.742311 (0.274231 / iter)
  NEW full took 2.330939 (0.233094 / iter)  --> 15.0% gain
using Taskmgr.exe (length : 1391096 / sha256 : 66d661ad70d39a6f0f6e50d76c2bd8676280552fca0e44e973b8422029bc615f
  OLD took 5.196224 (0.051962 / iter)
  NEW took 2.746977 (0.027470 / iter)  --> 47.1% gain
  OLD full took 3.533806 (0.353381 / iter)
  NEW full took 2.411817 (0.241182 / iter)  --> 31.8% gain
using VSSVC.exe (length : 1516544 / sha256 : bd85b673250c649a9cae751885fec71025ee71d50d23aabbfecda4af92557352
  OLD took 1.638460 (0.016385 / iter)
  NEW took 1.066383 (0.010664 / iter)  --> 34.9% gain
  OLD full took 3.534992 (0.353499 / iter)
  NEW full took 2.896263 (0.289626 / iter)  --> 18.1% gain
using certutil.exe (length : 1652736 / sha256 : 85dd6f8edf142f53746a51d11dcba853104bb0207cdf2d6c3529917c3c0fc8df
  OLD took 3.376673 (0.033767 / iter)
  NEW took 1.964556 (0.019646 / iter)  --> 41.8% gain
  OLD full took 3.979699 (0.397970 / iter)
  NEW full took 3.136575 (0.313657 / iter)  --> 21.2% gain
using cmd.exe (length : 278528 / sha256 : 3656f37a1c6951ec4496fabb8ee957d3a6e3c276d5a3785476b482c9c0d32ea2
  OLD took 1.461260 (0.014613 / iter)
  NEW took 0.937846 (0.009378 / iter)  --> 35.8% gain
  OLD full took 0.889153 (0.088915 / iter)
  NEW full took 0.719510 (0.071951 / iter)  --> 19.1% gain
using conhost.exe (length : 822272 / sha256 : ada97ec1f2bf175cb3a4d53e099347414444f6740c484d944229b6b8a383f606
  OLD took 1.319302 (0.013193 / iter)
  NEW took 0.865927 (0.008659 / iter)  --> 34.4% gain
  OLD full took 2.656672 (0.265667 / iter)
  NEW full took 2.246798 (0.224680 / iter)  --> 15.4% gain
using consent.exe (length : 159272 / sha256 : 8f112431143a22baaafb448eefd63bf90e7691c890ac69a296574fd07ba03ec6
  OLD took 1.380480 (0.013805 / iter)
  NEW took 0.892338 (0.008923 / iter)  --> 35.4% gain
  OLD full took 0.440173 (0.044017 / iter)
  NEW full took 0.340520 (0.034052 / iter)  --> 22.6% gain
using mavinject.exe (length : 185352 / sha256 : abb99f7cfd3e9eb294501aafa082a8d4841278cc39a4fb3dff9942ca1f71a139
  OLD took 0.829691 (0.008297 / iter)
  NEW took 0.526827 (0.005268 / iter)  --> 36.5% gain
  OLD full took 0.603883 (0.060388 / iter)
  NEW full took 0.530248 (0.053025 / iter)  --> 12.2% gain
using mshta.exe (length : 14848 / sha256 : e616c5ce71886652c13e2e1fa45a653b44d492b054f16b15a38418b8507f57c7
  OLD took 0.276202 (0.002762 / iter)
  NEW took 0.181934 (0.001819 / iter)  --> 34.1% gain
  OLD full took 0.059705 (0.005971 / iter)
  NEW full took 0.042857 (0.004286 / iter)  --> 28.2% gain
using msiexec.exe (length : 67072 / sha256 : dc52a89f2fcbd8e31f2d466bd2d35414a86bb907382251fabb88cd3969fb3ec8
  OLD took 0.708740 (0.007087 / iter)
  NEW took 0.412503 (0.004125 / iter)  --> 41.8% gain
  OLD full took 0.191598 (0.019160 / iter)
  NEW full took 0.135402 (0.013540 / iter)  --> 29.3% gain
using notepad.exe (length : 254464 / sha256 : a92056d772260b39a876d01552496b2f8b4610a0b1e084952fe1176784e2ce77
  OLD took 1.423278 (0.014233 / iter)
  NEW took 0.907549 (0.009075 / iter)  --> 36.2% gain
  OLD full took 0.428305 (0.042831 / iter)
  NEW full took 0.345434 (0.034543 / iter)  --> 19.3% gain
using rdpclip.exe (length : 430080 / sha256 : 3758d163de052ab532f41db4ea923beece0d5d842fa0db06a55d8ab38794691b
  OLD took 1.552141 (0.015521 / iter)
  NEW took 0.967572 (0.009676 / iter)  --> 37.7% gain
  OLD full took 0.970050 (0.097005 / iter)
  NEW full took 0.800273 (0.080027 / iter)  --> 17.5% gain
using regsvr32.exe (length : 24064 / sha256 : f098fa150d9199732b4ec2e81528a951503a30f75afebf7e7a48360301758c67
  OLD took 0.507421 (0.005074 / iter)
  NEW took 0.363584 (0.003636 / iter)  --> 28.3% gain
  OLD full took 0.101741 (0.010174 / iter)
  NEW full took 0.077322 (0.007732 / iter)  --> 24.0% gain
using rundll32.exe (length : 71168 / sha256 : 01b407af0200b66a34d9b1fa6d9eaab758efa36a36bb99b554384f59f8690b1a
  OLD took 0.799687 (0.007997 / iter)
  NEW took 0.549498 (0.005495 / iter)  --> 31.3% gain
  OLD full took 0.234626 (0.023463 / iter)
  NEW full took 0.168052 (0.016805 / iter)  --> 28.4% gain
using schtasks.exe (length : 222720 / sha256 : d6ba2cd73799477c051d9d864c47fcf5108064cde07d3565871afa10fc548086
  OLD took 0.909413 (0.009094 / iter)
  NEW took 0.603033 (0.006030 / iter)  --> 33.7% gain
  OLD full took 0.452776 (0.045278 / iter)
  NEW full took 0.368272 (0.036827 / iter)  --> 18.7% gain
using services.exe (length : 678680 / sha256 : 526f2447ad8da40cf3b969e98acd6621fe9f5ed94ee6a652661ffaa0e8628446
  OLD took 2.568089 (0.025681 / iter)
  NEW took 1.335513 (0.013355 / iter)  --> 48.0% gain
  OLD full took 1.981047 (0.198105 / iter)
  NEW full took 1.636670 (0.163667 / iter)  --> 17.4% gain
using svchost.exe (length : 51696 / sha256 : 7fd065bac18c5278777ae44908101cdfed72d26fa741367f0ad4d02020787ab6
  OLD took 0.804252 (0.008043 / iter)
  NEW took 0.468661 (0.004687 / iter)  --> 41.7% gain
  OLD full took 0.193497 (0.019350 / iter)
  NEW full took 0.158806 (0.015881 / iter)  --> 17.9% gain
using winlogon.exe (length : 779776 / sha256 : 01075068dc177ae38293a76c9da5b2e09426093eca4f6a53281cb6ec2fc6cd17
  OLD took 2.110183 (0.021102 / iter)
  NEW took 1.371571 (0.013716 / iter)  --> 35.0% gain
  OLD full took 2.017029 (0.201703 / iter)
  NEW full took 1.676142 (0.167614 / iter)  --> 16.9% gain

the benchmark script used:

import sys 
sys.path.insert(0,'.')
import os
import hashlib
import pefile
import pefilenew as pefilenew
import time

count = 100
count_full = 10
directories = [pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_IMPORT']]
binaries_directory = './binaries/'

for f in sorted(os.listdir(binaries_directory)):
        data = open(os.path.join(binaries_directory, f), 'rb').read()
        sha2 = hashlib.sha256(data).hexdigest()
        print("using {} (length : {} / sha256 : {}".format(f, len(data), sha2))
        
        begin = time.time()
        for _ in range(count):
                a = pefile.PE(data=data, fast_load=True)
                a.parse_data_directories(directories=directories)
                imphash = a.get_imphash()
        end = time.time()

        old_average = (end-begin) / count
        print("  OLD took {:04f} ({:04f} / iter)".format(end-begin, old_average))
        
        begin = time.time()
        for _ in range(count):
                a = pefilenew.PE(data=data, fast_load=True)
                a.parse_data_directories(directories=directories)
                imphash = a.get_imphash()
        end = time.time()

        new_average = (end-begin) / count
        print("  NEW took {:04f} ({:04f} / iter)  --> {:01f}% gain".format(end-begin, new_average, 100.0 - ((new_average * 100.0) / old_average) ))      
        
        
        begin = time.time()
        for _ in range(count_full):
                a = pefile.PE(data=data)
                imphash = a.get_imphash()
        end = time.time()

        old_average = (end-begin) / count_full
        print("  OLD full took {:04f} ({:04f} / iter)".format(end-begin, old_average))
        
        begin = time.time()
        for _ in range(count_full):
                a = pefilenew.PE(data=data)
                imphash = a.get_imphash()
        end = time.time()

        new_average = (end-begin) / count_full
        print("  NEW full took {:04f} ({:04f} / iter)  --> {:01f}% gain".format(end-begin, new_average, 100.0 - ((new_average * 100.0) / old_average) ))        

[erocarrera]

Thanks! I had to fix a small issue in AddressSet but it's a great PR!