Skip to content

Add guppy backend #256

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Conversation

dxe4
Copy link

@dxe4 dxe4 commented Nov 2, 2019

related to my comment here #236 (comment)
with implementation of guppy3 we can get the right values.

note this implementation does self.prevlines[-1] == self.prev_lineno: for guppy backend only, because repeating [i for i in range(0, 10000000)] makes the guppy backend extremely slow. although this may not be ideal or acceptable i thought it's worth sharing the code.


@profile
def myfn_py():
    ones = [1] * NBR_ITEMS
    twos = [i + 1 for i in ones]
    return twos


@profile
def myfn_np():
    ones_np = np.ones(NBR_ITEMS)
    twos_np = ones_np+1
    return twos_np


myfn_np()
myfn_py()


python -m memory_profiler --backend guppy prof.py 


Line #    Mem usage    Increment   Line Contents
================================================
    11   14.191 MiB   14.191 MiB   @profile
    12                             def myfn_py():
    13   18.006 MiB    3.815 MiB       ones = [1] * NBR_ITEMS
    14   22.098 MiB    4.091 MiB       twos = [i + 1 for i in ones]
    15                                 return twos


Filename: prof.py

Line #    Mem usage    Increment   Line Contents
================================================
    18   14.189 MiB   14.189 MiB   @profile
    19                             def myfn_np():
    20   18.005 MiB    3.817 MiB       ones_np = np.ones(NBR_ITEMS)
    21   21.820 MiB    3.815 MiB       twos_np = ones_np+1
    22   21.820 MiB    0.000 MiB       return twos_np

Charalampos.Papaloizou added 2 commits November 2, 2019 17:46
if self.backend == 'guppy':
if self.prevlines[-1] == self.prev_lineno:
# the reason for this is because for code like [i for i in range(0, 1000000)] guppy will be extremely slow
_run = False
Copy link
Author

@dxe4 dxe4 Nov 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can cause issues if you want to measure the memory of a side effect eg

[foo() for i in range(0, 500)]
def foo():
     make_big_array = [i for i in random.randint(0, int(1e9))]

i haven't tested it.
although it will give correct results for this case #236 (comment)

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant