Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Missing "Functions Statically Reachable" in Introspector Report #1010

Open
sg3-141-592 opened this issue May 1, 2023 · 2 comments
Open

Missing "Functions Statically Reachable" in Introspector Report #1010

sg3-141-592 opened this issue May 1, 2023 · 2 comments
Assignees

Comments

@sg3-141-592
Copy link
Contributor

sg3-141-592 commented May 1, 2023

Looking at a recent croniter coverage report https://storage.googleapis.com/oss-fuzz-introspector/croniter/inspector-report/20230501/fuzz_report.html the "Functions Statically Reachable" is very low 2/48.

However function coverage is pretty good. It looks like there are lots of functions that are counted as having coverage but not being "Reached by Fuzzers".

image

Picking croniter.croniter.croniter.get_next as a example, which is lines 191-193. We can see clear coverage in fuzz_iter - all_cov.json.

@DavidKorczynski DavidKorczynski self-assigned this May 1, 2023
@DavidKorczynski
Copy link
Contributor

There are a couple of issues at place, but I have some progress which I'll add for now, although I will try and come up with something better in the near future.

The first issue is due to some limitations in fuzz introspector's way of recognizing which modules are used. This can be overcome by helping fuzz introspector in identifying which package is being analysed. If you add this line to the build.sh in OSS-Fuzz:

export PYFUZZPACKAGE=$SRC/croniter/src/croniter

Then you will see something more meaningful, and my calltrees grow to size ~70/~80 for two of the fuzzers. Can you confirm?

The other issue at play is probably a deeper limitation in the callgraph analysis -- I'll need some more time to go over this though.

@sg3-141-592
Copy link
Contributor Author

Thanks, setting that variable gives an initial improvement 2 -> 15 functions covered. Then I updated how croniter was imported into the fuzzers which gave an additional improvement 15 -> 22 functions covered google/oss-fuzz#10207 . There's still a few patterns of issues, especially the nested croniter.croniter. When I get a chance I'll have a look how croniter is doing is defining these, I'm guessing this is a problem of a mismatch between how PyCG sees the callgraph compared to names in the coverage report.

image

DavidKorczynski pushed a commit to google/oss-fuzz that referenced this issue May 2, 2023
Work around for issue resolving croniter function names, see
ossf/fuzz-introspector#1010 .

Setting the `PYFUZZPACKAGE` environment variable takes coverage from
`2/48` functions reachable to an improved `22/48` reachable. I've also
updated how croniter is imported to have a higher number of matches.

There are still some issues in the callgraph analysis but this is a big
improvement in the metrics.
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants