Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

MemoryManagerOutOfMemory Error using docker (wikibase/wdqs:0.3.40) #403

Open
leilafedd opened this issue Feb 9, 2023 · 0 comments
Open

Comments

@leilafedd
Copy link

leilafedd commented Feb 9, 2023

Hello,

I followed this tutorial to setup a local wikidata query service using docker (wikibase/wdqs:0.3.40)
The data is loaded and I can send SPARQL queries.
I tested with some queries that worked, but using the following query (the query is working for other classes with less instances compared to "human"(Q5)):

SELECT DISTINCT ?c1 ?r ?c2
        WHERE
        {
          VALUES ?c1 {wd:Q5}
          ?x ?r ?y .
          ?x wdt:P31 ?c1 .
          ?y wdt:P31 ?c2 .
          FILTER NOT EXISTS { ?x wdt:P31 ?y } .
          FILTER NOT EXISTS { ?x wdt:P279 ?y } .
        }

I get the following MemoryManagerOutOfMemory error:

java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.openrdf.query.QueryEvaluationException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=36a73c99-710a-46a9-ba5e-779e3b0d9e83,bopId=6,partitionId=-1,sinkId=8,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: cause=com.bigdata.btree.EvictionError: com.bigdata.rwstore.sector.MemoryManagerOutOfMemory, state=HTreeHashJoinUtility{open=true,joinType=Exists,askVar=-exists-1,joinVars=[X1, X2],outputDistinctJVs=true,size=4281228,considered(left=0,right=0,joins=0),joinSetSize=0}
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:206)
        at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:292)
        at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlQuery(QueryServlet.java:678)
        at com.bigdata.rdf.sail.webapp.QueryServlet.doPost(QueryServlet.java:275)
        at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:269)
        .....

Based on this response, I should increase:

-XX:MaxDirectMemorySize
And
Dcom.bigdata.rdf.sparql.ast.QueryHints.analyticMaxMemoryPerQuery

  • is there an option changing those parameters using an option (like -p for port) while running the service with?
sudo docker run --entrypoint=/runBlazegraph.sh -d -v ${HOME}/wddump/dockerData:/wdqs/data -v ${HOME}/wddump:/wddump -e HEAP_SIZE="48g" -p 9999:9999 wikibase/wdqs:0.3.40
  • Are there other options how to test that with docker ?
  • Are there other solutions to the MemoryManagerOutOfMemory error ?

I would be very grateful for your help

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant