[jadx-gui] Searching for resources while decompilation is in progress causes various thread synchronization problems

[jpstotz]

1. Open a large APK with "Autostart background decompilation" disabled (e.g. the binary linked in #1631)
2. Open search dialog
3. Disable Code search, enable Resource search
4. Enter some characters in search text field to start decompilation and continue slow typing every few seconds

java.util.ConcurrentModificationException: null
	at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
	at java.base/java.util.ArrayList$Itr.next(ArrayList.java:997)
	at jadx.gui.treemodel.JResource.update(JResource.java:87)
	at jadx.gui.treemodel.JResource.loadNode(JResource.java:97)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:124)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:131)
	at jadx.gui.search.providers.ResourceSearchProvider.load(ResourceSearchProvider.java:112)
	at jadx.gui.search.providers.ResourceSearchProvider.next(ResourceSearchProvider.java:54)
	at jadx.gui.search.SearchJob.run(SearchJob.java:23)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
ERROR - Error load resource node: pdf
java.lang.ArrayIndexOutOfBoundsException: 0 >= 0
	at java.base/java.util.Vector.removeElementAt(Vector.java:580)
	at java.desktop/javax.swing.tree.DefaultMutableTreeNode.remove(DefaultMutableTreeNode.java:207)
	at java.desktop/javax.swing.tree.DefaultMutableTreeNode.removeAllChildren(DefaultMutableTreeNode.java:393)
	at jadx.gui.treemodel.JResource.update(JResource.java:80)
	at jadx.gui.treemodel.JResource.loadNode(JResource.java:97)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:124)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:131)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:131)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:131)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:131)
	at jadx.gui.search.providers.ResourceSearchProvider.load(ResourceSearchProvider.java:112)
	at jadx.gui.search.providers.ResourceSearchProvider.next(ResourceSearchProvider.java:54)
	at jadx.gui.search.SearchJob.run(SearchJob.java:23)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
ERROR - Error load resource node: lib
java.lang.ArrayIndexOutOfBoundsException: 42 >= 42
	at java.base/java.util.Vector.removeElementAt(Vector.java:580)
	at java.desktop/javax.swing.tree.DefaultMutableTreeNode.remove(DefaultMutableTreeNode.java:207)
	at java.desktop/javax.swing.tree.DefaultMutableTreeNode.removeAllChildren(DefaultMutableTreeNode.java:393)
	at jadx.gui.treemodel.JResource.update(JResource.java:80)
	at jadx.gui.treemodel.JResource.update(JResource.java:88)
	at jadx.gui.treemodel.JResource.loadNode(JResource.java:97)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:124)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:131)
	at jadx.gui.search.providers.ResourceSearchProvider.load(ResourceSearchProvider.java:112)
	at jadx.gui.search.providers.ResourceSearchProvider.next(ResourceSearchProvider.java:54)
	at jadx.gui.search.SearchJob.run(SearchJob.java:23)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
WARN  - UI thread interrupted
java.lang.InterruptedException: null
	at java.base/java.lang.Object.wait(Native Method)
	at java.base/java.lang.Object.wait(Object.java:328)
	at java.desktop/java.awt.EventQueue.invokeAndWait(EventQueue.java:1361)
	at java.desktop/java.awt.EventQueue.invokeAndWait(EventQueue.java:1342)
	at java.desktop/javax.swing.SwingUtilities.invokeAndWait(SwingUtilities.java:1480)
	at jadx.gui.utils.UiUtils.uiRunAndWait(UiUtils.java:376)
	at jadx.gui.jobs.BackgroundExecutor$TaskWorker.doInBackground(BackgroundExecutor.java:147)
	at jadx.gui.jobs.BackgroundExecutor$TaskWorker.doInBackground(BackgroundExecutor.java:113)
	at java.desktop/javax.swing.SwingWorker$1.call(SwingWorker.java:304)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.desktop/javax.swing.SwingWorker.run(SwingWorker.java:343)
	at jadx.gui.jobs.BackgroundExecutor.lambda$execute$0(BackgroundExecutor.java:58)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
ERROR - Error load resource node: layout
java.lang.ArrayIndexOutOfBoundsException: 1357 >= 1356
	at java.base/java.util.Vector.elementAt(Vector.java:497)
	at java.desktop/javax.swing.tree.DefaultMutableTreeNode.getChildAt(DefaultMutableTreeNode.java:246)
	at java.desktop/javax.swing.tree.DefaultMutableTreeNode.remove(DefaultMutableTreeNode.java:206)
	at java.desktop/javax.swing.tree.DefaultMutableTreeNode.removeAllChildren(DefaultMutableTreeNode.java:393)
	at jadx.gui.treemodel.JResource.update(JResource.java:80)
	at jadx.gui.treemodel.JResource.loadNode(JResource.java:97)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:124)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:131)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:131)
	at jadx.gui.search.providers.ResourceSearchProvider.load(ResourceSearchProvider.java:112)
	at jadx.gui.search.providers.ResourceSearchProvider.next(ResourceSearchProvider.java:54)
	at jadx.gui.search.SearchJob.run(SearchJob.java:23)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
ERROR - Error load resource node: resources.arsc
java.lang.ArrayIndexOutOfBoundsException: 15 >= 15
	at java.base/java.util.Vector.elementAt(Vector.java:497)
	at java.desktop/javax.swing.tree.DefaultMutableTreeNode.getChildAt(DefaultMutableTreeNode.java:246)
	at java.desktop/javax.swing.tree.DefaultMutableTreeNode.remove(DefaultMutableTreeNode.java:206)
	at java.desktop/javax.swing.tree.DefaultMutableTreeNode.removeAllChildren(DefaultMutableTreeNode.java:393)
	at jadx.gui.treemodel.JResource.update(JResource.java:80)
	at jadx.gui.treemodel.JResource.loadNode(JResource.java:97)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:124)
	at jadx.gui.search.providers.ResourceSearchProvider.traverseTree(ResourceSearchProvider.java:131)
	at jadx.gui.search.providers.ResourceSearchProvider.load(ResourceSearchProvider.java:112)
	at jadx.gui.search.providers.ResourceSearchProvider.next(ResourceSearchProvider.java:54)
	at jadx.gui.search.SearchJob.run(SearchJob.java:23)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

[skylot]

@jpstotz thanks!
Looks like I need to do a heavy refactoring for this resource search, main issues:

• load method is synchronized only for current instance, but every new search create a new instance
• load method is not cancelable, so it doesn't stop on search cancel
• also it is not really needed to preload anything, it is better to adapt iterations over resources tree and load nodes on demand

[skylot]

I commit changes I mention above. Can anyone check it?

[jpstotz]

@skylot Thanks for the fix.

The search dialog seem to be now safe in 99.99% of all cases.

One time I got the following stack trace, but no matter how hard I tried I wasn't able to reproduce it:

WARN  - Search error, provider: ResourceSearchProvider
java.util.NoSuchElementException: Vector Enumeration
	at java.base/java.util.Vector$1.nextElement(Vector.java:375)
	at jadx.gui.search.providers.ResourceSearchProvider.addChildren(ResourceSearchProvider.java:121)
	at jadx.gui.search.providers.ResourceSearchProvider.getNextNode(ResourceSearchProvider.java:114)
	at jadx.gui.search.providers.ResourceSearchProvider.getNextNode(ResourceSearchProvider.java:110) [repeated 130 times]
	at jadx.gui.search.providers.ResourceSearchProvider.getNextNode(ResourceSearchProvider.java:115)
	at jadx.gui.search.providers.ResourceSearchProvider.getNextNode(ResourceSearchProvider.java:110) [repeated 117 times]
	at jadx.gui.search.providers.ResourceSearchProvider.next(ResourceSearchProvider.java:55)
	at jadx.gui.search.SearchJob.run(SearchJob.java:23)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

After that I only noticed one minor issue in combination with UiUtils.uiRunAndWait which prints the stack trace when interrupted. I managed to trigger those one time while playing with the search dialog in a pretty "wild way":

WARN  - UI thread interrupted
java.lang.InterruptedException: null
	at java.base/java.lang.Object.wait(Native Method)
	at java.base/java.lang.Object.wait(Object.java:328)
	at java.desktop/java.awt.EventQueue.invokeAndWait(EventQueue.java:1361)
	at java.desktop/java.awt.EventQueue.invokeAndWait(EventQueue.java:1342)
	at java.desktop/javax.swing.SwingUtilities.invokeAndWait(SwingUtilities.java:1480)
	at jadx.gui.utils.UiUtils.uiRunAndWait(UiUtils.java:376)
	at jadx.gui.jobs.BackgroundExecutor$TaskWorker.doInBackground(BackgroundExecutor.java:147)
	at jadx.gui.jobs.BackgroundExecutor$TaskWorker.doInBackground(BackgroundExecutor.java:113)
	at java.desktop/javax.swing.SwingWorker$1.call(SwingWorker.java:304)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.desktop/javax.swing.SwingWorker.run(SwingWorker.java:343)
	at jadx.gui.jobs.BackgroundExecutor.lambda$execute$0(BackgroundExecutor.java:58)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

But as the task is already completed it not really a problem, just printing the strack trace looks like there is a problem.

[skylot]

@jpstotz thanks for check and feedback!
I commit a change to reduce possible concurrent modifications by loading resource table resources in "one change", previously tree was loaded incrementally, so it was hard to synchronize these updates.

WARN - UI thread interrupted

This is just a warning, indicating that search provider checks cancel state not frequently enough, so it can't stop in time (timeout is very strict). This is a good indicator of bad implementation, so I want to keep it. In last change I made an additional check, so it shouldn't show often 🙂.

[jpstotz]

@skylot If you prefer to keep the log message with stack trace as it is, I am also fine with this.

Thanks again for the immediate fix - looks like this issue comes it an end...