queueing: Fix cluster duplicate handler dying

With python3 the cluster duplicate handler would die from RuntimeErrors
due to the items() accessor of the duplicate backlog dict being a
view/iterator that doesn't respond well to the dict changing while being
iterated. Prevent the RuntimeError by iterating over the items of a copy
of the dict while changing the original, similar to what we're doing in
the cuckoo job tracke for alomst the same reason already.

Fixes scVENUS#160.

peekaboo/queuing.py

@@ -165,7 +165,11 @@ def submit_cluster_duplicates(self):
         with self.duplock:
             # try to submit *all* samples which have been marked as being
             # processed by another instance concurrently
-            for sample_hash, sample_duplicates in self.cluster_duplicates.items():
+            # get the items view on a copy of the cluster duplicate backlog
+            # because we will change it by removing entries which would raise a
+            # RuntimeException
+            cluster_duplicates = self.cluster_duplicates.copy().items()
+            for sample_hash, sample_duplicates in cluster_duplicates:
                 # try to mark as in-flight
                 try:
                     locked = self.db_con.mark_sample_in_flight(