[Windows] Extract off-targets: opened temporary file cannot be moved

[jakeb1996]

When running the extract off-targets script on Windows, an OS error may be thrown when trying to move the temporary file.

The temporary file is never closed and therefore, Windows does not want to move it.

The error occurs here:

Crackling/src/crackling/utils/extractOfftargets.py

Line 191 in bb50af9

shutil.move(mergedFile.name, fpOutput)

According to the Python docs, here, this behavior is expected:

tempfile.NamedTemporaryFile(mode='w+b', buffering=- 1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, *, errors=None)

This function operates exactly as TemporaryFile() does, except that the file is guaranteed to have a visible name in the file system (on Unix, the directory entry is not unlinked). That name can be retrieved from the name attribute of the returned file-like object. Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows). If delete is true (the default), the file is deleted as soon as it is closed. The returned object is always a file-like object whose file attribute is the underlying true file object. This file-like object can be used in a with statement, just like a normal file.

One way to fix this is by closing the temporary file inside the while loop:

Crackling/src/crackling/utils/extractOfftargets.py

Lines 161 to 191 in bb50af9

	while len(sortedFiles) > 1:
	# A file to write the merged sequences to
	mergedFile = tempfile.NamedTemporaryFile(delete = False)
	# Select the files to merge
	while True:
	try:
	sortedFilesPointers = [open(file, 'r') for file in sortedFiles[:maxNumOpenFiles]]
	break
	except OSError as e:
	if e.errno == 24:
	printer(f'Attempted to open too many files at once (OSError errno 24)')
	maxNumOpenFiles = max(1, int(maxNumOpenFiles / 2))
	printer(f'Reducing the number of files that can be opened by half to {maxNumOpenFiles}')
	continue
	raise e
	printer(f'Merging {len(sortedFilesPointers):,}')
	# Merge and write
	with open(mergedFile.name, 'w') as f:
	f.writelines(heapq.merge(*sortedFilesPointers))
	# Close all of the open files
	for file in sortedFilesPointers:
	file.close()
	# prepare for the next set to be merged
	sortedFiles = sortedFiles[maxNumOpenFiles:] + [mergedFile.name]
	shutil.move(mergedFile.name, fpOutput)

mergedFile.close()