We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
For the Read class, it appears that the sequence attribute is inaccessible when passed to a parallel pool. As a MWE, consider:
Read
sequence
import multiprocessing import khmer def my_func(record): return len(record.sequence) parser = khmer.ReadParser('test_file.fq') # read in a few records read_list = [] i = 0 for record in parser.iter_reads(): read_list.append(record) if i >= 10: break i += 1 pool = multiprocessing.Pool(processes=2) res = map(my_func, read_list) # works res2 = pool.map(my_func, read_list) # doesn't work: AttributeError: 'Read' object has no attribute 'sequence'. print(res) print(res2) pool.close()
This results in:
Traceback (most recent call last): File "MinimumWorkingError.py", line 22, in <module> res2 = pool.map(my_func, read_list) # doesn't work: AttributeError: 'Read' object has no attribute 'sequence'. File "/usr/lib/python2.7/multiprocessing/pool.py", line 251, in map return self.map_async(func, iterable, chunksize).get() File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get raise self._value AttributeError: 'Read' object has no attribute 'sequence'.
Same thing happens in python2 or python3 (khmer.__version__ 2.1.1). Unsure, but #645 may be relevant.
khmer.__version__
Motivation is I'd like to stream in a sequence file and immediately pass it off to pool workers for downstream analysis.
The text was updated successfully, but these errors were encountered:
No branches or pull requests
For the
Read
class, it appears that thesequence
attribute is inaccessible when passed to a parallel pool. As a MWE, consider:This results in:
Same thing happens in python2 or python3 (
khmer.__version__
2.1.1). Unsure, but #645 may be relevant.Motivation is I'd like to stream in a sequence file and immediately pass it off to pool workers for downstream analysis.
The text was updated successfully, but these errors were encountered: