Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

TPG创建数据时报错 #418

Open
liyifan20010515 opened this issue May 23, 2024 · 0 comments
Open

TPG创建数据时报错 #418

liyifan20010515 opened this issue May 23, 2024 · 0 comments

Comments

@liyifan20010515
Copy link

出现了多进程can‘t pickle问题
train_loader = DataLoader(train_dataset, sampler=LadderSampler(train_dataset, batch_size),
num_workers=num_workers, batch_size=batch_size,
collate_fn=lambda e: TPGDataset.collect_fn_quadkey(e, train_dataset,
sampler, self.QUADKEY,
self.loc2quadkey, k=num_neg, N_nearest=self.N_nearest, mix=mix, train=True, length=self.length))
报错信息如下:
EOFError: Ran out of input
return _MultiProcessingDataLoaderIter(self)
File "C:\some files\TPG-main\Bigscity-LibCity\venv\lib\site-packages\torch\utils\data\dataloader.py", line 914, in init
w.start()
File "C:\ruanjan\python 3.7\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "C:\ruanjan\python 3.7\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\ruanjan\python 3.7\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\ruanjan\python 3.7\lib\multiprocessing\popen_spawn_win32.py", line 89, in init
reduction.dump(process_obj, to_child)
File "C:\ruanjan\python 3.7\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'TPGDataset.get_data..'
请问应该怎么修改呢,是不是不能这样使用

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant