setting_up_environment_basic fails on python 3.6, Windows 7

[erlenlok]

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-914694f81f03> in <module>()
      3 
      4 
----> 5 MyEnvironment = BTgymEnv(filename='./data/DAT_ASCII_EURUSD_M1_2016.csv',)
      6 
      7 # Print environment configuration:

c:\users\e\github\btgym-master\btgym\envs\backtrader.py in __init__(self, *args, **kwargs)
    246         # Connect/Start data server (and get dataset statistic):
    247         self.log.info('Connecting data_server...')
--> 248         self._start_data_server()
    249         self.log.info('...done.')
    250         # ENGINE preparation:

c:\users\e\github\btgym-master\btgym\envs\backtrader.py in _start_data_server(self)
    736             )
    737             self.data_server.daemon = False
--> 738             self.data_server.start()
    739             # Wait for server to startup
    740             time.sleep(1)

C:\ProgramData\Anaconda3\lib\multiprocessing\process.py in start(self)
    103                'daemonic processes are not allowed to have children'
    104         _cleanup()
--> 105         self._popen = self._Popen(self)
    106         self._sentinel = self._popen.sentinel
    107         _children.add(self)

C:\ProgramData\Anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
    221     @staticmethod
    222     def _Popen(process_obj):
--> 223         return _default_context.get_context().Process._Popen(process_obj)
    224 
    225 class DefaultContext(BaseContext):

C:\ProgramData\Anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
    320         def _Popen(process_obj):
    321             from .popen_spawn_win32 import Popen
--> 322             return Popen(process_obj)
    323 
    324     class SpawnContext(BaseContext):

C:\ProgramData\Anaconda3\lib\multiprocessing\popen_spawn_win32.py in __init__(self, process_obj)
     63             try:
     64                 reduction.dump(prep_data, to_child)
---> 65                 reduction.dump(process_obj, to_child)
     66             finally:
     67                 set_spawning_popen(None)

C:\ProgramData\Anaconda3\lib\multiprocessing\reduction.py in dump(obj, file, protocol)
     58 def dump(obj, file, protocol=None):
     59     '''Replacement for pickle.dump() using ForkingPickler.'''
---> 60     ForkingPickler(file, protocol).dump(obj)
     61 
     62 #

TypeError: can't pickle _thread.RLock objects

[Kismuz]

Hmm... At very first glance it seems to be issue with the difference in Windows multiprocessing implementation. Since btgym has not been tested with Win, I need time to investigate it till I can fix it.

[mysl]

I spent some time to look into this since I am using windows. It turned out this has nothing to do with multiprocessing and it's the lock used in logging.Logger which cannot be pickled on windows ( I think this might be related to some platform dependent implementation of lock, but I didn't dig deeper). I found that using logger from logbook instead of python default logging module can workaround this issue and run the setting_up_environment_basic.ipynb to the end. In the last step of that notebook, it will run a few steps (1500 - 2000), output some plots, and then it will raise the following error: Resource temporarily unavailable. And this happens both on my windows and linux (ubuntu 14.04 LTS) systems. My python version is 3.6.2

Again Traceback (most recent call last)
in ()
23 # and when episode is finished:
24 if info[-1]['step'] % state_render == 0 or done:
---> 25 show_rendered_image(MyEnvironment.render('human'))
26
27 if done: break

c:\programdata\anaconda3\envs\btgym-env\lib\site-packages\gym\core.py in render(self, mode, close)
148 elif mode not in modes:
149 raise error.UnsupportedMode('Unsupported rendering mode: {}. (Supported modes for {}: {})'.format(mode, self, modes))
--> 150 return self._render(mode=mode, close=close)
151
152 def close(self):

c:\study\btgym\btgym\envs\backtrader.py in _render(self, mode, close)
724 raise ValueError('Unexpected render mode {}'.format(mode))
725 self.socket.send_pyobj({'ctrl': '_render', 'mode': mode})
--> 726 rgb_array_dict = self.socket.recv_pyobj()
727
728 self.rendered_rgb.update(rgb_array_dict)

c:\programdata\anaconda3\envs\btgym-env\lib\site-packages\zmq\sugar\socket.py in recv_pyobj(self, flags)
489 The Python object that arrives as a message.
490 """
--> 491 msg = self.recv(flags)
492 return self._deserialize(msg, pickle.loads)
493

zmq\backend\cython\socket.pyx in zmq.backend.cython.socket.Socket.recv()

zmq\backend\cython\socket.pyx in zmq.backend.cython.socket.Socket.recv()

zmq\backend\cython\socket.pyx in zmq.backend.cython.socket._recv_copy()

zmq\backend\cython\socket.pyx in zmq.backend.cython.socket._recv_copy()

c:\programdata\anaconda3\envs\btgym-env\lib\site-packages\zmq\backend\cython\checkrc.pxd in zmq.backend.cython.checkrc._check_rc()

Again: Resource temporarily unavailable

[Kismuz]

@mysl ,

I think this might be related to some platform dependent implementation of lock

• yes, it is indeed. Actually I haven't worked on issue either. Can you share a piece of code of this logger workaround?

error: Resource temporarily unavailable.
This error usually means orphaned child process is blocking zmq socket someway.

1. Have you made some modifications to the notebook? - Actually I can't replicate this error in notebook mentioned ( I'm running MacOS).

2. Have you manually interrupted jup. kernel? If yes, which command/tab used?

3. When you updated btgym last time?

[mysl]

@Kismuz

Can you share a piece of code of this logger workaround?

Sure. It's just simply using logger from logbook instead of logging for btgym.envs.backtrader.py, as the follow picture shows.

This error usually means orphaned child process is blocking zmq socket someway.

Yeah, looks like it. BTW, I forgot to mention, that when I running the notebook, I skipped the part of 'registering environment', because if I run it will also raise error 'Again, resource temporarily unavailable'. Does that imply that environment not clean up completely by calling close (at least in windows?)?

Have you made some modifications to the notebook?

No.

Have you manually interrupted jup. kernel? If yes, which command/tab used?

I probably did. I used ctrl + C. Actually I tried killing all the python processes today, and rerun this without manually interrupting jup.kernel. It shows the same behavior: run and plot a few steps, and Again, resource temporarily unavailable

[Kismuz]

That's rather strange. Unfortunately I don't have Win installed to replicate it.
BTGYM launches at least two separate processes, not counting jupyter kernel itself:

• btgym_server as backend for environment API, default port 5000

• data_server as data providing backend for one or more btgym_server(s), default port 4999

• calling env.close() should stop both and it usually does (at least on MACOS and Linux);

• interrupting parent kernel should stop childs as well, as they are not demonized, but:

• there is some caveat in interrupting jupyter kernel: It can not be done via Ctrl-C, equivalent is web interface [KERNEL]-->[INTERRUPT]. This combination correctly finishes all stuff, while hitting [KERNEL]-->[RESTART] or [RESTART AND CLEAR...] for some reasons leaves child processes orphaned.
  In this case list processes on specified ports:

lsof -i:5000
lsof -i:4999

...and do manual kill.

Note, that when running A3C examples there are also 12230 and 12231 to watch for.

Usually it throws errors like:

• resource temporarily unavailable
• could not start grpc server
• server unreachable with status: ....
• operation could not be accomplished in a current state

One more things: it has been a os-specific lurking rendering error, fixed today, related to episode rendering, worth checking here, bottom: #24

[mysl]

@Kismuz thanks for reply. I will update the code and have a try when I have a chance.

BTW, there is another 'incompatibility' with windows, this file https://github.com/Kismuz/btgym/blob/master/examples/data/test_bent_sine_1min_period_300%3E1500_delta0002.csv, it has a character '>' which is invalid for file name, and makes my checkout always fail. Would you pls fix that ? thanks!

[Kismuz]

done

[mysl]

I updated to latest code, it seems not helping. And this time, the More control section in the notebook raised exception as follows. It looks it's somehow blocked when the environment is trying closing.

=============================================================
Env.dataset: <btgym.datafeed.BTgymDataset object at 0x000000EEF8D65E80>

Env.strategy: <class 'btgym.strategy.base.BTgymBaseStrategy'>

Env.engine: <backtrader.cerebro.Cerebro object at 0x000000EE83F49F60>

Env.renderer: <btgym.rendering.renderer.BTgymRendering object at 0x000000EEF8D65630>

Env.network_address: tcp://127.0.0.1:5555

Parameters [engine]:
start_cash : 100
broker_commission : 0.002
fixed_stake : 10

Parameters [dataset]:
filename : ../examples/data/DAT_ASCII_EURUSD_M1_2016.csv
sep : ;
header : 0
index_col : 0
parse_dates : True
names : ['open', 'high', 'low', 'close', 'volume']
timeframe : 1
datetime : 0
open : 1
high : 2
low : 3
close : 4
volume : -1
openinterest : -1
start_weekdays : [0, 1, 2]
start_00 : True
episode_duration : {'days': 1, 'hours': 23, 'minutes': 55}
time_gap : {'days': 0, 'hours': 5}

Parameters [strategy]:
state_shape : {'raw_state': Box(30, 4)}
drawdown_call : 30
target_call : 10
dataset_stat : None
episode_stat : None
portfolio_actions : ('hold', 'buy', 'sell', 'close')
skip_frame : 1

Parameters [render]:
render_state_as_image : True
render_state_channel : 0
render_size_human : (6, 3.5)
render_size_state : (7, 3.5)
render_size_episode : (12, 8)
render_dpi : 75
render_plotstyle : seaborn
render_cmap : PRGn
render_xlabel : Relative timesteps
render_ylabel : Value
render_title : local step: {}, state observation min: {:.4f}, max: {:.4f}
render_boxtext : {'fontsize': 12, 'fontweight': 'bold', 'color': 'w', 'bbox': {'facecolor': 'k', 'alpha': 0.3, 'pad': 3}}
plt_backend : Agg

ZMQError Traceback (most recent call last)
in ()
56
57 # Clean up:
---> 58 MyEnvironment.close()

c:\programdata\anaconda3\envs\btgym-env\lib\site-packages\gym\core.py in close(self)
164 self.render(close=True)
165
--> 166 self._close()
167 env_closer.unregister(self._env_closer_id)
168 # If an error occurs before this line, it's possible to

c:\study\btgym-master\btgym\envs\backtrader.py in _close(self)
695 """
696 self._stop_server()
--> 697 self._stop_data_server()
698 self.log.info('Environment closed.')
699

c:\study\btgym-master\btgym\envs\backtrader.py in _stop_data_server(self)
836 if self.data_server is not None and self.data_server.is_alive():
837 # In case server is running and is ok:
--> 838 self.data_socket.send_pyobj({'ctrl': '_stop'})
839 self.data_server_response = self.data_socket.recv_pyobj()
840

c:\programdata\anaconda3\envs\btgym-env\lib\site-packages\zmq\sugar\socket.py in send_pyobj(self, obj, flags, protocol)
474 """
475 msg = pickle.dumps(obj, protocol)
--> 476 return self.send(msg, flags)
477
478 def recv_pyobj(self, flags=0):

zmq\backend\cython\socket.pyx in zmq.backend.cython.socket.Socket.send()

zmq\backend\cython\socket.pyx in zmq.backend.cython.socket.Socket.send()

zmq\backend\cython\socket.pyx in zmq.backend.cython.socket._send_copy()

zmq\backend\cython\socket.pyx in zmq.backend.cython.socket._send_copy()

c:\programdata\anaconda3\envs\btgym-env\lib\site-packages\zmq\backend\cython\checkrc.pxd in zmq.backend.cython.checkrc._check_rc()

ZMQError: Unknown error

[mysl]

It looks there might be a long way to go to support running btgym on windows. I might need to switch to linux. Thanks anyway!

[Kismuz]

@mysl ,
I have pushed branch:
https://github.com/Kismuz/btgym/tree/force_data_server_shutdown
try if it works for you.

[mysl]

@Kismuz thanks, tried the force_data_server_shutdown, it looks the issue that environment.close failed resolved. But in the last step of running the agent, there is an AssertionError. I checked the console running the jupyter kernel, it shows some pickling error of pandas dataframe, which I guess might be the reason for the AssertionError.

==========================================================
AssertionError Traceback (most recent call last)
c:\study\btgym-master\btgym\envs\backtrader.py in _reset(self, state_only)
614 try:
--> 615 assert self.observation_space.contains(self.env_response[0])
616
AssertionError:

[Kismuz]

draw_episode() again...
Have you ever got correct episode renderings under Win?

[mysl]

yeah, I got episode rendering under windows the first time I posted in this thread. It will output several plots and raise the resource unavailable error. See my previous response, there is a picture attached there.
After I updated the code, I can't get the rendering anymore.

[Kismuz]

At picture attached there is state rendering. I meant picture with entire episode like this one:

https://github.com/Kismuz/btgym/blob/master/examples/img/2017-11-24_18.37.50.png

[mysl]

then NO

[Kismuz]

...than it definitely was draw_cerebro() right from the start. Pickle serialisation error when starting DrawCerebro subprocess. Exactly, it fails to serialise pandas.dataframe object and latter can only be found inside cerebro instance containing final episode data. Something similar to logger issue; Win specific. Have to figure how to sort it out.

[Kismuz]

Switched entire package to logbook module; rendering still unresolved.

[Kismuz]

Closed due to long inactivity period.