Page 1 of 1
Simulation freezes in Stepping Mode
Posted: 04 Jan 2024, 23:47
by major_david
Hello,
I'm working on a project using Coppelia as a simulation environment for reinforcement learning. The simulation is configured on Stepping Mode. Everything looks great, and most of the time I get the desired results from Coppelia. However, during the training, sometimes the simulation freezes without any errors. I used some tools to find which line the Python stopped working. Apparently, the simStep() function is called but the ZMQremote API fails to execute it, and the program freezes. The traceback is as follows:
Code: Select all
self.sim.step()
File "/home/major/anaconda3/envs/Main3/lib/python3.11/site-packages/coppeliasim_zmqremoteapi_client/__init__.py", line 161, in <lambda>
setattr(ret, k, lambda *a, func=f'{name}.{k}': self.call(func, a))
^^^^^^^^^^^^^^^^^^
File "/home/major/anaconda3/envs/Main3/lib/python3.11/site-packages/coppeliasim_zmqremoteapi_client/__init__.py", line 147, in call
reply = self._recv()
^^^^^^^^^^^^
File "/home/major/anaconda3/envs/Main3/lib/python3.11/site-packages/coppeliasim_zmqremoteapi_client/__init__.py", line 106, in _recv
rawResp = self.socket.recv()
^^^^^^^^^^^^^^^^^^
File "zmq/backend/cython/socket.pyx", line 805, in zmq.backend.cython.socket.Socket.recv
File "zmq/backend/cython/socket.pyx", line 841, in zmq.backend.cython.socket.Socket.recv
File "zmq/backend/cython/socket.pyx", line 194, in zmq.backend.cython.socket._recv_copy
File "zmq/backend/cython/checkrc.pxd", line 13, in zmq.backend.cython.checkrc._check_rc
I'm using the latest version of Coppelia(rev16) alongside Python 3.11.
Thanks
Re: Simulation freezes in Stepping Mode
Posted: 05 Jan 2024, 07:25
by coppelia
Hello,
I think there are missing lines in the error output.
Best would be if you could provide a minimalistic, self-contained script that triggers the error, but I guess that would be difficult.
Can you try to use the local source instead, i.e.:
Code: Select all
# instead of:
from coppeliasim_zmqremoteapi_client import RemoteAPIClient
# do:
from zmqRemoteApi import RemoteAPIClient
Then you could modify the code via some prints in
programming/zmqRemoteApi/clients/python/src/coppeliasim_zmqremoteapi_client/__init__.py
But it appears that ZMQ fails to receive some data, but the reason is not indicated/missing in your error output
Cheers
Re: Simulation freezes in Stepping Mode
Posted: 12 Mar 2024, 08:07
by Aidewen
Hi, I seem to be facing the same issue.
I'm using ZMQRemoteAPI and I'm running python code in a multiprocessing thread (Manager). The python code opens the coppelia executable in headless mode as follows.
Code: Select all
args = {
'stdin' : subprocess.DEVNULL,
'stderr': subprocess.PIPE,
'stdout': subprocess.PIPE,
'startupinfo': startupinfo,
'env' : env,
'creationflags': subprocess.CREATE_NEW_PROCESS_GROUP + subprocess.CREATE_NEW_CONSOLE
}
subprocess.Popen([coppeliapath, '-GzmqRemoteApi.rpcPort=' + str(port), '-h'], shell=True, **args)
SetStepping is set to true, but the code seems to freeze after the third step in "self.cntsocket.recv(0 if wait else zmq.NOBLOCK)" in getStepCount from RemoteAPIClient. I recently discovered the emulated headless mode, so this issue is not present if I don't use it. Also, headless mode is not an issue if I don't use threading python packages. Any suggestions?
Context:
- Packages: `subprocess`, `psutil`, `multiprocessing.Manager`
Re: Simulation freezes in Stepping Mode
Posted: 12 Mar 2024, 16:28
by coppelia
Hello,
make sure to use the latest CoppeliaSim, since CoppeliaSim V4.6 features a much improved Python support.
Cheers
Re: Simulation freezes in Stepping Mode
Posted: 12 Mar 2024, 16:44
by Aidewen
Unfortunately, I am not able to upgrade to a higher edition as the current code I am working with requires a previous version.
Fortunately, I was able to locate the source of my problems. I was loading the RG2 model with a lua script code that seem to be incompatible with the simulation during runtime in headless mode.
Code: Select all
function sysCall_sensing()
local s=sim.getObjectSel()
local show=(s and #s==1 and s[1]==modelHandle)
if show then
if not ui then
local xml =[[<ui title="xxxx" closeable="false" placement="relative" layout="form">
<button id="1" text="open" checkable="true" checked="true" auto-exclusive="true" on-click="openClicked"/>
<button id="2" text="close" checkable="true" auto-exclusive="true" on-click="closeClicked"/>
</ui>]]
ui=simUI.create(xml)
if uiPos then
simUI.setPosition(ui,uiPos[1],uiPos[2])
else
uiPos={}
uiPos[1],uiPos[2]=simUI.getPosition(ui)
end
simUI.setTitle(ui,sim.getObjectAlias(modelHandle,1))
simUI.setButtonPressed(ui,1,not closing)
simUI.setButtonPressed(ui,2,closing)
end
else
if ui then
uiPos[1],uiPos[2]=simUI.getPosition(ui)
simUI.destroy(ui)
ui=nil
end
end
end
After that, everything runs smoothly.
Re: Simulation freezes in Stepping Mode
Posted: 13 Mar 2024, 07:22
by coppelia
Aidewen wrote: ↑12 Mar 2024, 16:44
Unfortunately, I am not able to upgrade to a higher edition as the current code I am working with requires a previous version.
...
Can you tell us what exactly is not compatible, so that we can improve on this?
Cheers
Re: Simulation freezes in Stepping Mode
Posted: 19 Mar 2024, 13:31
by Aidewen
So if the python code (using ZeroMQ) is running coppelia in headless mode and there is a lua script attached to a model that runs code during actuation as follows,
or
Code: Select all
if not ui then
local xml =[[<ui title="xxxx" closeable="false" placement="relative" layout="form">
</ui>]]
ui=simUI.create(xml)
end
It will freeze the simulation stepping of the python code.
You can try the following python code:
Code: Select all
try:
# If Coppelias Sim Version 4.5.1
from src.coppeliasim_zmqremoteapi_client import RemoteAPIClient
except:
# If Coppelias Sim Version 4.3 to 4.4
from zmqRemoteApi import RemoteAPIClient
import sys
sys.path.insert(0,'..')
from mmsl_lib.utils import global_configs
import os
import time
import subprocess, psutil
def _get_full_path(filename:str) -> str:
"Retrieves the full path of a file."
filename = os.path.abspath(filename)
assert os.path.exists(filename), 'File path does not exist!'
return filename
def _popen(cmd: list) -> subprocess.Popen:
"Opens a new process using subprocess.Popen."
env = os.environ
env.setdefault("QT_DEBUG_PLUGINS","1")
startupinfo = subprocess.STARTUPINFO()
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
args = {
'stdin' : subprocess.DEVNULL,
'stderr': subprocess.PIPE,
'stdout': subprocess.PIPE,
'startupinfo': startupinfo,
'env' : env,
'creationflags': subprocess.CREATE_NEW_PROCESS_GROUP + subprocess.CREATE_NEW_CONSOLE
}
process = subprocess.Popen(cmd, shell=True, **args)
return process
def run(headless: bool = False):
gconfigs = global_configs('../configs/configs.ini')
coppeliapath = 'path/to/coppelia/exe'
host='localhost'
port = 23002
model_path = 'path/to/model/ttm'
#--------------------------------------------------------------------------
'Opening CoppeliaSim'
print('Opening Coppelia')
coppeliapath = _get_full_path(coppeliapath)
options = [coppeliapath, '-GzmqRemoteApi.rpcPort=' + str(port)]
if headless: options += ['-h']
app = _popen(options)
time.sleep(5) #Waiting for CoppeliaSim to Open
parent = psutil.Process(app.pid) # getting id from opened CoppeliaSim exe
app = parent.children(recursive=True)
#--------------------------------------------------------------------------
client = RemoteAPIClient(host, port)
client.setStepping(True)
sim = client.getObject('sim')
print('Loading Model')
sim.loadModel(_get_full_path(model_path))
time.sleep(5) #Waiting for CoppeliaSim to Open
print('Starting Simulation Model')
sim.startSimulation()
print('Waiting 10 Secs (Simulation)')
client._wait(10, True)
print('Ending Simulation Model')
sim.stopSimulation()
#--------------------------------------------------------------------------
print('Closing Coppelia')
if app != None:
time.sleep(2.0) #Sleeping before Terminating CoppeliaSim Executable
for ps in app:
subprocess.check_output("Taskkill /PID %d /F" % ps.pid)
app = None
if __name__ == '__main__':
run(headless=True)
print('Completed')
And create the following nonthreaded lua script in a model:
Code: Select all
function sysCall_init()
-- do some initialization here
end
function sysCall_actuation()
-- put your actuation code here
end
function sysCall_sensing()
--#Issue num 1
--local s=sim.getObjectSel()
--#Issue num 2
if not ui then
local xml =[[<ui title="xxxx" closeable="false" placement="relative" layout="form">
</ui>]]
ui=simUI.create(xml)
end
end
function sysCall_cleanup()
-- do some clean-up here
end
This will replicate the issue I'm facing. We have to be careful what functions are used in headless mode!
Re: Simulation freezes in Stepping Mode
Posted: 26 Mar 2024, 14:48
by coppelia
Hello,
it is very difficult to find out what exactly is going on. Normally one should not start CoppeliaSim via a Python script, since this could confuse Qt which requires CoppeliaSim to be started in the main thread.
There is however the
Python application client in V4.6, that allows to start and run CoppeliaSim via Python, e.g. python coppeliaSim.py
In there however too, you can easily block CoppeliaSim if not careful, since Python code will run the Python script and the main simulation loop successively.
Cheers