Tuesday 20 August 2013

Retrieving results from IPython parallel hub database

Retrieving results from IPython parallel hub database

I've been running parallel jobs on a SGE cluster using IPython parallel. I
submit my jobs and retrieve the results from the hub database (SQlite) at
a later time when all the jobs have finished, using the jobs message ID.
This worked fine till my controller crashed; on restarting the controller,
I couldn't retrieve the jobs submitted to the old controller. I got this
error:
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/IPython/parallel/controller/hub.py",
line 1281, in get_results
raise KeyError('No such message: '+msg_id)
KeyError: u'No such message: 7f1996c0-deb0-4d7c-8782-619c86d2d064'
The database file (tasks.db) still exists and has the same size as before
the hub crashed. So, I'm sure the results are in the database. Can I
retrieve them using the new controller? Also, if I use the bd_query
command:
rc.db_query({'msg_id' : '7f1996c0-deb0-4d7c-8782-619c86d2d064'})
I get an empty result.

No comments:

Post a Comment