Saving and restoring model state
This task is related to the task of creating a canvas-based modeller for ASCEND, because we would like for canvas-based models to be saved including their current solved state. This will make them more useful for review by other users, and for referring back to the solution of a previous study-case.
The current status of this work is that Arijit implemented a Python-based way of saving the model state; we would like to think about other more portable approaches though.
Requirements for saving and restoring model state:
- must store all non-constant values defined in the model
- this include booleans, integers, symbols and reals.
- includes not just solver_var's but also attributes of variables, such as lower_bound and upper_bound
- note that attributes are not hard-wired into ASCEND but are defined on the fly in .a4c and .a4l files.
- should not make use of solvers_var_list etc, because we want our saved state to be solver-agnostic.
- should be possible to save state of a whole model or any sub-tree (starting from a certain instance and working down)
- must be able to deal with arrays, whens, conditionals, etc.
- need to identify what should be done in the case of 'sets'.
- need to avoid storing multiple copies of 'cliqued' variables (ARE_THE_SAME, ALIASES).
- should be implemented at the libascend C level, so that it can be readily be used by across all GUIs if desired
- would replace the current hybrid Tcl+C apporach (producing .a4v files)
- needs to be interfaced with Python in a way that the resulting return value can be pickled and unpickled efficiently.
- suggested Python datatype is the 'dict'
- some care required when dealing with type conversions between Python and C and vice versa.
- suggest a new SWIG object being the 'CachedValue' or similar, able to hold bool, int, real, string (and set?) values.
More detailed design should be discussed and more specific design worked out here.
Note current partial functionality implemented in ascend/compiler/instance_io.c (a 'restore' function was never implemented).
Note current Tcl/Tk GUI functionality implemented in tcltk/interface/BrowserProc.c and tcltk/tk/BrowserProc.tcl. This approach generates the stored file using C code, then utilised the Tcl interpreter to act as a parser for the stored file when reading it back (see Brow_parse_values).
Functions could be written that would create an opaque object via SWIG that would do the job of serialising and unserialising values from the instance hierarchy. The opaque object could be interfaced with the 'copy_reg' module or augmented with __setstate__ and __getstate__ functions to allow the object model state object to be saved in a pickle.
Python-to-C flat data conversion
This approach would serialise an instance sub-tree into a set of key-value pairs which could then be exchanged easily both to and from Python. A problem might be the 'heaviness' of the storage format (a lot of space consumed storing strings like 'lower_bound', 'upper_bound', and perhaps a lot more processor-intensive. Some care required with data types -- eg converting from PyObject to string/int/real/bool as required for the Instance type. But relatively easy to implement.
Python-to-C tree conversion
As above, but return a hierarchical dict.
Pure Python implementation
Is it possible to iterate through the tree in Python and produce something workable that's reasonably fast?
Wrapping the old Tcl functionality
The old Tcl/C code uses C code to traverse the model tree and output a text file containing essentially a Tcl script that, when run, does the job of reinserting values into the instance tree by looking up identifiers one-by-one using the 'qlfdid' routines. This approach to restoring model values is possibly not particular efficient, but does currently work. The hazards of using the Tcl interpreter from within Python are a bit scary, however, and it would be nicer to keep language 'cleanness' in the Python code.
Different solvers work in different ways. All ASCEND solvers work off the 'solver lists' that contain the 'flat' form of an ASCEND model including real-valued variables and real-valued relations, boolean-valued vars and rels, WHENs, etc. Instead of solving values from the instance tree, an alternative approach would be to save the contents of the solver lists.
One problem with this approach is that it could make it harder to restore model state for modified models -- what would happen if a variable had been added to or removed from the model?
This approach would however make saving and loading quite a bit easier: all the problems of cliques are eliminated, and there is no need to recursively descend into a tree structure -- everything's flat.
Current method of implementation
Currently, the Canvas Modeler employs Python based way of saving canvas data. When the canvasmodel is solved, it stores the values of all the variables present in the canvasmodel, supplied by the <simulation>.getallVariables() function, into a dictionary named 'saved_data' belonging to the BlockCanvas class. The code which does this in present in the method 'run_canvas' of 'blocklist.py'. It is where the 'saved_data' dictionary gets populated. This is then pickled upon saving of the canvasmodel. Thats how the data gets stored.
Next, comes the retrieval of data upon loading of canvas pickle files, which is carried out in the following manner. First of all, loading of canvas pickle files is performed by the 'fileopen' method of 'blocklist.py', which upon successful retrieval of pickled data, calls another method 'load_presaved_canvas', again belonging to 'blocklist.py'. This method 'load_presaved_canvas' then instantiates the canvasmodel. It then retrieves the values from the 'saved_data' dictionary and assigns them to the variables of instantiated canvasmodel. That is how data is retrieved back from the canvas pickle file.