Importing Multiple TensorFlow Models (Graphs)

Updated 6th March 2018: importing operations by name.

There are a lot of great things about TensorFlow. BUT once I figure out how to import my trained model (graph), I wasn’t able to import second model and use it alongside with the first one. The importing is pretty slow and I don’t want to do it more than once. On the other hand, squashing everything into one model seems to me pretty impractical.

In this tutorial, I will show how to save and import TensorFlow model. Even more, how to import multiple models alongside.

Importing TensorFlow Model

Before we dive into importing multiple models, let’s go over importing single model (officia documentation). First, we need to create a model, train it and save it. I don’t want to go into details; just pay attention to the way of saving the model and don’t forget to name the operations.

Here comes the first important point! If you ever want to access some variable after importing, you have to assign a name to it, or it will be named something like ‘Placeholder_1’. Similarly, you should name every important operation in your code. In complex models, it is good practice to use scopes, but that’s a different topic. The important point is that if you want to later use some operation, you have to either name it or put it into a collection.

Once we save the model, we should have these files: model_name.index, model_name.meta and some other files. If we use checkpoints, there will be additional files like model_name-1000 with a number corresponding to the global_step variable.

Now we are ready to import the model. Importing is fairly simple and we need just two functions: tf.train.import_meta_graph and saver.restore(). We also have to provide a correct path to the saved files. In case that we want to use the model on a different machine, we have to set: clear_devices=True. Then we can access saved “operation” from previously created collections or by name. If you use scopes, you have to include them in name of a operation. During actual usage of the operation, we have to feed input placeholders as {'PlaceholderName:0': data}. It won’t work without the ‘:0’ part.

Multiple Models (Graphs)

When we finally figure out how to import single TensorFlow model, another issue occurs. If we try to import multiple modules using this way, there will be variable collision and it won’t work. The reason behind that is something called a default graph. The collision occurs, because we are importing everything into the default graph which is used in the session. When we start the session, we can specify different previously created graph as tf.Session(graph=MyGraph). So, in order to use multiple models alongside, we have to import them into two different graphs and then run them in different sessions.

For this porous I created a custom class which imports the model from given location into a local graph. This class also provides run function which feeds the model with the data. This class works for me because I always put the output operation into activation collection (or name it activation_opt), and I name the input placeholder as ‘x’. You may need to extend this class in case you do differently.

Summary

In the end, importing multiple modules isn’t that hard if you understand TensorFlow mechanics. This solution may not be perfect, but it is quick and easy. You can also choose between importing operations form collections of by name. For a demonstration of the whole process, there is a special notebook:

BTW, I’m a bit confused about the meaning of the words ‘model’ and ‘graph’. Because to me they seem as synonyms, but TensorFlow documentation uses word ‘model’ somehow randomly. Let me know if you know the difference.