Multiple Graphs in one Tensorflow Session

Question

I am currently trying to implement a code that will allow my drone to navigate indoor using tensorflow. I need to run two models in a single session.

One is for the main navigation--which is a retrained Inception V3 model responsible for classifying hallway images and performing move forward, left, or right decision--the second is an object tracking model that will track objects and compute there relative distance to the camera.

I don't know how to use multiple graphs in one session so I tried to create a separate session inside the loop which creates a big overhead and causes my script to run at 0 FPS.

def inception_model():
# Graph for the InceptionV3 Model
graph = load_graph('inception_v3_frozen/inception_v3_2016_08_28_frozen.pb')

with tf.Session(graph = graph) as sess:
    while camera.isOpened():
        ok, img = camera.read()
        cv.imwrite("frame_temp.jpeg", img)
        t = read_tensor_from_image('frame_temp.jpeg')

        input_layer = "input"
        output_layer = "InceptionV3/Predictions/Reshape_1"

        input_name = "import/" + input_layer
        output_name = "import/" + output_layer

        input_operation = graph.get_operation_by_name(input_name)
        output_operation = graph.get_operation_by_name(output_name)

        results = sess.run(output_operation.outputs[0], {
            input_operation.outputs[0] : t
        })
        results = np.squeeze(results)

        top_k = results.argsort()[-5:][::-1]
        for i in top_k:
            print(labels[i], results[i])

# inception_model()
with tf.Session(graph = object_detection_graph) as sess:
    while camera.isOpened():
        ok, img = camera.read()
        cv.imwrite("frame_temp.jpeg", img)
        img = np.array(img)
        rows = img.shape[0]
        cols = img.shape[1]

        inp = cv.resize(img, (299, 299))

        # inception_model()
        # # Graph for the InceptionV3 Model
        # graph = load_graph('inception_v3_frozen/inception_v3_2016_08_28_frozen.pb')

        # t = read_tensor_from_image('frame_temp.jpeg')

        # input_layer = "input"
        # output_layer = "InceptionV3/Predictions/Reshape_1"

        # input_name = "import/" + input_layer
        # output_name = "import/" + output_layer

        # input_operation = graph.get_operation_by_name(input_name)
        # output_operation = graph.get_operation_by_name(output_name)

        # with tf.Session(graph = graph) as sess:
        #     results = sess.run(output_operation.outputs[0], {
        #         input_operation.outputs[0] : t
        #     })
        # results = np.squeeze(results)

        # top_k = results.argsort()[-5:][::-1]
        # for i in top_k:
        #     print(labels[i], results[i])


        inp = inp[:, :, [2, 1, 0]]  # BGR2RGB


        # Run the model
        out = sess.run([object_detection_graph.get_tensor_by_name('num_detections:0'),
                        object_detection_graph.get_tensor_by_name('detection_scores:0'),
                        object_detection_graph.get_tensor_by_name('detection_boxes:0'),
                        object_detection_graph.get_tensor_by_name('detection_classes:0')],
                    feed_dict={'image_tensor:0': inp.reshape(1, inp.shape[0], inp.shape[1], 3)})

Answer 1

You don't have to create new sessions on each iteration. Create them once and keep calling their run methods. Tensorflow support multiple active sessions.

Another option is to have a single Graph object and a single Session. The graph can contain both of your models as disconnected sub-graphs. When you ask for a tensor in Session.run() Tensorflow will run only what is necessary to compute the tensor you asked for. So, the other sub-graph will not run (though it will take some, probably very small, time to prune it away)