How to run some code developed in Jupyter-Notebook?

Many developers, analysts and data scientists use Jupyter Notebook to develop new algorithms. Here, we will walk you through all the steps required to move your code from the Notebook to a production-ready environment: the Ryax platform.

This guide assumes that you are already familiar with the Ryax platform. If not, please follow the tutorials first.

What do we want to achieve?

We want to create a service that detects objects on a given video.

As we go down that path, we recommend answering the 4 following questions that help designing Ryax workflows:

1. What triggers the workflow? In our case, users will send their video using an online form.

2. Who needs the results of the workflow? Here, only a Ryax user wants to access the final video.

3. What data do we need to run the required computations? We have one main computation: an algorithm that detects objects on an image. It needs to access images, then output the same images with the detected object circled and named.

4. Are my computations done in several steps? Even if the selected algorithm has several steps, we don’t see how splitting it will make it more re-usable: let’s keep it as a single workflow step.

As a conclusion, we will need the following modules:

• a form source

• a processor able to split a video frame by frame

• the algorithm detecting objects on images

• a processor that creates a video from a set of images

On this tutorial, we will focus on the most interesting module: tfdetection. This processor does the detection of objects on images. The source code of all modules, as well as the Juptyer-Notebook sample can be found here.

The notebook

The developped Notebook can be found here.

It uses Tensorflow and a local library called object_detection. We can see at the end of the notebook an example of a run.

Create the module

We put everything required to run the code in a new directory called “tfdetection” in modules/. To be able to run the code in Ryax, we add 3 files:

• handler.py, to launch the code

• requirements.txt, to declare the dependencies

• ryax_metadata.yaml, to declare the meta-data

NB: more detailed explanations on how to build Python modules in Ryax can be found here

Step1: Dependencies

What are the dependencies of our code? We can see at the beginning of the notebook that the code depends on pillow, matplotlib, opencv4, tensorflow, urllib3 and, object_detection.

object_detection is a local module available through the object_detection/ sub-directories. Let’s copy the whole directory in the module directory. Ryax integrates all files present in the module directory inside the Ryax module.

We’ll declare the other dependencies using the standard requirements.txt file:

pillow
matplotlib
ryaxpkgs.opencv4-full
tensorflow
urllib3

Note: ryaxpkgs.opencv4-full is an optimized version of opencv4 made for Ryax.

Then, we will use the code in the notebook to create the handler.py file required to run some python code in Ryax.

Step2: extract code

The entry point of a Ryax processor in python is a file called handler.py. This file has to declare a function called handle that takes a dict as parameter and returns a dict.

You can see the code here

The handle function will act as an interface between Ryax’s input/output system and the main object detection code. The other functions will make the right calls to perform object detections.

Step3: ryax_metadata.yaml

Finally, we need to declare all the metadata required by Ryax to understand the code. This is done in the ryax_metadata.yaml file.

apiVersion: "ryax.tech/v1"
kind: Functions
spec:
  id: tfdetection
  human_name: Tag images using Tensorflow
  type: python3
  version: "1.0"
  logo: logo.png
  inputs:
  - help: Model
    human_name: Model name
    name: model
    type: string
  - help: Directory with images to be tagged in any format accepted by OpenCV
    human_name: Images
    name: images
    type: directory
  outputs:
  - help: Path of tagged images
    human_name: Tagged images
    name: tagged_images
    type: directory

Don’t forget to commit and push your code :).

Load modules

Now that the Ryax modules are ready, we will get them in the platform. Log in to Ryax. Go to the “Repositories” screen. Create a new repository called video_detection with the URL https://gitlab.com/ryax-tech/workflows/video_detection.git

Then, scan this repository to detect the modules. This repository is set up for public access, just type anonymous in both login & password fields.

Once the scanning ends, scan results should show the 3 modules that we want to build: Reassemble video from frames., Cut video frame by frame and, Tag images using Tensorflow. Let’s click on the build button for each one of them.

Note

“Building a module” consists of preparing the modules code to be able to run on the Ryax platform. Ryax fetches all the dependencies and packs them together for your code to run within the platform. More info on the “Module build” concept [here](https://docs.ryax.tech/tutorials/handling.html#module-builds)

You can go to the “module builds” screen and refresh the page until the 3 modules are built. Once they are build, these modules are available in the module store. We can now start assembling the workflow.

Create the workflow

Navigate to the Studio and create a new workflow. Click on the “+” button in the right panel. Add the following modules, you can use the search bar to find them: “Integrated form to trigger new streams”, “Cut video frame by frame”, “Tag detected objects on images using tensorflow” and “Create a video from multiple images”.

Link them together this way:

The last step is to fill in inputs for our modules.

Let’s start with the “Integrated form to trigger new streams” source. Click on this module, then go to the “input” tab. Fill the introduction text with “Put a video here to detect objects on it.”. To configure the form’s fields, we’ll edit the output of this module. Go to the “output” tab. Add a new output called “video” with a type “file”.

Now let’s fill the input of “Cut video frame by frame”. It only require one input file: set it to to the output of the previous module (use “reference value” and select the previous module’s output).

Then, for the module “Tag detected objects on images using tensorflow” we will use the ssdlite_mobilenet_v2_coco_2018_05_09 model and the images coming from the previous module.

Finally, the “Frame” input of “Create a video from multiple images” will assemble the tagging’s results.

Run it

The workflow is ready to be deployed.

Deploying a workflow conists of launching all modules in the infrastructure so that they are ready to ingest data. More info here.

Click on the deploy button. Refresh the page until all modules are deployed (the status should be green), and the workflow is in the “deployed” status.

Now, click on the “Integrated form to trigger new streams” module. On the information tab, there should be a new button available: “access workflow”. Click on it.

The form created in the previous steps is now open. Submit a video in which you want to detect objects. You can use this one. Submit the form.

We can now observe executions. Don’t forget to refresh the screen using the refresh button to update the status of the executions.

Once the last module has been executed, click on it. In the output data, you can download the result file: our tagged video.