Building custom Docker images for custom blocks

Omniscope Evo can be configured to run custom blocks in docker containers. . When this option is enabled it is possible to use custom images to run the scripts. This can be useful if, for example, the script needs some bespoke binary that are not normally available on the system, or a more elaborate sandbox environment.

This article will describe how to build a custom image that can be used in custom blocks, and provide an example of such images for a community block. You can find more information on how to configure Omniscope to execute custom blocks in docker containers  here.

By default, Omniscope will run custom blocks from containers instantiated from the Docker images Viosokio provides. The Images contain both the language interpreters (R or Python) as well as the Omniscope Api library used by the custom block script to communicate with Omniscope. It is possible however, to specify a custom image in the advanced options:

In order for the image to be used in the custom block, it must satisfy some requirements. 

Building custom images based on Visokio Images

The easiest way to create a custom image is to use one of Visokio's provided docker images as base image, and add the components that are necessary.

The base image for python is: europe-west2-docker.pkg.dev/visokiowebsite/visokio-custom-block/visokio-python-custom-block:0.0.1 (Python).

The base image for R is: europe-west2-docker.pkg.dev/visokiowebsite/visokio-custom-block/visokio-r-custom-block:0.0.1 (R)

In the next section we will describe how to create a custom image for a custom block.

Building custom images not based on Visokio Images

If it is not possible to use one of Visokio base image for your custom image (for example it needs to be based on another image), then there are some requirements the image needs to satisfy:

• Linux based image: currently there is no support for windows containers
  
  
• Debian based image: To be able to install system packages, Omniscope assume a Debian based image, (e.g., Debian, Ubuntu ....) with apt installed and available in the path.
• Python3 interpreter: If the image will execute a Python script, then only python 3 is supported in docker containers. Omniscope will use the python3 and pip3 commands, and it assumes both of them are present and available in the path 
• R interpreter: If the image will execute an R script, then Omniscope will assume Rscript interpreter to be installed and available in the path.

If your script needs access to Omniscope Api (as it most likely the case), then the Omniscope api libraries needs to be installed before running the container. To make Omniscope install the client library tick the 'Installed Omniscope Api libraries' option.  The first time the dependencies are installed from the image it can take several minutes, especially for custom blocks using R. Later executions of the custom block will be much faster as Omniscope will be able to re-use the image generated previously.

In images using one of Visokio images as a base, it is not necessary to install the Omnisope Api libraries. 

Image Retention

By default Omniscope is configured to prune images it generated if their collective disk space usage is beyond a configurable threshold. You can read more about how to configure the threshold here.  Images are re-created on demand when custom blocks execute.

By default base images are not pruned by Omniscope: this is because Omniscope cannot be sure the image can be re-obtained if necessary (e.g., a custom block needing the image is executed again). If you want Omniscope to be able to delete you custom image if more disk space is needed you can add 

LABEL  "visokio-image-retention"="prune-on-cleanup"

in the Dockerfile

An Example of a Custom Image

In this example we will build a custom image based on the Visokio base image, that can be used to print Omniscope reports to PDF files. This is a variation of the Report to Pdf community block. The script uses both a chromium browser and a custom binary called omniprint-linux  to create PDF file from Omniscope reports.

This is a good use case to use a custom image: by using a custom image the script can be aware of the location of the chromium browser, and the omniprint-linux utility, and it is not necessary for the user to configure the location of those binaries.

Also, as long as Docker is installed on the system, the process is independent from the operation system used, and whether or chromium, python or omniprint-linux are actually present on the machine.

The custom block uses a Python script, hence in the Dockerfile will use europe-west2-docker.pkg.dev/visokiowebsite/visokio-custom-block/visokio-python-custom-block:0.0.1 as base, and simply add omniprint-linux and install the chromium package via apt.

FROM europe-west2-docker.pkg.dev/visokiowebsite/visokio-custom-block/visokio-python-custom-block:0.0.1
ADD Omniprint-linux.tar.gz .
RUN apt update -y && \
    apt install -y chromium

The script itself is a variation of the Report tab to PDF community block.

You can download Dockerfile, custom block and build script here.