Deploying a Notebook

This example highlights the Non-Production environment within kaos.

This example assumes you are a Data Scientist using kaos with a running endpoint

Prerequisites

The following steps are required before being able to deploy a hosted notebook. Details on the Notebook Pipeline internals can be found here.

Initialization

The kaos ML platform is fully functional when initialized with a running endpoint from a System Administrator. See Workflows for more information regarding different kaos personas.

kaos init -e <running_endpoint>

Create a workspace

A workspace is required within kaos for organizing multiple environments and code. Refer to Workspaces for additional information.

$ kaos workspace create -n local
Successfully set local workspace

1. Deploy a basic notebook

kaos provides a basic no-frills hosted notebook via kaos notebook deploy. It is the ideal approach to quickly and easily having an experimental ML environment. Keep note of the token for accessing the notebook (upon receiving the running url).

$ kaos notebook deploy
✔ Notebook deployed - check status with kaos notebook list
Please use "jfriedman" as the notebook token

The status of the notebook can be queried with kaos notebook list

The status of the notebook provides information regarding its creation, author, name, state and url. The notebook can be accessed when the state indicates PIPELINE_RUNNING.

$ kaos notebook list
+----------------------------------------------------------------------------------------------------------------------------------------+
| RUNNING |
+-----+-------------------------------+--------------------------+------------------+----------------------------------------+-----------+
| ind | created_at | name | state | url | user |
+-----+-------------------------------+--------------------------+------------------+----------------------------------------+-----------+
| 0 | Mon, 29 Jul 2019 19:35:55 GMT | notebook-local-jfriedman | PIPELINE_RUNNING | localhost/notebook-local-jfriedman/lab | jfriedman |
+-----+-------------------------------+--------------------------+------------------+----------------------------------------+-----------+

Resulting Notebook

Connecting to the presented url will present the following familiar landing page. Note that the token was previously displayed when deploying the notebook (i.e. jfriedman).

localhost/notebook-local-jfriedman/lab

The resulting basic notebook is as advertised - free of any code, data and/or requirements. Files can still be added by either dragging and dropping into the browser or by following the File/<Open from Path> option.

hosted notebook python3 kernel

kaos deploys basic notebooks for experimentation with one simple command

2. Deploy a custom notebook

The previously explained basic notebook is slightly limiting when specific packages or large amounts of code and/or data are desired. For this reason, kaos also supports custom notebooks based on the same structure as a source bundle.

A custom template is provided within kaos to ensure deploying an experimental environment is extremely straightforward. The template can be accessed as follows.

$ kaos template get --name notebook
Successfully loaded notebook template

The same kaos notebook deploy command with an optional --source_bundle option allows a more custom experimental ML environment. Once again, please keep note of the token for accessing the notebook (upon receiving the running url).

$ kaos notebook deploy -s templates/notebook
Submitting source bundle: templates/notebook
Compressing data bundle: 100%|███████████████████████████|
✔ Notebook deployed - check status with kaos notebook list
Please use "jfriedman" as the notebook token

The notebook can be accessed when the state indicates PIPELINE_RUNNING.

$ kaos notebook list
+----------------------------------------------------------------------------------------------------------------------------------------+
| RUNNING |
+-----+-------------------------------+--------------------------+------------------+----------------------------------------+-----------+
| ind | created_at | name | state | url | user |
+-----+-------------------------------+--------------------------+------------------+----------------------------------------+-----------+
| 0 | Mon, 29 Jul 2019 19:56:48 GMT | notebook-local-jfriedman | PIPELINE_RUNNING | localhost/notebook-local-jfriedman/lab | jfriedman |
+-----+-------------------------------+--------------------------+------------------+----------------------------------------+-----------+

Resulting Notebook

The same landing page will greet upon connection. Note that the token was previously displayed when deploying the notebook (i.e. jfriedman). The difference is that numpy was added to the requirements.txt when building the notebook, enabling direct access to the package.

hosted notebook python3 kernel with numpy package

kaos deploys custom notebooks for experimentation with one simple command

Adding Data

The custom notebook also contains a useful example_notebook.ipynb for accessing data loaded by kaos. The "abbreviated" MNIST dataset is used to show how data is attached to a running notebook.

$ kaos notebook deploy -d templates/mnist/data
Attaching data bundle: templates/mnist/data
Compressing data bundle: 100%|███████████████████████████|

The resulting templates/mnist/data directory is immediately attached and ready for experimentation within the hosted notebook.

kaos can attach any data to a running notebook via kaos notebook deploy