In the latest update for Gradient Notebooks, we've introduced a number of new features and improvements – including Gradient Datasets, support for interactive widgets, better cell, file, and kernel management, and plenty more!

Let's get right to the updates!

Introducing public and private Gradient Datasets

Gradient Datasets provide an alternative to local notebook storage that can be used across Gradient teams and resources. With this release, it's now possible to mount a dataset within a notebook.

Datasets are available in the Datasets tab of the IDE and files are stored in the /datasets directory.

As part of this release, we've made available a number of common datasets available to all notebooks as public datasets. These datasets are available in the Public tab within the Datasets menu in the IDE.

Public datasets are updated frequently. The initial batch of public datasets includes:

• Tiny ImageNet 200: 100000 images of 200 classes (500 for each class) downsized to 64×64 colored images. Each class has 500 training images, 50 validation images and 50 test images.
• OpenSLR: The LibriSpeech ASR corpus is composed of approximately 1000 hours of 16kHz read English speech, prepared by Vassil Panayotov with the assistance of Daniel Povey. The data is derived from read audiobooks from the LibriVox project, and has been carefully segmented and aligned.
• MNIST: A database of handwritten digits has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image.
• LSUN: contains 10 scene categories, such as dining room, bedroom, chicken, outdoor church, and so on. For training data, each category contains a huge number of images, ranging from around 120,000 to 3,000,000. The validation data includes 300 images, and the test data has 1000 images for each category.
• FastAI: Paperspace's Fast.ai template is built for getting up and running with Practical Deep Learning for Coders. The corresponding public dataset enables notebooks created using the FastAI runtime quick access to the needed demo data.
• COCO: this dataset is a large-scale object detection, segmentation, key-point detection, and captioning dataset. The dataset consists of 328K images, bounding boxes, and their labels.

If you want to create your own dataset, simply use the Team tab within the datasets menu in the notebook IDE to upload and mount your own data.

Here's an example of how to upload a dataset of historic artwork and then accessing the dataset in a notebook with the command !ls ../datasets/historic_art/:

For datasets larger than 5 GB, you will be prompted to upload your data using a Gradient Workflows or the Gradient CLI.

For more information on Gradient's public datasets and data storage, please read through the relevant documentation.

Interactive widgets

In addition to datasets, notebooks now support interactive ipywidgets out of the box!

Jupyter widgets or ipywidgets are useful in a variety of contexts, from modifying the appearance and interpretability of a notebook to enabling a number of deep learning functionalities. Widgets are now available natively – just import the ipywidgets module!

ipywidgets also enable a host of other widgets natively within a notebook, from input prompts to sliders, multiselects, calendars, and more.

Check out the ipywidgets docs for a full list of widgets.

IProgress and PyTorch/TensorFlow dataloaders

Notebooks now also support IProgress text progress bars.

IProgress is used frequently to display useful information like how how many epochs for training remain in a session. With this update, IProgress now renders fully in the notebook.

This extends in particular to both TensorFlow and PyTorch's extended suite of libraries. Functions from libraries like TensorFlow Datasets and Pytorch Lightning now work with no issue.

In addition, progress bars are now enabled for TensorFlow and PyTorch data loaders and training functions to display natively in the notebook. It's no longer necessary to add code into a Python script executed through the terminal to run code containing these functions.

plotly and other plotting libraries with HTML outputs

With ipywidgets facilitating HTML outputs, we can now make full use of the interactive plotting library plotly.

Unlike Matplotlib or Seaborn, plotly allows for making interactive, dynamic changes to the existing plot like removing data classes or zooming. Now we can create plotly graphics and even save them to be embedded on other web pages.

We look forward to seeing what you can do with full widget support.

Now let's talk about updates to the IDE itself.

Cell management updates

First up, we've greatly enhanced and expanded cell operations within notebooks to include familiar concepts from JupyterLab like join, split, insert, and more.

The updated Notebooks IDE also makes it easy to hide cell outputs to clear up more space within a notebook.

We've also expanded the command palette to make it easier to access helpful shortcuts and commands. The list of command palette entities is growing rapidly so be sure to check it out!

The command palette is accessible with the keyboard shortcut command + p.

Updated file manager

We've also introduced drag-and-drop functionality and a few other useful actions to the file manager.

In addition, it's now possible to upload multiple files to a folder and there are a number of new file management options available when right-clicking on a file or folder.

Kernel management

It's now easier than ever to select, stop, and restart a notebook kernel.

Navigate to the kernel sessions manager in the bottom left of the notebooks IDE to interact with each notebook kernel in real time. Use the appropriate buttons to stop and restart the kernel.

Bonus: Terminal updates!

As a bonus for Pro and Growth users, we've moved the terminal into the notebook IDE as a split-screen item!

It's now possible to issue terminal commands without leaving the notebook file itself.

Try it out

Ready to try out the new Gradient Notebooks IDE?

Try creating a new notebook in Gradient or go ahead and fork a project from the ML Showcase.  

Be sure to let us know about your projects don't hesitate to contact support if you have any questions!