Neptune objects overview#
Neptune objects refer to entities that can contain metadata: runs, models, model versions, and projects.
Object type | Represents | Description | Section in app | Demo |
---|---|---|---|---|
Run |
A tracked experiment | Object containing model-building metadata. Typically, you create a run every time you execute a script that does model training, retraining, or inference. | Runs | View example |
Model |
An ML model | Collection of metadata common to all versions of a model you're working on. | Models | View example |
ModelVersion |
A version of an ML model | Collection of metadata specific to a version of a model. | Models → Specific model | View example |
Project |
A Neptune project (an ML task you're solving) | An object that can be used to log and query project-level metadata. | Project metadata | View example |
How it works in the code#
The below code creates a run in your Neptune project, as specified by the NEPTUNE_PROJECT
environment variable. Run
objects can contain any metadata you want to track, plus some automatically logged system metrics.
import neptune
run = neptune.init_run()
run["params"] = {"lr": 0.01, "batch_size": 64, "activation": "ReLU"}
run["data/sample"].upload("datasets/sample.csv")
...
run.stop()
Once the run is stopped, you get a link to the run in the Neptune app.
Register a model once, then create as many versions as you need. As with runs, you can store any metadata you like, in a structure of your choosing.
import neptune
model = neptune.init_model(key="FOREST")
model["signature"].upload("model_signature.json")
model_version = neptune.init_model_version(model="CLS-FOREST") # (1)!
model_version["model/binary"].upload("model.pt")
model_version.change_stage("production")
- Pass the ID of an existing model
A Neptune project can itself be treated as a Neptune object, which lets you track and query metadata that applies on project level.
You can use the same logging methods as for runs and model objects.
import neptune
project = neptune.init_project(project="ml-team/classification")
project["dataset/v0.1"].track_files("s3://datasets/images")
project["dataset/sample"].upload("s3://datasets/sample.csv")
This way, you can have a single source of truth for artifact versions or other metadata that's shared between all runs and models in the project.
Related
- Create a run
- API reference ≫
Run
- Model registry overview
- API reference ≫
Model
object – general model metadata - API reference ≫
ModelVersion
object – version-specific model metadata - Project metadata
- API reference ≫
Project
Breakdown of a Neptune object#
object
|-- namespace
|-- field - Float
|-- field - FileSeries
|-- field - String
|-- namespace
|-- namespace
|-- field - String
|-- field - FloatSeries
-
Namespace: A "folder" inside a Neptune object.
The following namespaces are always created and, by default, populated with some basic metadata:
monitoring
,source_code
, andsys
. -
Field: Location inside a namespace where a piece of metadata is stored. Each field is of a certain type, which depends on the type of metadata and how it was logged.
If you think of the data structure like a tree, fields are the "leaves". A field can be flat (stored at the root of the object) or nested (stored under one or more namespaces).
Similarities and differences between Neptune objects#
In your code, you work with the objects in much the same way:
-
You initialize the object with its own
init()
function, either to create a new object or log to an existing one.You can resume a connection to an object any number of times.
-
It's up to you to define what types of metadata to log and in what structure.
- When you're done logging, you should stop the connection to the object.
The main differences are:
- The available methods and properties. For example, managing the stage is only available for model versions.
- Model and project objects do not monitor things in the background, like runs do.
- Run and model version objects are designed for multiple trials and comparison, so their Neptune IDs include a counter.