Skip to main content

Log metadata with Neptune

To log metadata, use:

  • log_configs() for single values.
  • log_metrics() for a series of numerical values.

Pass the metadata as a dictionary {key: value} with

  • key: path to where the metadata should be stored in the run.
  • value: a numerical, string, datetime, or Boolean value. If logging metrics, a float or int value to append to the series.

In the attribute path, each forward slash (/) nests the attribute under a namespace. Use namespaces to structure the metadata into meaningful categories.

Example
from neptune_scale import Run

run = Run(
    run_id=...
    experiment_name=...
)

run.log_configs(
    {
        "parameters/use_preprocessing": True,
        "parameters/learning_rate": 0.001,
        "parameters/batch_size": 64,
        "parameters/optimizer": "Adam",
    }
)

for step in epoch:
    # your training loop
    run.log_metrics(
        data={
            "train/accuracy": 0.87,
            "train/loss": 0.14,
        }
        step=step,
    )

In the above example, the attributes are organized into two namespaces:

  • parameters, containing all the config attributes
  • train, containing all the metric attributes

A sensible structure makes it more convenient to work with the metadata later.

Related
  • Log configs – save single values and display them in the experiments table, custom dashboards, or reports.
  • Log metrics – log metrics and losses as series of values and visualize them as charts.
  • Attribute types – the type determines how you can display and interact with the attribute in Neptune.

Date and time​

You can track dates and times by assigning a Python datetime object to a attribute of your choice.

Logging custom training end time
from datetime import datetime

run = Run(...)
run.log_configs(
    {"train/end": datetime.now()}
)
Logging other time-related metadata
run.log_configs(
    {"dataset/creation_date": datetime.fromisoformat("1998-11-01")}
)
note

Any datetime values that don't have the tzinfo attribute set are assumed to be in the local timezone.

Script execution command​

When executing a script through the command line, you can use the Python sys module to log the command.

Add the following to the Python script where a Neptune run is initialized:

import sys

run = Run(...)
run.log_configs(
    {
        "execution_command": f"python {sys.argv[0]} {' '.join(sys.argv[1:])}"
    }
)

The above example uses sys.argv to parse the command line arguments and add them to the execution command and script name. The resulting string is then logged to Neptune.

If you use the following command to execute the script, the entire command is logged under a String attribute named execution_command:

python test.py --fname=Jackie --lname=Neptuner

Text​

Tags​

To apply tags to a run, call add_tags() and pass a collection of strings:

run = Run(...)
run.add_tags(tags=["tag1", "tag2", "tag3"])
Related

Run description​

Adding a description is one way to annotate or comment on the experiment. You can add the description when creating the run or later through the web app.

Run information displayed in the Neptune app

To add a run description in the app:

  1. Next to the run ID, click the menu () and select Run information.
  2. Edit the Description attribute.
  3. Save your changes.

The description is stored in the sys/description attribute.

To edit the run description through API, use:

run.log_configs({"sys/description": "This run was special ⭐️"})
tip

To display descriptions in the experiments table, add the sys/description attribute as a column.

Small piece of text​

You can assign any single string to a attribute:

run.log_configs({"my_text": "Text I keep track of, like query or tokenized word"})
Related