Skip to main content
App version: 3.4.8

Log from different processes

You can log metadata to the same run from multiple separate processes at once.

Use the custom run ID to access the same run in each process:

Process A
run = Run(
    run_id="seagull-s68kj78",
)

for step in epoch:
    # some training loop
    run.log_metrics(
        data={"train/loss": 0.14},
        step=step,
    )
Process B
run = Run(
    run_id="seagull-s68kj78",
    resume=True,
)

for step in epoch:
    # some validation loop
    run.log_metrics(
        data={"val/loss": 0.16},
        step=step,
    )

Metrics​

In the above example, the two processes are logging metrics separately, to different attribute paths: train/loss and val/loss, respectively. You can log the same metric across processes, as long as steps aren't logged out of order within the same attribute. For details, see Log metrics: Setting step values.

note

If using resume=True with log_metrics(), Neptune checks that a run identified by the run_id argument already exists. If such a run doesn't exist, the NeptuneRunNotFound error is printed when Neptune attempts to synchronize the data with the server, and no data is logged.

If you omit the resume parameter or set it to False, and Neptune can't find a run with the specified ID in the project, a new run with the specified ID is created. In this case, no data is lost, but it ends up distributed across multiple runs.

Configs​

For configs or other single values logged with log_configs(), logging a value to an existing attribute path overwrites the previous value.

Tags​

Adding tags with add_tags() appends the provided tags to the existing tagset.

Related