Skip to content

Kubeflow Pipeline Metadata UI

For KFP v1, metrics and metadata for ui can be generated within the kfp components by creating mlpipeline-metrics.json and mlpipeline-ui-metadata.json argo artifacts.

kfx.vis.kfp_metrics and kfx.vis.kfp_ui_metadata are functions to help compose the content of mlpipeline-metrics.json and mlpipeline-ui-metadata.json respectively.

kfx.vis also provides helper functions to create the different sort of UI artifacts.

NOTE:

Moving forward, KFP v2 will not be supporting this way of generating metadata ui artifacts.

See https://github.com/kubeflow/pipelines/issues/5675

import functools

import kfp.components


# install kfx
kfx_component = functools.partial(kfp.components.func_to_container_op, packages_to_install=["kfx"])


@kfx_component
def some_op(
    # mlpipeline_metrics is a path - i.e. open(mlpipeline_metrics, "w")
    mlpipeline_metrics: kfp.components.OutputPath(str),
    # mlpipeline_ui_metadata is a FileLike obj - i.e. mlpipeline_ui_metadata.write("something")
    mlpipeline_ui_metadata: kfp.components.OutputTextFile(str),
):
    "kfp operator that provides metrics and metadata for visualizations."

    # import inside kfp task
    import kfx.vis
    import kfx.vis.vega

    # output metrics to mlpipeline_metrics path
    kfx.vis.kfp_metrics([
        # render as percent
        kfx.vis.kfp_metric("recall-score", 0.9, percent=true),
        # override metric format with custom value
        kfx.vis.kfp_metric(name="percision-score", value=0.8, metric_format="PERCENTAGE"),
        # render raw score
        kfx.vis.kfp_metric("raw-score", 123.45),
    ]).write_to(mlpipeline_metrics)

    # output visualization metadata to mlpipeline_ui_metadata obj
    kfx.vis.kfp_ui_metadata(
        [
            # creates a confusion matrix vis
            kfx.vis.confusion_matrix(
                source="gs://your_project/your_bucket/your_cm_file",
                labels=["True", "False"],
            ),
            # creates a markdown with inline source
            kfx.vis.markdown(
                "# Inline Markdown: [A link](https://www.kubeflow.org/)",
                storage="inline",
            ),
            # creates a markdown with a remote source
            kfx.vis.markdown(
                "gs://your_project/your_bucket/your_markdown_file",
            ),
            # creates a ROC curve with a remote source
            kfx.vis.roc(
                "gs://your_project/your_bucket/your_roc_file",
            ),
            # creates a Table with a remote source
            kfx.vis.table(
                "gs://your_project/your_bucket/your_csv_file",
                header=["col1", "col2"],
            ),
            # creates a tensorboard viewer
            kfx.vis.tensorboard(
                "gs://your_project/your_bucket/logs/*",
            ),
            # creates a custom web app from a remote html file
            kfx.vis.web_app(
                "gs://your_project/your_bucket/your_html_file",
            ),
            # creates a Vega-Lite vis as a web app
            kfx.vis.vega.vega_web_app(spec={
                "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
                "description": "A simple bar chart with embedded data.",
                "data": {
                    "values": [
                        {"a": "A", "b": 28}, {"a": "B", "b": 55}, {"a": "C", "b": 43},
                        {"a": "D", "b": 91}, {"a": "E", "b": 81}, {"a": "F", "b": 53},
                        {"a": "G", "b": 19}, {"a": "H", "b": 87}, {"a": "I", "b": 52}
                    ]
                },
                "mark": "bar",
                "encoding": {
                    "x": {"field": "a", "type": "ordinal"},
                    "y": {"field": "b", "type": "quantitative"}
                }
            })
        ]
    ).write_to(mlpipeline_ui_metadata)

kfx.vis._helpers.kfp_metrics(metrics)

Describes a list of kubeflow pipeline metrics.

Parameters:

Name Type Description Default
metrics Union[Iterable[kfx.vis.models.KfpMetric], Iterable[dict], Iterable[Union[kfx.vis.models.KfpMetric, dict]]]

Any iterable of dict or KfpMetric.

required

Returns:

Type Description
KfpMetrics

KfpMetrics: an instance of KfpMetrics which can be stream to the output.

kfx.vis._helpers.kfp_ui_metadata(outputs, version=1)

Helper function to create a KfpUiMetadata object.

Parameters:

Name Type Description Default
outputs List[Union[kfx.vis.models.ConfusionMatrix, kfx.vis.models.Roc, kfx.vis.models.Markdown, kfx.vis.models.Table, kfx.vis.models.Tensorboard, kfx.vis.models.WebApp, dict]]

List of KfpVis objects.

required
version Union[int, str]

Schema version. Defaults to 1.

1

Returns:

Type Description
KfpUiMetadata

KfpUiMetadata: pydantic data object.

kfx.vis._helpers.confusion_matrix(source, labels, artifact_format='csv', **kwargs)

Helper function to create a KfpUiMetadata ConfusionMatrix object.

The source artifact must be a CSV with the following 3 columns: - target - predicted - count

Parameters:

Name Type Description Default
source Union[str, kfx.dsl._artifact_location.KfpArtifact]

Full path to the data artifact.

required
labels List[str]

Names of the classes to be plotted on the x and y axes.

required
artifact_format Union[kfx.vis.enums.KfpArtifactDataFormat, str]

Data format for the artifact. Defaults to "csv".

'csv'

Returns:

Type Description
ConfusionMatrix

ConfusionMatrix: pydantic data object.

kfx.vis._helpers.markdown(source, storage=None, **kwargs)

Helper function to create a KfpUiMetadata Markdown object.

Parameters:

Name Type Description Default
source Union[str, kfx.dsl._artifact_location.KfpArtifact]

Full path to the markdown or the actual markdown text.

required
storage Union[kfx.vis.enums.KfpStorage, str]

Set "inline" if source has the actual markdown text. Defaults to None.

None

Returns:

Type Description
Markdown

Markdown: pydantic data object.

kfx.vis._helpers.roc(source, artifact_format='csv', **kwargs)

Helper function to create a KfpUiMetadata Roc object.

The source artifact must be a CSV with the following 3 columns: - fpr (false positive rate) - tpr (true positive rate) - thresholds

Parameters:

Name Type Description Default
source Union[str, kfx.dsl._artifact_location.KfpArtifact]

Full path to roc data.

required
artifact_format Union[kfx.vis.enums.KfpArtifactDataFormat, str]

Data format for the artifact. Defaults to "csv".

'csv'

Returns:

Type Description
Roc

Roc: pydantic data object.

kfx.vis._helpers.table(source, header, artifact_format='csv', **kwargs)

Helper function to create a KfpUiMetadata Table object.

Parameters:

Name Type Description Default
source Union[str, kfx.dsl._artifact_location.KfpArtifact]

Full path to the data.

required
header List[str]

Headers to use for the table.

required
artifact_format Union[kfx.vis.enums.KfpArtifactDataFormat, str]

Data format for the artifact. Defaults to "csv".

'csv'

Returns:

Type Description
Table

Table: pydantic data object.

kfx.vis._helpers.tensorboard(source, **kwargs)

Helper function to create a KfpUiMetadata Tensorboard object.

Parameters:

Name Type Description Default
source Union[str, kfx.dsl._artifact_location.KfpArtifact]

The full path to the tensorboard logs. Supports * wildcards.

required

Returns:

Type Description
Tensorboard

Tensorboard: pydantic data object.

kfx.vis._helpers.web_app(source, **kwargs)

Helper function to create a KfpUiMetadata WebApp object.

Parameters:

Name Type Description Default
source Union[str, kfx.dsl._artifact_location.KfpArtifact]

The full path to the html content or inlined html.

required

Returns:

Type Description
WebApp

WebApp: pydantic data object.

kfx.vis.vega.vega_web_app(spec, opts=None, title='Generated by kfx.vis', vega=5, vega_lite=4)

Provides the metadata needed for kubeflow pipeline UI to render a Vega <https://vega.github.io/> or Vega-Lite <https://vega.github.io/vega-lite/> vis in as a custom web app.

This web app uses vega embed <https://github.com/vega/vega-embed>_ to render the vis.

Parameters:

Name Type Description Default
spec dict

Vega or Vega-Lite spec as a dict.

required
opts dict

Options to pass to vega-embed. Defaults to None.

None
title str

Title for the web app. Defaults to "Generated by kfx.vis".

'Generated by kfx.vis'
vega int

Version of Vega to use. Defaults to 5.

5
vega_lite int

Version of Vega-Lite to use. Defaults to 4.

4

Returns:

Type Description
WebApp

kfx.vis.models.WebApp: pydantic data object describing a Vega/Vega-Lite web app.


Last update: June 9, 2021