Kubeflow Pipeline Metadata UI¶
For KFP v1, metrics and metadata for ui can be generated within the kfp components by
creating mlpipeline-metrics.json
and mlpipeline-ui-metadata.json
argo artifacts.
kfx.vis.kfp_metrics
and kfx.vis.kfp_ui_metadata
are functions to help compose
the content of mlpipeline-metrics.json
and mlpipeline-ui-metadata.json
respectively.
kfx.vis
also provides helper functions to create the different sort of UI artifacts.
NOTE:
Moving forward, KFP v2 will not be supporting this way of generating metadata ui artifacts.
import functools
import kfp.components
# install kfx
kfx_component = functools.partial(kfp.components.func_to_container_op, packages_to_install=["kfx"])
@kfx_component
def some_op(
# mlpipeline_metrics is a path - i.e. open(mlpipeline_metrics, "w")
mlpipeline_metrics: kfp.components.OutputPath(str),
# mlpipeline_ui_metadata is a FileLike obj - i.e. mlpipeline_ui_metadata.write("something")
mlpipeline_ui_metadata: kfp.components.OutputTextFile(str),
):
"kfp operator that provides metrics and metadata for visualizations."
# import inside kfp task
import kfx.vis
import kfx.vis.vega
# output metrics to mlpipeline_metrics path
kfx.vis.kfp_metrics([
# render as percent
kfx.vis.kfp_metric("recall-score", 0.9, percent=true),
# override metric format with custom value
kfx.vis.kfp_metric(name="percision-score", value=0.8, metric_format="PERCENTAGE"),
# render raw score
kfx.vis.kfp_metric("raw-score", 123.45),
]).write_to(mlpipeline_metrics)
# output visualization metadata to mlpipeline_ui_metadata obj
kfx.vis.kfp_ui_metadata(
[
# creates a confusion matrix vis
kfx.vis.confusion_matrix(
source="gs://your_project/your_bucket/your_cm_file",
labels=["True", "False"],
),
# creates a markdown with inline source
kfx.vis.markdown(
"# Inline Markdown: [A link](https://www.kubeflow.org/)",
storage="inline",
),
# creates a markdown with a remote source
kfx.vis.markdown(
"gs://your_project/your_bucket/your_markdown_file",
),
# creates a ROC curve with a remote source
kfx.vis.roc(
"gs://your_project/your_bucket/your_roc_file",
),
# creates a Table with a remote source
kfx.vis.table(
"gs://your_project/your_bucket/your_csv_file",
header=["col1", "col2"],
),
# creates a tensorboard viewer
kfx.vis.tensorboard(
"gs://your_project/your_bucket/logs/*",
),
# creates a custom web app from a remote html file
kfx.vis.web_app(
"gs://your_project/your_bucket/your_html_file",
),
# creates a Vega-Lite vis as a web app
kfx.vis.vega.vega_web_app(spec={
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"description": "A simple bar chart with embedded data.",
"data": {
"values": [
{"a": "A", "b": 28}, {"a": "B", "b": 55}, {"a": "C", "b": 43},
{"a": "D", "b": 91}, {"a": "E", "b": 81}, {"a": "F", "b": 53},
{"a": "G", "b": 19}, {"a": "H", "b": 87}, {"a": "I", "b": 52}
]
},
"mark": "bar",
"encoding": {
"x": {"field": "a", "type": "ordinal"},
"y": {"field": "b", "type": "quantitative"}
}
})
]
).write_to(mlpipeline_ui_metadata)
kfx.vis._helpers.kfp_metrics(metrics)
¶
Describes a list of kubeflow pipeline metrics.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
metrics |
Union[Iterable[kfx.vis.models.KfpMetric], Iterable[dict], Iterable[Union[kfx.vis.models.KfpMetric, dict]]] |
Any iterable of dict or KfpMetric. |
required |
Returns:
Type | Description |
---|---|
KfpMetrics |
KfpMetrics: an instance of KfpMetrics which can be stream to the output. |
kfx.vis._helpers.kfp_ui_metadata(outputs, version=1)
¶
Helper function to create a KfpUiMetadata object.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
outputs |
List[Union[kfx.vis.models.ConfusionMatrix, kfx.vis.models.Roc, kfx.vis.models.Markdown, kfx.vis.models.Table, kfx.vis.models.Tensorboard, kfx.vis.models.WebApp, dict]] |
List of KfpVis objects. |
required |
version |
Union[int, str] |
Schema version. Defaults to 1. |
1 |
Returns:
Type | Description |
---|---|
KfpUiMetadata |
KfpUiMetadata: pydantic data object. |
kfx.vis._helpers.confusion_matrix(source, labels, artifact_format='csv', **kwargs)
¶
Helper function to create a KfpUiMetadata ConfusionMatrix object.
The source artifact must be a CSV with the following 3 columns: - target - predicted - count
Parameters:
Name | Type | Description | Default |
---|---|---|---|
source |
Union[str, kfx.dsl._artifact_location.KfpArtifact] |
Full path to the data artifact. |
required |
labels |
List[str] |
Names of the classes to be plotted on the x and y axes. |
required |
artifact_format |
Union[kfx.vis.enums.KfpArtifactDataFormat, str] |
Data format for the artifact. Defaults to "csv". |
'csv' |
Returns:
Type | Description |
---|---|
ConfusionMatrix |
ConfusionMatrix: pydantic data object. |
kfx.vis._helpers.markdown(source, storage=None, **kwargs)
¶
Helper function to create a KfpUiMetadata Markdown object.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
source |
Union[str, kfx.dsl._artifact_location.KfpArtifact] |
Full path to the markdown or the actual markdown text. |
required |
storage |
Union[kfx.vis.enums.KfpStorage, str] |
Set "inline" if source has the actual markdown text. Defaults to None. |
None |
Returns:
Type | Description |
---|---|
Markdown |
Markdown: pydantic data object. |
kfx.vis._helpers.roc(source, artifact_format='csv', **kwargs)
¶
Helper function to create a KfpUiMetadata Roc object.
The source artifact must be a CSV with the following 3 columns: - fpr (false positive rate) - tpr (true positive rate) - thresholds
Parameters:
Name | Type | Description | Default |
---|---|---|---|
source |
Union[str, kfx.dsl._artifact_location.KfpArtifact] |
Full path to roc data. |
required |
artifact_format |
Union[kfx.vis.enums.KfpArtifactDataFormat, str] |
Data format for the artifact. Defaults to "csv". |
'csv' |
Returns:
Type | Description |
---|---|
Roc |
Roc: pydantic data object. |
kfx.vis._helpers.table(source, header, artifact_format='csv', **kwargs)
¶
Helper function to create a KfpUiMetadata Table object.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
source |
Union[str, kfx.dsl._artifact_location.KfpArtifact] |
Full path to the data. |
required |
header |
List[str] |
Headers to use for the table. |
required |
artifact_format |
Union[kfx.vis.enums.KfpArtifactDataFormat, str] |
Data format for the artifact. Defaults to "csv". |
'csv' |
Returns:
Type | Description |
---|---|
Table |
Table: pydantic data object. |
kfx.vis._helpers.tensorboard(source, **kwargs)
¶
Helper function to create a KfpUiMetadata Tensorboard object.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
source |
Union[str, kfx.dsl._artifact_location.KfpArtifact] |
The full path to the tensorboard logs. Supports * wildcards. |
required |
Returns:
Type | Description |
---|---|
Tensorboard |
Tensorboard: pydantic data object. |
kfx.vis._helpers.web_app(source, **kwargs)
¶
Helper function to create a KfpUiMetadata WebApp object.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
source |
Union[str, kfx.dsl._artifact_location.KfpArtifact] |
The full path to the html content or inlined html. |
required |
Returns:
Type | Description |
---|---|
WebApp |
WebApp: pydantic data object. |
kfx.vis.vega.vega_web_app(spec, opts=None, title='Generated by kfx.vis', vega=5, vega_lite=4)
¶
Provides the metadata needed for kubeflow pipeline UI to render a Vega <https://vega.github.io/>
or Vega-Lite <https://vega.github.io/vega-lite/>
vis in as a custom web app.
This web app uses vega embed <https://github.com/vega/vega-embed>
_ to render the vis.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
spec |
dict |
Vega or Vega-Lite spec as a dict. |
required |
opts |
dict |
Options to pass to vega-embed. Defaults to None. |
None |
title |
str |
Title for the web app. Defaults to "Generated by kfx.vis". |
'Generated by kfx.vis' |
vega |
int |
Version of Vega to use. Defaults to 5. |
5 |
vega_lite |
int |
Version of Vega-Lite to use. Defaults to 4. |
4 |
Returns:
Type | Description |
---|---|
WebApp |
kfx.vis.models.WebApp: pydantic data object describing a Vega/Vega-Lite web app. |