Blog: What’s new in Watson OpenScale: beta experience with advanced metrics, Fast Path tutorial
With Watson OpenScale, we envision helping users understand the impact that AI has on their business outcomes. The new Watson OpenScale beta experience presents the foundational metrics which are important to be collected in order to find the impact on an AI application’s KPIs. This new experience which can be turned on for instances provisioned in IBM Cloud; it gives you access to new quality monitors, to understand model performance. In addition to accuracy, the beta experience allows you to track a variety of new quality monitors to meet your different needs, including precision, recall, F1-measure, and more. These metrics are chosen based on the type of algorithm that you deploy in your model. We have also introduced the capability to define and monitor custom metrics beyond these out-of-the-box options. All of the metrics can be visualized and monitored on the OpenScale dashboards.
Along with the expansion of performance metrics available through Watson OpenScale, the beta experience also provides a deeper look into the payload analytics associated with your models. We have introduced a chart builder that allows you to create custom visualizations, so you can better understand model predictions and inputs at runtime. This chart builder provides an ability to view the output of the model’s prediction against the features or data ranges that a business considers important. It helps uncover new trends in the data which may prompt the business and data science teams to consider changes to the AI model. E.g. in the credit risk example below, you can see the split in predicted classes for different ranges of the attribute Credit History. You can also see how confident the model is, when predicting for these ranges of Credit History.
All of these new monitoring features have been wrapped in an updated UI that brings these data to life with beautiful, simplified charts. With more ways to understand model performance the model’s impact on new business data coming in, you will be able to address issues as they arise more quickly and effectively, especially in changing business situations.
Plus, we’ve made it easier than ever to get started with Watson OpenScale with a new Fast Path tutorial that will automatically connect your instance to a sample model and configure your monitors. In less time that it takes to make a cup of coffee, you will be monitoring a credit risk model and exploring the value that Watson OpenScale can bring your AI governance practice. Simply create a new instance of Watson OpenScale on IBM Cloud and try out the fastpath.
If you are interested in becoming a sponsor user of Watson OpenScale, to provide feedback and help us determine the future direction of this product, please let us know.