AI Testing
& Audit
SOLUTIONS >AI TESTING & AUDIT
The data-driven analysis of PyThia in combination with the high-end expertise of our AI engineers and advisors gives us the capability to test and audit the behavior and trustworthiness of AI-based systems at every stage of their life cycle, whether they are purchased, built or just operated.
Type of project: Either one-off
or with audits at predefined intervals
Identifying complex AI issues at the root
Our experts start an AI Testing & Audit project by first understanding the business (or social) and technical context and then digging deep into the provided dataset(s) and model(s) to perform a comprehensive analysis from different perspectives. Based on the analysis’s results our team works to reveal the root causes of core issues and provide the respective recommendations.
Typically, the AI Testing & Audit service tests AI systems from the perspectives of Bias, Transparency and Robustness characteristics (or quality aspects) as defined by the ISO 29119-11 standard. These analyses provide the insights needed for improving the behavior and trustworthiness of a given AI-based system.
Features
During an audit our team is able to perform a comprehensive set of specialized tests, which provide deep insight into the state of your AI-system and input for improvement or risk management programs.
Root-cause analysis
Mitigating risks related to the behavior of an AI-system requires a proper factual and data-driven analysis that brings all facts to the table. Our expert advisors audit your AI-system to get to the core of any issue. Our fact-based reports will align your organization to address the root causes immediately.
ISO 29119-11 Testing Capabilities
By following the guidelines of the ISO 29119-11 standard for testing AI-based systems, PyThia provides testing and analyses on the Bias, Explainability and Robustness perspectives. These analyses provide the insights for understanding and improving the behavior of any AI-based system.
Model and Data-Agnostic analysis based on our own R&D outcomes
PyThia supports the analysis of any type of AI-model (from a simple rule-based to a deep learning one) and data (i.e. from credit risk datasets to image and video data). In that way we are flexible enough to help organizations across different business domains, from healthcare and high-tech to telecoms, banking and government. It has to be noted here that our analyses capabilities are being fueled by our own R&D whose goal is to either improve the existing or create a potential new state-of-the-art.
Human-in-the-Loop principle
PyThia follows the Human-in-the-Loop principle, thus designed to enable and provide relevant insights to different types of users
Benefits
Our AI Testing & Audit solution follows a proven, fact-based method for enhancing the AI strategy and informed decision-making of an organization as well as their ability to mitigate any risks or mistrust related their AI-based systems. The service provides the following benefits:
Independent, objective advisory, so no strings attached with AI vendors or big tech companies
Our team provides actionable advice and recommendations that are independent, impartial and objective. We have no stake in the outcome and focus only on the facts and identifying those issues that can create problems in the short-term as well as in long-term.
Pragmatic and actionable suggestions for improvements, so no theoretical or out of context advice
Our guidance and recommendations are practical, pragmatic and can lead to measurable improvements in PyThia, our AI testing platform. That means you can start your improvement plan right away alongside our guidance and advisory.
FURTHER READING
Tips on how to secure your ML model
Figuring out the reasons why your ML model might be consistently less accurate in certain classes than others, might help you increase not only its total accuracy but also its adversarial robustness.
In Algorithms We Need To Trust? Not There Yet
Artificial Intelligence is impacting our lives for good, so we need to take a closer look
Fix your recommender system to fix your profitability
How bias in Recommender Systems affects e-commerce, society and eventually your profits