code4thought

TRUSTWORTHY AI

SOFTWARE QUALITY

TRUSTWORTHY AI

AI Testing
& Audit

SOLUTIONS >AI TESTING & AUDIT
The data-driven analysis of PyThia in combination with the high-end expertise of our AI engineers and advisors gives us the capability to test and audit the behavior and trustworthiness of AI-based systems at every stage of their life cycle, whether they are purchased, built or just operated.

Type of project: Either one-off
or with audits at predefined intervals

Identifying complex AI issues at the root
Our experts start an AI Testing & Audit project by first understanding the business (or social) and technical context and then digging deep into the provided dataset(s) and model(s) to perform a comprehensive analysis from different perspectives. Based on the analysis’s results our team works to reveal the root causes of core issues and provide the respective recommendations.
Typically, the AI Testing & Audit service tests AI systems from the perspectives of Bias, Transparency and Robustness characteristics (or quality aspects) as defined by the ISO 29119-11 standard. These analyses provide the insights needed for improving the behavior and trustworthiness of a given AI-based system.
Ensuring compliance with legislation
We have the flexibility to adopt our AI testing & auditing processes to specific requirements, generated either by clients’ needs and/or aiming compliance with existing and upcoming regulations such as the EU AI Act, UK’s frameworκ for AI Regulation and the Blueprint For An AI Bill Of Rights in the USA. Our solution NYC AI Bias Audit is an indicative example.
Features
During an audit our team is able to perform a comprehensive set of specialized tests, which provide deep insight into the state of your AI-system and input for improvement or risk management programs.
Root-cause analysis
Mitigating risks related to the behavior of an AI-system requires a proper factual and data-driven analysis that brings all facts to the table. Our expert advisors audit your AI-system to get to the core of any issue. Our fact-based reports will align your organization to address the root causes immediately.
ISO 29119-11 Testing Capabilities
By following the guidelines of the ISO 29119-11 standard for testing AI-based systems, PyThia provides testing and analyses on the Bias, Explainability and Robustness perspectives. These analyses provide the insights for understanding and improving the behavior of any AI-based system.
Model and Data-Agnostic analysis based on our own R&D outcomes
PyThia supports the analysis of any type of AI-model (from a simple rule-based to a deep learning one) and data (i.e. from credit risk datasets to image and video data). In that way we are flexible enough to help organizations across different business domains, from healthcare and high-tech to telecoms, banking and government. It has to be noted here that our analyses capabilities are being fueled by our own R&D whose goal is to either improve the existing or create a potential new state-of-the-art.
Human-in-the-Loop principle

PyThia follows the Human-in-the-Loop principle, thus designed to enable and provide relevant insights to different types of users

Benefits
Our AI Testing & Audit solution follows a proven, fact-based method for enhancing the AI strategy and informed decision-making of an organization as well as their ability to mitigate any risks or mistrust related their AI-based systems. The service provides the following benefits:
Independent, objective advisory, so no strings attached with AI vendors or big tech companies
Our team provides actionable advice and recommendations that are independent, impartial and objective. We have no stake in the outcome and focus only on the facts and identifying those issues that can create problems in the short-term as well as in long-term.
Pragmatic and actionable suggestions for improvements, so no theoretical or out of context advice
Our guidance and recommendations are practical, pragmatic and can lead to measurable improvements in PyThia, our AI testing platform. That means you can start your improvement plan right away alongside our guidance and advisory.

FURTHER READING