What is Explainable Artificial Intelligence (XAI)?

Main

• Explainable AI (XAI) is a branch of AI research that seeks to create systems and models that can explain their actions and make decisions in a way that humans can understand.

• One of the main problems in training modern artificial intelligences is the “black box”: systems can give accurate answers and perform complex tasks, but it is often difficult to understand how they arrived at these results.

• XAI can be useful in areas where a high level of explainability of decisions is required, such as medicine, finance, and law.

How to define AI?

Artificial intelligence can be complex and opaque to understand. Because of this, a request for explainable artificial intelligence (Explainable AI, XAI) logically arises.

To understand what XAI is, we first need to take a closer look at AI in general. Artificial intelligence technologies are a vast field that continues to grow rapidly. However, there is no universal definition for it yet.

In 2021, the European Commission submitted a regulatory proposal that would provide a legally binding definition of AI. The document says that artificial intelligence includes systems that draw conclusions, forecasts, recommendations or make decisions that affect their environment.

According to lawyer and AI researcher Jacob Turner, artificial intelligence can also be defined as “the ability of an unnatural entity to make decisions based on an evaluation process.” Combining the definitions of the European Commission and Turner, we can say that artificial intelligence systems are able to “learn” and influence their environment. Artificial intelligence is not limited to software, it can manifest itself in various forms, including robotics.

What is a “black box” in AI?

Artificial intelligence makes “decisions” or creates outputs based on inputs and algorithms. Thanks to the ability to learn and apply various techniques and approaches, AI is able to do this without direct human intervention. This leads to the fact that AI systems are often perceived as a “black box”.

The “black box” in this case refers to the difficulty of understanding and controlling the decisions and actions that AI systems and algorithms produce. This creates problems with transparency and accountability, which in turn has various legal and regulatory implications.

How does XAI solve this problem?

The concept of explainable artificial intelligence emerged as a response to the black box problem. XAI is an approach aimed at creating AI systems whose results can be explained in human-understandable terms. The main goal of explainable artificial intelligence is to make decision making in AI systems transparent and accessible.

We can highlight the following factors that make XAI a significant component in the development and use of AI:

• Responsibility. If an AI system makes a decision that is important to a person (for example, denying a loan or making a medical diagnosis), people need to understand how and why this decision was made. The XAI concept will increase the transparency and responsibility of such processes, rid society of fears associated with the use of AI technologies.
• Confidence. People are more likely to trust systems they understand. If an AI system can explain its decisions in an accessible way, people will be more willing to accept its decisions.
• Model improvement. If we can understand how the AI ​​system makes its decisions, we can use that information to improve the model. This will effectively detect and eliminate bias, make the system more accurate, reliable and ethical.
• Legal Compliance. Some jurisdictions, such as the European Union with the introduction of the General Data Protection Regulation (GDPR), require organizations to be able to explain decisions made using automated systems.

Transparency and explainability may give way to other interests such as profit or competitiveness. This highlights the need to strike the right balance between innovation and ethical considerations in the development and application of artificial intelligence.

Increasing confidence in public and private AI systems is essential. It encourages developers to be more responsible and ensures that their models do not spread discriminatory ideas. In addition, it contributes to the prevention of illegal use of databases.

XAI plays a key role in this process. Explainability means transparency of the key factors and parameters that determine AI decisions. While complete explainability may not be achievable due to the inherent complexity of AI systems, it is possible to set certain parameters and values. This is what artificial intelligence does.

What are some XAI examples?

Various machine learning techniques can serve as examples of explainable artificial intelligence. They increase the explainability of AI models through different approaches:

• decision trees. Provides a clear visual representation of the AI ​​decision making process.
• Rule based systems. Define algorithmic rules in a format that humans can understand. They may be less flexible in terms of interpretation.
• Bayesian networks. Probabilistic models that show cause and effect relationships and uncertainties.
• Linear models and similar techniques in neural networks. These models show how each input affects the output.

Various approaches are used to achieve XAI, including visualization, natural language explanations, and interactive interfaces. Interactive interfaces, for example, allow users to explore how model predictions change as input changes.

Visual tools such as heat maps and trees

What are the disadvantages of XAI?

Explainable artificial intelligence has several limitations, some of them related to its application:

• Complexity of development. Large teams of engineers can work on algorithms for a long time. This makes it difficult to understand the entire development process and the principles embedded in AI systems.
• The ambiguity of the term “explainability”. This is a broad concept that can lead to different interpretations when implementing XAI. When analyzing the key parameters and factors in AI, questions arise: what exactly is considered “transparent” or “explainable” and what are the limits of this explainability?
• The rapid development of AI. Artificial intelligence is developing exponentially. Combined with unsupervised systems and deep learning, it could theoretically reach the level of general intelligence. This opens the door to new ideas and innovations, but it also introduces additional complexities when implementing XAI.

What are the prospects for XAI?

Consider study about “generative agents”, whose authors integrated AI language models with interactive agents. During the experiment, a virtual sandbox was created, which is a small town with twenty-five virtual “residents”. Communicating in natural language, they demonstrated realistic individual and social behavior. So, one agent “wanted” to organize a party, after which the agents began to send out invitations on their own.

The word “self” is extremely important here. If AI systems exhibit behavior that is difficult to trace back to individual components, it can have consequences that are difficult to predict.

XAI is able to prevent or at least mitigate some of the risks of using AI. It is important to remember that the ultimate responsibility for AI-based decisions and actions lies with humans, even if not all AI decisions can be explained.

The material was prepared with the participation of language models developed by OpenAI. The information presented here is partly based on machine learning and not real experience or empirical research.

Main

• Explainable AI (XAI) is a branch of AI research that seeks to create systems and models that can explain their actions and make decisions in a way that humans can understand.

• One of the main problems in training modern artificial intelligences is the “black box”: systems can give accurate answers and perform complex tasks, but it is often difficult to understand how they arrived at these results.

• XAI can be useful in areas where a high level of explainability of decisions is required, such as medicine, finance, and law.

How to define AI?

Artificial intelligence can be complex and opaque to understand. Because of this, a request for explainable artificial intelligence (Explainable AI, XAI) logically arises.

To understand what XAI is, we first need to take a closer look at AI in general. Artificial intelligence technologies are a vast field that continues to grow rapidly. However, there is no universal definition for it yet.

In 2021, the European Commission submitted a regulatory proposal that would provide a legally binding definition of AI. The document says that artificial intelligence includes systems that draw conclusions, forecasts, recommendations or make decisions that affect their environment.

According to lawyer and AI researcher Jacob Turner, artificial intelligence can also be defined as “the ability of an unnatural entity to make decisions based on an evaluation process.” Combining the definitions of the European Commission and Turner, we can say that artificial intelligence systems are able to “learn” and influence their environment. Artificial intelligence is not limited to software, it can manifest itself in various forms, including robotics.

What is a “black box” in AI?

Artificial intelligence makes “decisions” or creates outputs based on inputs and algorithms. Thanks to the ability to learn and apply various techniques and approaches, AI is able to do this without direct human intervention. This leads to the fact that AI systems are often perceived as a “black box”.

The “black box” in this case refers to the difficulty of understanding and controlling the decisions and actions that AI systems and algorithms produce. This creates problems with transparency and accountability, which in turn has various legal and regulatory implications.

How does XAI solve this problem?

The concept of explainable artificial intelligence emerged as a response to the black box problem. XAI is an approach aimed at creating AI systems whose results can be explained in human-understandable terms. The main goal of explainable artificial intelligence is to make decision making in AI systems transparent and accessible.

We can highlight the following factors that make XAI a significant component in the development and use of AI:

• Responsibility. If an AI system makes a decision that is important to a person (for example, denying a loan or making a medical diagnosis), people need to understand how and why this decision was made. The XAI concept will increase the transparency and responsibility of such processes, rid society of fears associated with the use of AI technologies.
• Confidence. People are more likely to trust systems they understand. If an AI system can explain its decisions in an accessible way, people will be more willing to accept its decisions.
• Model improvement. If we can understand how the AI ​​system makes its decisions, we can use that information to improve the model. This will effectively detect and eliminate bias, make the system more accurate, reliable and ethical.
• Legal Compliance. Some jurisdictions, such as the European Union with the introduction of the General Data Protection Regulation (GDPR), require organizations to be able to explain decisions made using automated systems.

Transparency and explainability may give way to other interests such as profit or competitiveness. This highlights the need to strike the right balance between innovation and ethical considerations in the development and application of artificial intelligence.

Increasing confidence in public and private AI systems is essential. It encourages developers to be more responsible and ensures that their models do not spread discriminatory ideas. In addition, it contributes to the prevention of illegal use of databases.

XAI plays a key role in this process. Explainability means transparency of the key factors and parameters that determine AI decisions. While complete explainability may not be achievable due to the inherent complexity of AI systems, it is possible to set certain parameters and values. This is what artificial intelligence does.

What are some XAI examples?

Various machine learning techniques can serve as examples of explainable artificial intelligence. They increase the explainability of AI models through different approaches:

• decision trees. Provides a clear visual representation of the AI ​​decision making process.
• Rule based systems. Define algorithmic rules in a format that humans can understand. They may be less flexible in terms of interpretation.
• Bayesian networks. Probabilistic models that show cause and effect relationships and uncertainties.
• Linear models and similar techniques in neural networks. These models show how each input affects the output.

Various approaches are used to achieve XAI, including visualization, natural language explanations, and interactive interfaces. Interactive interfaces, for example, allow users to explore how model predictions change as input changes.

Visual tools such as heat maps and trees

What are the disadvantages of XAI?

Explainable artificial intelligence has several limitations, some of them related to its application:

• Complexity of development. Large teams of engineers can work on algorithms for a long time. This makes it difficult to understand the entire development process and the principles embedded in AI systems.
• The ambiguity of the term “explainability”. This is a broad concept that can lead to different interpretations when implementing XAI. When analyzing the key parameters and factors in AI, questions arise: what exactly is considered “transparent” or “explainable” and what are the limits of this explainability?
• The rapid development of AI. Artificial intelligence is developing exponentially. Combined with unsupervised systems and deep learning, it could theoretically reach the level of general intelligence. This opens the door to new ideas and innovations, but it also introduces additional complexities when implementing XAI.

What are the prospects for XAI?

Consider study about “generative agents”, whose authors integrated AI language models with interactive agents. During the experiment, a virtual sandbox was created, which is a small town with twenty-five virtual “residents”. Communicating in natural language, they demonstrated realistic individual and social behavior. So, one agent “wanted” to organize a party, after which the agents began to send out invitations on their own.

The word “self” is extremely important here. If AI systems exhibit behavior that is difficult to trace back to individual components, it can have consequences that are difficult to predict.

XAI is able to prevent or at least mitigate some of the risks of using AI. It is important to remember that the ultimate responsibility for AI-based decisions and actions lies with humans, even if not all AI decisions can be explained.

The material was prepared with the participation of language models developed by OpenAI. The information presented here is partly based on machine learning and not real experience or empirical research.