“Argument and debate — essentially, it’s the engine that drives the process of science, characterizes what happens in most political forums, and even frames most conceptions of modern religion. Argumentation is one of the defining features of what it is to be human, and if we can convey part of that then I think that means something very important is starting to change,” said Chris Reed, director of the Center for Argument Technology, when IBM was debuting its autonomous debating system, Project Debater, back in 2018.

Computational argumentation technologies is an emerging field of AI research. Teaching machines how to debate pushes the AI system out of its ‘comfort zone’ and calls for the adoption of novel patterns to take on human counterparts. Building a debater system requires simultaneous application of language understanding and language generation capabilities, going beyond the scope of many existing language research works.

The Big Blue has now released the architecture of its debater that can engage humans in competitive debates.

The Project Debater has four main modules–

• Argument mining: This module pinpoints specific arguments and counter-arguments relevant to the motion within a large text corpus.
• Argument Knowledge Base (AKB): It contains arguments, counter-arguments and other texts relevant to the general classes of debates.
• Argument rebuttal: This module generates potential responses to the opponent’s actual speech by matching it with opposing arguments from the modules mentioned above.
• Debate construction: This module selects ideas used and arranges them into a coherent narrative.

Debater Architecture. Credit: Nature

Argument Mining

The argument mining is done in two stages:

• First is the offline stage, where 400 million newspaper articles worth of text is processed. Here, the articles are broken down into sentences, which are indexed by words and the Wikipedia concepts they refer to, the entities they mention, and predefined words. 

• Second is the online stage, where the system relies on the given index to perform a sentence-level argument mining, and retrieves claims and evidence related to the given motion.

Three main tasks are performed during argument mining:

• Sentences with high probability of containing supporting arguments are retrieved using customized queries
• Neural models rank the retrieved sentences based on how closely they represent the arguments.
• Finally, the stance of each argument is classified using neural and knowledge-based methods.

Argument Knowledge Base

This module formally captures the commonalities between different debates. AKB contains principled arguments, counter-arguments and relevant examples.

These texts are either authored manually or generated automatically to be manually edited later. After the text generation, they are grouped into thematic classes. When given a motion, the system uses a feature-based classifier to determine the most suitable class. The class comprises arguments, inspiring quotes, analogies, and instructions to organize them. It also contains sentimental tropes for debates. 

AKB also contributes to the rebuttal module by mapping principled arguments to counter-arguments.

Argument Rebuttal

This module compiles the list of claims that the opponent could potentially make, using the argument mining module, AKB, and the arguments extracted from iDebate, a leading provider of debating resources. IBM’s Watson is then used to convert the opponent’s speech to text using the in-built speech-to-text service for custom language and acoustic models.

Debate Construction

It is a rule-based system that integrates cluster analysis. Arguments predicted to be redundant are removed, and the remaining ones are clustered based on their semantic similarity. Each cluster is assigned a theme.

The system selects arguments for the debate. To enhance fluency, various text normalization and rephrasing techniques are used. Finally, each speech is generated paragraph-by-paragraph using a predefined template. To vocalize the speech, the system applies a custom text-to-speech service.

Wrapping Up

The IBM team said tasks like machine playing games with a human opponent are still very much within the comfort zone of AI because of the following reasons:

• In games, there is a clear definition of a winner. Hence we can use reinforcement learning to train the AI.
• Since individual games have clearly defined moves, the value of each move can be objectively measured.
• An AI system can sometimes use tactics and moves which are not easily interpretable to humans.
• There is a massive amount of relevant structured data available for the development of the system.

However, these points don’t hold water in debating systems that require sophisticated human language, subjectivity and interpretation faculties.

The post Deconstructing IBM’s Project Debater appeared first on Analytics India Magazine.