The 61st chapter of the Association for Computational Linguistics is ongoing in Toronto, Canada bringing together researchers and practitioners from the field of computational linguistics.

Out of the plethora of research showcased at the annual meet, we have picked out the 8 brilliant papers at the conference that caught our attention!

Backpack Language Models

AI language models exhibit gender bias in pronoun distributions, favoring gendered pronouns based on context. This bias can be flipped by replacing stereotypically associated professions. However, achieving consistent debiasing across all contexts is challenging. 

Backpack LM addresses this by leveraging non-contextual sense vectors, capturing multiple aspects of a word’s meaning. By incorporating Backpack LM, we can mitigate biases and create fairer, more inclusive language models with improved interpretability and control.

Authors: John Hewitt, John Thickstun, Christopher D. Manning, Percy Liang

Do Androids Laugh at Electric Sheep? Humor “Understanding” Benchmarks from The New Yorker Caption Contest

Can AI models truly grasp humor? In the research paper, resarchers test them with New Yorker Cartoon Caption Contest tasks: matching jokes to cartoons, identifying winning captions, and explaining their humor.

The authors explored the capabilities of both multimodal models, which directly engage with cartoon images, and language-only models, which are provided with rich descriptions. 

Authors: Jack Hessel, Ana Marasovic, Jena D. Hwang, Lillian Lee, Jeff Da, Rowan Zellers, Robert Mankoff and Yejin Choi

Don’t Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environments

Current language models lack the ability to ground to real-world environments. Existing approaches place the burden of generating executable plans on the language models themselves, which leads to challenges in maintaining grammaticality, faithfulness, and controllability. 

To address this, in this paper, researchers introduce Pangu, a framework that leverages the discriminative power of language models for grounded language understanding, instead of relying on their generative capabilities.

Authors: Yu Gu, Xiang Deng and Yu Su

Minding Language Models’ (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker

Large-scale neural language models lack basic Theory of Mind (ToM) — the ability to reason about the mental states of other people. 

Researchers propose SymbolicToM, a plug-and-play approach that enables reasoning about belief states of multiple characters using explicit symbolic representation. It tracks each entity’s beliefs, estimations of others’ beliefs, and higher-order reasoning through graphical representations, enhancing precision and interpretability in reading comprehension tasks.

Authors: Melanie Sclar, Sachin Kumar, Peter West, Alane Suhr, Yejin Choi and Yulia Tsvetkov

The Mechanical Bard: An Interpretable Machine Learning Approach to Shakespearean Sonnet Generation

Researchers explore automated generation of Shakespearean sonnets, utilizing constrained decoding to adhere to meter, rhyme scheme, length, and poetic conventions.

The approach produces sonnets resembling human-authored ones, with lyrical language, literary devices, and adherence to genre constraints, as confirmed by human evaluation.

Authors: Edwin Agnew, Michelle Qiu, Lily Zhu, Sam Wiseman and Cynthia Rudin

World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models

Grounding language in the physical world is crucial for understanding word meanings. To introduce the factor in language models, researchers present Grounded Open Vocabulary Acquisition (GOVA), which explores grounding and bootstrapping in open-world language learning. 

Their initial approach is object-oriented BERT (OctoBERT), a visually-grounded language model that pre-trains on image-text pairs with a focus on grounding.

Authors: Ziqiao Ma, Jiayi Pan and Joyce Chai

Forgotten Knowledge: Examining the Citational Amnesia in NLP

Have you ever wondered how old are the papers you cite? Or If we fail to read older papers and benefit from important ideas? In this paper, researchers explore questions like these about Natural Language Processing (NLP) papers with data and graphs. 

Authors: Janvijay Singh, Mukund Rungta, Diyi Yang and Saif Mohammad

Causes and Cures for Interference in Multilingual Translation

This research paper from Meta explores the little-understood phenomenon of interference, broadly defined as a negative interaction between different translation directions in a multilingual machine translation model.

“Interference trends can be tricky to measure,” lead author Uri Shaham acknowledged in a December 16, 2022 tweet, summing up the paper’s central questions: “What causes interference or synergy between language pairs in multilingual translation? Do we actually need specialized algorithms to alleviate interference?”

Authors: Uri Shaham, Maha Elbayad, Vedanuj Goswami, Omer Levy, Shruti Bhosale

The post 8 Outstanding Papers Presented at the ACL 2023 appeared first on Analytics India Magazine.