Emotion Cause Analysis (ECA)
In recent years, there has been a large body of research work on emotion classification from text. Most existing work only focused on the classification of emotions into one of the pre-defined emotion categories, for example, Ekman`s six basic emotion categories. Some corresponding bakeoffs or evaluations were organized in the past three years, such as the Emotion Analysis in Chinese Weibo text task on NLP&CC 2013 and NLP&CC 2014. However, in many cases, we care more about the stimuli, or the cause of emotions. For instance, business organizations are more interested in finding out why people like or dislike products or services offered by them from users` comments or reviews rather than a simple categorization of sentiments. Similarly, instead of gauging public opinions towards policies or political issues using frequency counts, governments would like to know the triggering factors of negative attitudes expressed online. As such, there has been an increasing interest in the research on emotion cause analysis more recently. Unfortunately, the lack of annotated corpora and standard metrics for this task has limited the research on this topic. Thus, we propose to organize a new task, emotion cause analysis (ECA), in NTCIR 13.
(1) Construction of corpora:
The city news in the last two years in English and Chinese are collected as the raw data. The emotion expressions in the text are firstly identified and the emotion categories are annotated. Next, the direct cause(s) stimulates such an emotion is identified from its context and is annotated accordingly. In some cases, the direct cause is elaborated in more details in another sentence. For such cases, both the direct cause and elaborated cause are annotated. Here is an example:
"ISIS blow up pagan temple in Palmyra that has stood for 2,000 years. The world community is infuriated by this destruction".
Its annotations are:
Emotion category: Anger
Direct cause: "this destruction"
Elaborated cause: "ISIS blow up pagan temple in Palmyra that has stood for 2,000 years"
A Chinese example is given below:
(Known the things happened in the last 10 minutes of his life, a lot of people left the sad tears)
The annotations are:
Emotion category: Sad
Direct cause:(Known the things happened in the last 10 minutes of his life)
There are also some cases where the emotion cause is not given. For such cases, only the emotion categories are annotated.
(2) Subtasks and evaluation metrics:
a) Emotion cause detection at the clause level
This subtask aims to evaluate the techniques for detecting the clause which contains emotion cause. It can be treated as a clause-level binary text classification problem. The clauses will be classified as containing emotion cause or not. The metrics is based on the standard text classification metrics:
Here, we set lenient and strict evaluations, respectively. For lenient evaluation, the clauses containing either direct cause or elaborated cause are regarded as cause clauses. As for the strict evaluation, only the clauses containing elaborated cause are regarded as cause clauses.
b) Emotion cause extraction
This subtask is designed to evaluate the techniques for detecting the boundary of the emotion cause. It can be regarded as an information extraction task. The metric is based on the overlapping between the detected emotion cause and the annotation at the phrase level:
Similar to Subtask 1, we set lenient and strict evaluations, respectively. For lenient evaluation, both the boundaries of direct cause and elaborated cause are valid. As for the strict evaluation, only the boundaries of elaborated cause are regarded correct.
To evaluate the state-of-the-art emotion cause extraction/detection methods on Chinese and English news text.
Automatic evaluation for each subtask with Precision, Recall and F-measure using clause level classification and phrase boundary level metrics, respectively
The most up-to-date emotion cause analysis techniques on news text involving Simplified Chinese and English.
Ruifeng Xu, Professor, Laboratory of Network Oriented Intelligent Computation, Shenzhen Graduate School, Harbin Institute of Technology, firstname.lastname@example.org
Yulan He, Reader, School of Engineering and Applied Science, Aston University, UK email@example.com
Qin Lu, Professor, Department of Computing, The Hong Kong Polytechnic University, firstname.lastname@example.org
Kam-Fai Wong, Professor, Department of Systems Engineering & Engineering Management, the Chinese University of Hong Kong, email@example.com
Universities / research institutes / companies interested in multi-lingual emotion analysis, especially emotion cause analysis, in Simplified Chinese and English.
We plan to develop an emotion cause corpus on multi-lingual news text including Simplified Chinese and English.
For each language, 3,000 documents will be annotated with emotion causes, in which, 300 documents will be released as the sample data, 2,200 documents will be released as the training data and the rest 500 documents will be used as the testing data. Furthermore, 10,000 documents with emotions will be released for building semi-supervised systems.
|2016-08-15||First call for participation|
|2017-01-08||Sample data release||Local Download: http://hlt.hitsz.edu.cn/?page_id=74|
|Dropbox Download: https://www.dropbox.com/sh/8o8m4ls1txjj6f8/AABtQTVidFTp6-_WjfJWunSMa?dl=0|
|2017-05-04||Training data release|
|2017-07-03||Formal run threads release|
|2017-07-07||Formal run result due|
|2017-08-01||Delivery of evaluation results|
|2017-12||NTCIR-13 Conference & EVIA 2017 in NII, Tokyo, Japan|