Evaluation system for the test data is now running! (Submission deadline: Feb. 4, 2016)
Number of submissions
466
Number of users
81
Number of participating groups
22
Announcements
MobileClick-2 Evaluation Scripts Released
"mobileclick-eval" is now ready for non-participants to evaluate their runs.
Evaluation System for Test Data Launched
Please submit your runs for the test data and get evaluation results! Please check guideline before submission.
MobileClick-2 Tools Ver0.2 Released
Python package "mobileclick" is now ready for test runs.
MobileClick-2 Test Data Released (Ver. 3)
Please download complete test data for submitting formal runs.
Updates on Task Description
We added "length limit" in the iUnit summarization subtask based on a suggestion by a participating group. Some ambiguous/wrong information were fixed.
MobileClick-2 Test Data Released (Ver. 2)
Please download test data for submitting formal runs. Note that the current test data has not included English iUnits and intents as of October 1, 2015.
MobileClick-2 accepts late registration until February 1, 2016
Since we can provide evaluation results as soon as runs are submitted, we decided to accept late participation as much as possible.
MobileClick-2 Test Data Released (Ver. 1)
Please download test data for submitting formal runs. Note that the current test data include only queries but not iUnits and intents as of August 13, 2015.
Web Search Query Data from Yahoo! JAPAN
You may obtain Web search query data from Yahoo! JAPAN if you register for MobileClick-2. Don't miss this great opportunity!!
MobileClick-2 Training Data Updated (Ver. 2)
We revised the weights of English iUnits since many iUnits have the same importance score. Please download the data again.
MobileClick-2 Tools Released
Please install useful scripts and codes!
Leader Board
- Q-measure is used to order submitted runs in iUnit Ranking
- M-measure is used to order submitted runs in iUnit Summarization
English iUnit Ranking (Test)
| Rank | Group ID | Score | Time |
|---|---|---|---|
| 1 | cuis | 0.9042 | 2016-02-03 18:07:07 UTC |
| 2 | TITEC | 0.9003 | 2016-02-04 06:40:47 UTC |
| 3 | UHYG | 0.8994 | 2016-01-06 05:34:43 UTC |
| 4 | ORG | 0.8975 | 2015-10-31 15:03:26 UTC |
| 5 | RISAR | 0.8972 | 2016-01-29 04:13:47 UTC |
Japanese iUnit Ranking (Test)
| Rank | Group ID | Score | Time |
|---|---|---|---|
| 1 | UHYG | 0.8388 | 2015-11-09 09:30:18 UTC |
| 2 | UHYG | 0.8123 | 2016-02-02 03:24:36 UTC |
| 3 | ORG | 0.7411 | 2015-10-31 15:03:44 UTC |
| 4 | ORG | 0.7269 | 2015-10-31 15:10:56 UTC |
English iUnit Summarization (Test)
| Rank | Group ID | Score | Time |
|---|---|---|---|
| 1 | TITEC | 18.2596 | 2016-02-04 22:33:51 UTC |
| 2 | ORG | 16.8975 | 2015-10-31 15:08:56 UTC |
| 3 | RISAR | 16.047 | 2016-02-05 06:27:34 UTC |
| 4 | ORG | 14.1051 | 2015-10-31 15:08:30 UTC |
| 5 | ORG | 13.2689 | 2015-10-31 15:08:41 UTC |
Japanese iUnit Summarization (Test)
| Rank | Group ID | Score | Time |
|---|---|---|---|
| 1 | UHYG | 22.8342 | 2016-01-22 09:54:39 UTC |
| 2 | UHYG | 21.1107 | 2016-02-01 03:26:58 UTC |
| 3 | ORG | 17.4376 | 2015-10-31 15:10:05 UTC |
| 4 | ORG | 15.0373 | 2015-10-31 15:09:39 UTC |
| 5 | ORG | 12.799 | 2015-10-31 15:09:52 UTC |
English iUnit Ranking (Training)
| Rank | Group ID | Score | Time |
|---|---|---|---|
| 1 | RISAR | 0.8878 | 2016-01-27 14:40:11 UTC |
| 2 | UHYG | 0.8788 | 2016-02-04 03:23:36 UTC |
| 3 | ORG | 0.8772 | 2015-06-30 08:40:39 UTC |
| 4 | ALICA | 0.8621 | 2016-01-30 12:54:00 UTC |
| 5 | ALICA | 0.86 | 2016-02-05 14:01:57 UTC |
Japanese iUnit Ranking (Training)
| Rank | Group ID | Score | Time |
|---|---|---|---|
| 1 | UHYG | 0.8339 | 2016-02-04 03:24:56 UTC |
| 2 | UHYG | 0.8018 | 2016-02-04 03:25:18 UTC |
| 3 | ORG | 0.7901 | 2015-06-26 09:17:30 UTC |
| 4 | ORG | 0.7728 | 2015-06-25 12:56:50 UTC |
News
Schedule
| Date | Event |
|---|---|
| Apr 1, 2015 | Webpage launch |
| Aug 1, 2015 | Test query release |
| Nov 1, 2015 | Formal evaluation starts |
| Feb 4, 2016 (UTC-12) | Run submission due |
| Jun 7-10, 2016 | NTCIR-12 @ Tokyo |