Evaluation system for the test data is now running! (Submission deadline: Feb. 4, 2016)

Number of submissions

462

Number of users

67

Number of participating groups

22

Announcements

MobileClick-2 Evaluation Scripts Released

"mobileclick-eval" is now ready for non-participants to evaluate their runs.

Evaluation System for Test Data Launched

Please submit your runs for the test data and get evaluation results! Please check guideline before submission.

MobileClick-2 Tools Ver0.2 Released

Python package "mobileclick" is now ready for test runs.

MobileClick-2 Test Data Released (Ver. 3)

Please download complete test data for submitting formal runs.

Updates on Task Description

We added "length limit" in the iUnit summarization subtask based on a suggestion by a participating group. Some ambiguous/wrong information were fixed.

MobileClick-2 Test Data Released (Ver. 2)

Please download test data for submitting formal runs. Note that the current test data has not included English iUnits and intents as of October 1, 2015.

MobileClick-2 accepts late registration until February 1, 2016

Since we can provide evaluation results as soon as runs are submitted, we decided to accept late participation as much as possible.

MobileClick-2 Test Data Released (Ver. 1)

Please download test data for submitting formal runs. Note that the current test data include only queries but not iUnits and intents as of August 13, 2015.

Web Search Query Data from Yahoo! JAPAN

You may obtain Web search query data from Yahoo! JAPAN if you register for MobileClick-2. Don't miss this great opportunity!!

MobileClick-2 Training Data Updated (Ver. 2)

We revised the weights of English iUnits since many iUnits have the same importance score. Please download the data again.

MobileClick-2 Tools Released

Please install useful scripts and codes!

Leader Board

  • Q-measure is used to order submitted runs in iUnit Ranking
  • M-measure is used to order submitted runs in iUnit Summarization

English iUnit Ranking (Test)

Rank Group ID Score Time
1 TITEC 0.9003 2016-02-04 06:40:47 UTC
2 UHYG 0.8994 2016-01-06 05:34:43 UTC
3 ORG 0.8975 2015-10-31 15:03:26 UTC
4 RISAR 0.8972 2016-01-29 04:13:47 UTC
5 RISAR 0.8962 2016-02-05 04:19:19 UTC

Japanese iUnit Ranking (Test)

Rank Group ID Score Time
1 UHYG 0.8388 2015-11-09 09:30:18 UTC
2 UHYG 0.8123 2016-02-02 03:24:36 UTC
3 ORG 0.7411 2015-10-31 15:03:44 UTC
4 ORG 0.7269 2015-10-31 15:10:56 UTC

English iUnit Summarization (Test)

Rank Group ID Score Time
1 TITEC 18.2596 2016-02-04 22:33:51 UTC
2 ORG 16.8975 2015-10-31 15:08:56 UTC
3 RISAR 16.047 2016-02-05 06:27:34 UTC
4 ORG 14.1051 2015-10-31 15:08:30 UTC
5 ORG 13.2689 2015-10-31 15:08:41 UTC

Japanese iUnit Summarization (Test)

Rank Group ID Score Time
1 UHYG 22.8342 2016-01-22 09:54:39 UTC
2 UHYG 21.1107 2016-02-01 03:26:58 UTC
3 ORG 17.4376 2015-10-31 15:10:05 UTC
4 ORG 15.0373 2015-10-31 15:09:39 UTC
5 ORG 12.799 2015-10-31 15:09:52 UTC

English iUnit Ranking (Training)

Rank Group ID Score Time
1 RISAR 0.8878 2016-01-27 14:40:11 UTC
2 UHYG 0.8788 2016-02-04 03:23:36 UTC
3 ORG 0.8772 2015-06-30 08:40:39 UTC
4 ALICA 0.8621 2016-01-30 12:54:00 UTC
5 ALICA 0.86 2016-02-05 14:01:57 UTC

Japanese iUnit Ranking (Training)

Rank Group ID Score Time
1 UHYG 0.8339 2016-02-04 03:24:56 UTC
2 UHYG 0.8018 2016-02-04 03:25:18 UTC
3 ORG 0.7901 2015-06-26 09:17:30 UTC
4 ORG 0.7728 2015-06-25 12:56:50 UTC

News

Schedule

Date Event
Apr 1, 2015 Webpage launch
Aug 1, 2015 Test query release
Nov 1, 2015 Formal evaluation starts
Feb 1, 2016 Run submission due
Feb 4, 2016 (UTC-12) Run submission due
Jun 7-10, 2016 NTCIR-12 @ Tokyo