Initiative for the Evaluation of XML Retrieval

April 2003 - December 2003
[ Home | News | Participants | Schedule | {down,up}load Area | Workshop | INEX 2004 Workshop Proposal | Organisers ]

Mar 17, 2004 Assessments Version 2.5
  Assessments for topic 61,118,122 have been added.Latest version can be downloaded.
 
Jan 19, 2004 Submission of all the Participants
  Submission of all the participants have been made available.Click here to download.
 
Jan 16, 2004 Assessments Version 2.4
  Assessments for topic 75,83 and 84 have been added.Latest version can be downloaded.
 
Jan 05, 2004 INEX 2003 Workshop Presentations
  Available workshop presentations have been published at Workshop page.
 
Dec 16, 2003 INEX 2003 Workshop Presentations
  Evaluation Metrics related workshop presentations are made available here.
 
Dec 11, 2003 INEX 2004 Workshop Program ( final ) and Pre Proceedings
  The final version of the workshop program is now available here. Workshop Pre Proceedings can also be downloaded from here.
 
Dec 08, 2003 INEX 2004 Workshop Program ( draft )
  The draft version of the workshop program is now available here.
 
Dec 01, 2003 INEX 2004 Track Proposals
  Track proposal are published.
 
Nov 27, 2003 Assessments Version 2.2
  Assessments set contains separate assessment set for strict cas submissions and vague cas submissions. Result evaluated with this set is availabe now.
 
Nov 21, 2003 Assessments Version 2.1
  Topic 115 assessment is also available now.
 
Nov 19, 2003 Official Evaluation Results
  official Evaluation results along with downloadable metrics and assessments are made available to all the participants. Currently, only evaluation results for the CO and SCAS tasks are available. Evaluation results for the VCAS are currently been prepared, and will be distributed shortly.
 
Nov 11, 2003 Assessment Version 1.0
  Assessments version 1.0 is available at http://inex.is.informatik.uni-duisburg.de:2003/internal/.
 
Nov 11, 2003 Delay in Distribution
  There is delay in the distribution of complete INEX collection due to the fact that a number of queries were completed at the end of the October month. (Data about some - very few - quries will be distriuted later since they are still pending).Distribution is planned in beginning of next week.
 
Oct 23, 2003 INEX Workshop - Call for Papers
  Participants are invited to submit papers, not exceeding 8 pages, describing the approach taken to XML retrieval within INEX 2003. Papers should be submitted electronically via email to mounia@dcs.qmul.ac.uk. The paper should be formatted according to the following guidelines.

size: A4
format: pdf
borders: top, left and right 2.5 cm ; bottom 3 cm
title: Times 14 pt bold centered
author(s): Times 10 pt centered
abstract: Times 10 pt justified
body text: Times 10 pt justified
Headlines: Times 12 pt bold left aligned
Emphasis: Times 10 pt italic

Paper submissions in the above general format or in SIGIR style in PDF are highly appreciated. LaTeX and Word templates for SIGIR style submissions are available from http://www.acm.org/sigs/pubs/proceed/template.html.
Submitted papers will be published in the workshop pre-proceedings.

Submission deadline (camera ready): November 28

Note that there will be a formal INEX proceedings published in the beginning of the year 2004. We are also planning a special journal issue, with deadline submission early Spring 2004. Details will be given later.
Oct 21, 2003 Workshop Registration
  Registration for the INEX'03 workshop is now open. Please fill in the registration form, and fax it to Duisburg-Essen (full details can be found on the form) before November 21, 2003.

The workshop will be held at the Dagstuhl Conference Centre from December 15 (Monday) to December 17 (Wednesday).

Please note that arrival before December 14 is not possible, and that the latest departure is after breakfast on December 18. The conference web site, however, provides a list of hotels, e.g. in Mainz for those arriving before Sunday and/or staying after Wednesday (http://www.dagstuhl.de/TravelInfo/Accomodation/).

Detailed traveling information can be found at http://www.dagstuhl.de/TravelInfo/index.en.html

If you need any further information, please do not hesitate to contact inex@is.informatik.ui-duisburg.de or mounia@dcs.qmul.ac.uk
Oct 20, 2003 Call for track proposals for INEX 2004
  At the workshop, we would like to discuss the future of INEX, and in particular proposals for any future tracks, investigating particular aspects of XML retrieval. Currently in INEX we run one track, namely the ad-hoc retrieval track (with three sub-tracks: CO, SCAS and VCAS). There are of course many other retrieval problems that XML search engines aim to target (e.g. question answering, interactive XML retrieval, etc.). Therefore in order to provide an evaluation framework for such systems we aim to extend the list of tracks run at INEX.

We invite interested participating groups to submit track proposals. The proposals at this stage are informal, and should include 2 or 3 paragraphs, containing the following information:

  • aim / motivation of the track
  • effort involved for the participants, in particular with respect to the topic development and the relevance assessment task
  • commitment of the organising institute(s)/group(s)
Proposal should be sent to Mounia Lalmas (mounia@dcs.qmul.ac.uk) by the 28 November 2003.

Received proposals will be overviewed and discussed within a special session organised at the workshop. A selected number of tracks will then be chosen for INEX 2004, where the selection will also take into account interest shown by the participating organisations.
Sep 22, 2003 Topic Assignment v 3.0
  Here is the new topic assignment. Groups concerned are Duisburg-Essen,CWI+Twente and Nils. In the last version, Topic Nos 63 and 67 were assigned to CWI+Twente and Duisburg-Essen respectively. Now these are assigned to new assessor Nils.
 
Sep 18, 2003 Online Assessment
  Online assessment can be accessed using the link http://inex.lip6.fr with your inex usename and password combination. Topics are assigned to all the participants according to the Topic Assignment document that was distributed earlier.
 
Sep 18, 2003 Relevance Assessment Guide
  Relevance Assesment guide is distributed among the participants.
 
Sep 17, 2003 Extention to relevance Assessment Deadline
  Relevance Assesment submission deadline is extended to October 15, 2003.
 
Sep 08, 2003 Topic Assignment for Relevance Assessments
  Topis are assigned to all the participants for relevance assessments.Click here to view details.
 
Aug 13, 2003 Schedule Change
  Submission deadline of the official retrieval results is extended from August 15,2003 to August 29,2003 and Distribution of pooled retreval results deadline is extended from August 25,2003 to September 12,2003.
 
Aug 08, 2003 INEX'03 Topics Version (1.4.7)
  INEX'03 topics version 1.4.7 is available .In this version, last condition of topic 89 title is changed from ./au to ./au/snm .
 
Aug 05, 2003 INEX'03 Topics Version (1.4.6)
  INEX'03 topics version 1.4.6 is available at http://inex.is.informatik.uni-duisburg.de:2003/internal/#topics2003.A minor change has been made in keywords of topic 61, phrase "clustering distributed" is changed to two comma separated words clustering,distributed.
 
Aug 01, 2003 INEX'03 Topics Version (1.4.5)
  INEX'03 topics version 1.4.5 is available at http://inex.is.informatik.uni-duisburg.de:2003/internal/#topics2003 In topics 64,65,78 and 87, "./" is replaced with "." and in topic 75, symbol"´" is replaced with single quotes(').
 
Jul 31, 2003 Online Run Submissions
  Runs can be submitted online at http://inex.is.informatik.uni-duisburg.de:2003/cgi-bin/inex?mode=browse_submissions. Please use your username and password to access this area.
 
Jul 24, 2003 INEX'03 Topics Version (1.4.4)
  INEX'03 topics version 1.4.4 is available at http://inex.is.informatik.uni-duisburg.de:2003/internal/#topics2003 In topic 81,Title element .//atl is changed to ./fm//atl.
 
Jun 23,2003 Retrieval Task and Result Submission Format Specification Guide(Version 1.1.3)
  A paragraph(highlighted on page 4) is added to the INEX03 Retrieval Task and Result Format Guide explicitly specifying that the rules for equivalent tags should be ignored when generating the XPath of the result elements.
 
Jul 23, 2003 INEX'03 Topics Version (1.4.3)
  INEX'03 topics version 1.4.3 is available at http://inex.is.informatik.uni-duisburg.de:2003/internal/#topics2003 Changes are made in topics 65,66,76,85,87,88 and 90 for title and narrative consistancy.
 
Jul 22, 2003 Schedule Change
  Submission deadline of the official retrieval results is extended from August 1,2003 to August 15,2003 and Distribution of pooled retreval results deadline is extended from August 15,2003 to August 25,2003.
 
Jul 21, 2003 WIKI Server
  In order to facilitate collaboration, exchange of files, and discussions, a Wiki server has been set up for use by the INEX community by QUT(many thanks to Shlomo Geva for setting up the server).
http://inex.fit.qut.edu.au
 
Jul 11, 2003 INEX'03 Topics Final Version(1.4)
  Final version of INEX'03 topics is available to be used for retreival runs. It consists of 66 topics,36 CO and 30 CAS.
 
Jul 08, 2003 INEX'03 Topics Version 1.3
  Four more topics have been added to INEX'03 topics.Remaining one topic will be included on the receipt of its correction.
 
Jul 04, 2003 INEX'03 Topics Version 1.2
  The following candidate topics are added to this version:46, 49, 81, 82, 87, 91, 92, 106.Remaining topics will be included on the receipt of their corrections.
 
Jul 01, 2003 INEX'03 Topics
  INEX'03 topics, including most of the topics, are available to be used for retrieval runs. Remaining topics will be included on the receipt of their corrections.
 
Jun 25,2003 INEX'03 Retrieval Task and Result Submission Format Specification
  Document describing this year retrieval task and result submission format is available here
 
Jun 25,2003 Topic Correction Request
  In order to view the topics selected for INEX'03 along with details of required corrections/modifications, please click here. Corrected topics should be submitted to Saadia Malik via email ( malik@is.informatik.uni-duisburg.de ) till June 27,2003
 
Jun 20,2003 Delay in distribution of final set of topics
  There will be slight delay in the distribution of final set topics.
 
Jun 19,2003 INEX'03 Tasks
  As last year, the task to be performed by the participating groups of INEX'03 is defined as the ad-hoc retrieval of XML documents. This year, within the ad-hoc retrieval task we define the following three sub-tasks:

CO: Content-oriented XML retrieval using content-only queries.

SCAS: Content-oriented XML retrieval based on content-and-structure queries, where the structural constrainsts of a query must be strictly matched.

VCAS: Content-oriented XML retrieval based on content-and-structure queries, where the structural constrainsts of a query can be treated as vague conditions, which can be matched by XML elements not explicitly specified in the query.

While the first two sub-tasks are the same as last year, the VCAS task is new for INEX. The idea behind the VCAS task is to allow the evaluation of XML retrieval systems that aim to implement a more fuzzy approach to XML retrieval, where not only the content conditions within a user query are treated with uncertainty but also the expressed structural conditions. We are currently defining this task in details. Any feedback for this task are welcome.
 
Jun 13,2003 Deadline for Candidate Topic Submission is Extended
  Deadline for Candidate topics submission is extended to June 13,2002.Please submit your 6 topics, 3 Content-only and 3 content-and-structure till this date. To view submitted topics information, click here

May 30,2003 Candidate Topic Submission
  Each participating organisation should submit 6 candidate topics, 3 Content-only and 3 Content and Structure, till May 30,2003. Click here to view Topic development guide