|
The links below will be active as the information becomes available.
Paper Submissions
Use these Templates
to submit camera-ready
Research Papers in pdf:
|
|
|
|
Description:
***Full-day workshop: morning and afternoon
Who should attend
Technical project leaders and program managers who can share their experiences in deploying machine translation, or who want to learn from the experience of others.
If you have participated in an MT deployment, please contact the workshop organizer about being a Contributing participant in one of the panels. Anyone may register as an Observer. For the purpose of registration, there is no distinction between Contributing participants and Observers, but you must notify the organizer in advance if you want to speak on a panel
Premise
Getting from concept to success with MT (or any language technology) deployment requires more than just budgeting license fees and installing some software. There are typically a variety of technical and human issues involved at every stage, from getting management and user buy-in, to successful sustainment after deployment. Many of these technical and human issues are easy to overlook or underestimate, though they can undermine an otherwise promising project at any stage.
Workshop goal
Develop a set of "recipes" for navigating the technology adoption process. These would be written up to be shared with the larger community to help technology adopters understand the challenges and rewards. Technology vendors and developers may find material to help them help their customers. Depending on the participants, the outcomes could focus on a particular type of deployment (e.g. translation productivity, or information gathering) or cover a broader range of deployment scenarios.
Outcome
The initial report would serve as the first draft of a Shareable handbook/AMTA Resource with same title as the Workshop.
Workshop Format Update:
The morning session will begin with “introductions”. Each contributing participant is invited to give a short (less than 10 minute) high level introduction to the MT project(s) they have been involved in, and their role in that deployment. Up to 3 slides, if desired. Total time – about 90 minutes.
In order to focus the discussion and yield useful insights on MT deployment, we adopted a panel format. 4-5 panels will focus around the topics listed below. The panel discussions will be very interactive, with plenty of contributions from the audience. Presenters are currently being recruited for the panels (contact the organizer to volunteer gerbl@pacbell.net). We'll pick the 3 most popular topics from the list below for exploration in the workshop and allow 10-20 minutes for each panelist on each topic.
Topics:
Level of Automation
What level of automation has worked best in your deployment? (May include division of labor between automation and human interaction.)
Project justification
What information (operational, technical, financial) was used to justify the application. How has the justification and ROI evolved over time?
Languages
Which languages (language pairs and directions) are part of the deployment? Does the deployment include coverage of dialect? How has each language/direction worked out in practice?
Text Type and Quality
What type of language is translated? For example genre (news? personal correspondence?), and subject domain (chemistry? specific product literature?) What is the quality of the text (or transcript) being translated? For example, edited versus spontaneous writing, and original electronic document versus automatically transcribed or recognized text from paper or audio.
Evaluation
How was translation software (and possibly the whole application or components) evaluated before and after deployment?
Organizational change management
Human/cultural: User involvement in planning? Training and role adjustment? Human/Technical: IT Systems? Procedures?
Other Topics? (Write in your suggestion!)
|
|
|
Description:
***Full-day workshop: morning and afternoon
See more information at the workshop website here.
NIST is pleased to introduce the MetricsMATR Challenge, a new series of research challenge events for machine translation (MT) metrology promoting the development of innovative, even revolutionary, MT metrics. MetricsMATR focuses entirely on MT metrics.
Who should participate
The MetricsMATR challenge is designed appeal to a wide and varied audience including researchers of MT technology and metrology, acquisition programs such as SEQUOYAH, and commercial vendors. We welcome submissions from a wide range of disciplines including computer science, statistics, mathematics, linguistics, and psychology. NIST encourages submissions from participants not currently active in the field of MT.
Procedure
- NIST provides the evaluation infrastructure, the source files being MT system output
- The participants develop MT metrics to assess the quality of the source files
The goal is to create intuitively interpretable automatic metrics which correlate highly with human assessment of MT quality.
Motivation and Goals
There are several drawbacks to the current methods employed for the evaluation of MT technology:
- Automatic metrics have not yet been proved able to predict the usefulness and reliability of MT technologies with respect to real applications with confidence.
- Automatic metrics have not demonstrated that they are meaningful in target languages other than English.
- Human assessments are expensive, slow, subjective, and difficult to standardize.
These problems, and the need to overcome them through the development of improved automatic (or even semi-automatic) metrics, have been a constant point of discussion at past NIST MT evaluation events.
MetricsMATR aims to provide a platform to address these shortcomings. Specifically, the goals of MetricsMATR are:
- To inform other MT technology evaluation campaigns and conferences with regard to improved metrology.
- To establish an infrastructure that encourages the development of innovative metrics.
- To build a diverse community which will bring new perspectives to MT metrology research.
- To provide a forum for MT metrology discussion and for establishing future directions of MT metrology.
|
|
president@amtaweb.org
Last updated:
1 August, 2008
|
|
Full Program