• Home
• Dates
• Registration

 

 

• English tasks
• Arabic tasks
• English Downloads
• Arabic Downloads
• Submissions
• Recipe
• Workshops

 

sh download.sh <your_username> <your_password> train18.list|dev18.list <your_local directory> wav

RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, mono 16000 Hz

Content of XML files

all XML files contain at least three versions of the transcript:

• transcript_orig is a normalized (tokenized) version original transcription provided by the bbc including colour speaker change
• transcript_align is the lightly supervised alignment with associated PMER(phone level MER), WMER(word level MER), WAD(Word Average Duration)
• transcript_lsdecode is the output of the lightly supervised decoding with associated confidence scores (CS)

• transcript_human: the manual transcript. This is currently not normalized and will contain characters as per the transcription guidelines. Scoring will not be done against this version; a normalized version will be provided for that purpose.

• segmentation_only: a baseline segmentation for experiments

Extracting information from XML files

XML files conform to this schema and can be used with any XML-aware software. The examples here use XMLStarlet. These examples will work on a single XML file but you can automate for all files in a particular data set using a command like:

for file in `cat dev.short` ; do <some action on $file.xml> ; done

• Extract the speaker colours and captions from the original BBC subtitles:

   xml sel -t -m "//segments[@annotation_id='transcript_orig']" -m "segment"
  -n -v "concat(//speaker[@id=current()/@who]/@name,'|',text())" 20080522_190000_bbctwo_jonathan_meades_magnetic_north.xml

• Find PMER(phone level MER), WMER(word level MER), AWD(Average Word Duration) for each segment of the lightly supervised alignment - the first element in the concat clause finds the original speaker name instead of generated ID.

  xml sel -t -m "//segments[@annotation_id='transcript_align']" -m "segment" -n
     -v "concat(//speaker[@id=current()/@who]/@name,'|',@starttime,'|',@endtime,'|',@PMER,'|',@WMER,'|',@AWD,'|')" 
     -m "element" -v "concat(text(),' ')" 20080505_183000_bbcfour_the_sky_at_night.xml

Manual Transcription Guidelines

These guidelines were used for manual transcription of the development and evaluation sets.

Scoring Scripts

• Task 1 - transcription: scoring tools with STM and GLM files