Problem
| To meet accessibility standards (Section 508) we needed provide captioning for videos. Doing this manually meant creating a VTT file, including timings for each line, and associating it with the video we were uploading. This was a very time consuming process and there were hundreds of videos to caption. |
Solution
- Transcription services from AWS, Azure, and Google were all tested.
- All results were very close
- AWS and Google could transcribe from most common video formats. Azure required .wav format.
- The only time a human would need to intervene was when the two algorithms did not agree.
Impact
Even though they are very accurate, someone still needs to review the captions.
Important to review results because there is a big difference between “can” and “can’t”.
Resources (Side by Side Comparisons)
About the testing
- Two videos used from the Dell Med School were used.
- Stroke video: 001_Stroke_768x432.mp4; duration 50 seconds
- Enzyme video: “Overview of Enzymes” Ch02-14.mp4; duration 58 seconds
- AWS and Google – the user interface could accommodate testing
- Azure – required more effort as downloading an SDK as well as converting MP4 to WAV format was needed to perform the tests.
- Tests were performed within a six week period between 9/3/19 – 10/16/19.
Results Summary
All results…
Timing was not in-sync with video (vtt file option was unavailable)
Meaning – closed captioning needs to be in-sync with speaker on the video
Punctuation was limited
AWS
Service: Transcribe
Cumulative Results: 95.61%
Videos Tested: 2
The Test: Used AWS Console UI
Description: Uploaded the two videos and results were generated within 1-2 minutes.
Date: 9/3/19
Additional:
Storage: S3 Bucket
Bucket Name: Blausen-transcribe-tests
Names: Stroke-> 001_Stroke_768x432.mp4; Overview of Enzymes-> Ch02-14.mp4
| Answer -> Stroke Video | Results: 99.17% |
| As with all organs in the body, the brain needs oxygen and nutrients in order to function properly. These life-sustaining products are delivered to the brain via the blood that travels through the circulatory system. A stroke occurs when there is a lack of blood to a portion of the brain, resulting in tissue death and loss of brain function. A stroke can be caused by a ruptured blood vessel, such as an aneurysm, or by an embolism, which is a small free-floating clot or particle that becomes lodged in one of the arteries of the brain and interrupts blood flow. Depending on which area of the brain is affected, a stroke can cause speech impairments, paralysis, unconsciousness, or even death. | As with all organs in the body, the brain needs oxygen and nutrients in order to function properly. Thes life sustaining products are delivered to the brain via the blood that travels through the circulatory system. A stroke occurs when there is a lack of blood toe a portion of the brain, resulting in tissue death and loss of brain function. A stroke can be caused by a ruptured blood vessel, such as an aneurysm or by an embolism, which is a small free floating clot or particle that becomes lodged in one of the arteries of the brain and interrupts blood flow. Depending on which area of the brain is affected, a stroke can cost speech impairments, paralysis, unconsciousness or even death. |
| Words: 121 // Errors: 1 |
| Answer -> Enzyme Video | Results: 92.04% |
| Enzymes have a unique surface group termed active site. In chemical reactions, only specific substrate molecules can enter and bind an active site to be converted to products. Enzymes accelerate reactions by: forcing specific orientation of substrates, changing shape of substrate, forming temporary bonds, or creating bond strain. When speeding up reactions is not desirable, enzyme activity may be inhibited by either competitive inhibition in which a non-reactive molecule blocks the active site so a substrate cannot enter. Or allosteric inhibition, whereby enzyme activity is inhibited when molecules bind to the enzyme away from the active site and change the shape of the active site so that substrate can not enter and bind. | enzymes have a unique surface groove termed active site. In chemical reactions on Lee specific substrate molecules can’t enter and buying an active site to be converted to products. Enzymes accelerate reactions by forcing specific orientation of substrates, changing shape of substrate, forming temporary bonds or creating bonds. Strength wind Speeding up reactions is not desirable. Enzyme activity may be inhibited by either competitive inhibition, in which a non reactive molecules blocks the active site so it’s substrate cannot enter or a list eric inhibition whereby enzyme activity is inhibited when molecule’s bind to the enzyme away from the active site and changed the shape of the active site so that the substrate cannot enter and bind. |
| Words: 113 // Errors: 9 |
Azure
Service: Cloud Speech-to-Text
Cumulative Results: 96.49%
Videos Tested: 2
The Test:
- Speech SDK for Python SDK
- Testing via UI option not available
- Rest API test was limited to durations < 10 seconds
- Big Takeaway: Azure speech to text is not available on MP4s. MP4 videos had to be converted to wav files.
- Date: 10/16/19
| Answer -> Stroke Video | Results: 99.17% |
| As with all organs in the body, the brain needs oxygen and nutrients in order to function properly. These life-sustaining products are delivered to the brain via the blood that travels through the circulatory system. A stroke occurs when there is a lack of blood to a portion of the brain, resulting in tissue death and loss of brain function. A stroke can be caused by a ruptured blood vessel, such as an aneurysm, or by an embolism, which is a small free-floating clot or particle that becomes lodged in one of the arteries of the brain and interrupts blood flow. Depending on which area of the brain is affected, a stroke can cause speech impairments, paralysis, unconsciousness, or even death. | As with all organs in the body, the brain needs oxygen and nutrients in order to function properly. These life, sustaining products are delivered to the brain via the blood that travels through the circulatory system. A stroke occurs when there is a lack of blood to a portion of the brain, resulting in tissue death and loss of brain function a stroke can be caused by a ruptured blood vessel such as an aneurysm. Or by an embolism, which is a small free floating clot or particle. It becomes lodged in one of the arteries of the brain and interrupts blood flow. Depending on which area of the brain is affected a stroke can cause speech impairments paralysis unconsciousness or even death. |
| Words: 121 // Errors: 1 |
| Answer -> Enzyme Video | Results: 93.81% |
| Enzymes have a unique surface group termed active site. In chemical reactions, only specific substrate molecules can enter an bind an active site to be converted to products. Enzymes accelerate reactions by: forcing specific orientation of substrates, changing shape of substrate, forming temporary bonds, or creating bond strain. When speeding up reactions is not desirable, enzyme activity may be inhibited by either competitive inhibition in which a non-reactive molecule blocks the active site so a substrate cannot enter. Or allosteric inhibition, whereby enzyme activity is inhibited when molecules bind to the enzyme away from the active site and change the shape of the active site so that substrate can not enter and bind. | Enzymes have a unique surface groove termed active site in chemical reactions only specific substrate molecules can enter an binding active site to be converted to products. Enzymes accelerate reactions by forcing specific orientation of substrates changing shape of substrate. Forming temporary bonds. Or creating bond strength when speeding up reactions is not desirable enzyme activity may be inhibited by either competitive inhibition in which a non reactive molecule blocks. The active site so its substrate cannot enter or allosteric inhibition whereby enzyme activity is inhibited when molecules bind to the enzyme away from the active site and change the shape of the active site, so that the substrate cannot enter and bind. |
| Words: 113 // Errors: 7 |
Google Compute Platform
Service: Cloud Speech-to-Text
Cumulative Results: 96.05%
Videos Tested: 2
The Test: Used UI located at https://cloud.google.com/speech-to-text/
Description: Uploaded the two videos and results were generated within 1-2
minutes.
Date: 10/8/19
| Answer -> Stroke Video | Results: 99.17% Portion of one word – lodged vs. lodge |
| As with all organs in the body, the brain needs oxygen and nutrients in order to function properly. These life-sustaining products are delivered to the brain via the blood that travels through the circulatory system. A stroke occurs when there is a lack of blood to a portion of the brain, resulting in tissue death and loss of brain function. A stroke can be caused by a ruptured blood vessel, such as an aneurysm, or by an embolism, which is a small free-floating clot or particle that becomes lodged in one of the arteries of the brain and interrupts blood flow. Depending on which area of the brain is affected, a stroke can cause speech impairments, paralysis, unconsciousness, or even death. | As with all organs in the body the brain needs oxygen and nutrients in order to function properly. These life-sustaining products are delivered to the brain via the blood that travels through the circulatory system. a stroke occurs when there is a lack of blood to a portion of the brain resulting in tissue death and loss of brain function a stroke can be caused by a ruptured blood vessel such as an aneurysm or by an embolism, which is a small free floating clot or particle that becomes Lodge in one of the arteries of the brain and interrupts blood flow depending on which area of the brain is affected by stroke can cause speech impairments paralysis unconsciousness or even death |
| Words: 121 // Errors: 1 |
| Answer -> Enzyme Video | Results: 92.92% |
| Enzymes have a unique surface group termed active site. In chemical reactions, only specific substrate molecules can enter and bind an active site to be converted to products. Enzymes accelerate reactions by: forcing specific orientation of substrates, changing shape of substrate, forming temporary bonds, or creating bond strain. When speeding up reactions is not desirable, enzyme activity may be inhibited by either competitive inhibition in which a non-reactive molecule blocks the active site so a substrate cannot enter. Or allosteric inhibition, whereby enzyme activity is inhibited when molecules bind to the enzyme away from the active site and change the shape of the active site so that substrate can not enter and bind. | Enzymes have a unique surface group term active site. In chemical reactions only specific substrate molecules can enter and buying an active site to be converted to products enzymes accelerate reactions by forcing specific orientation of substrates changing shape of substrate. forming temporary bonds or creating Pawn strain wind speeding up reaction does not desirable enzyme activity may be inhibited by either competitive inhibition in which a non reactive molecule block the active site. So its substrate cannot enter or allosteric inhibition, whereby an enzyme activity is inhibited when molecules bind to the enzyme away from the active site and change the shape of the active site so that the substrate cannot enter a combine |
| Words: 113 // Errors: 8 |
