Video thumbnail generator fails for videos with a single key frame

• Type: Bug
• Status: Closed
• Resolution: Fixed
• Affects Version/s: 6.2.10 EE GA1, 7.0.0 CE GA1
• Fix Version/s: 6.2.X EE, 7.0.0 CE GA1
• Component/s: Documents & Media
• Labels:
  • ee-ts
  • liferay-fixpack-portal-101-6210
• Branch Version/s:
  6.2.x
• Backported to Branch:
  Committed
• Story Points: 1
• Fix Priority: 2
• Git Pull Request: https://github.com/brianchandotcom/liferay-portal/pull/38173

When enabling xuggler for video thumbnail generation, the process fails for certain videos. After some investigation, it turns out that those videos have just one key frame.

Our generation process tries to seek the keyframe closest to the timestamp calculated from the property dl.file.entry.thumbnail.video.frame.percentage, considering the video length. If there is no keyframe from that point, then we "rewind" the stream to get the first keyframe.

This last method fails when there is only one keyframe in the video.

Some additional tests with the "Presentation Time Stamp" (PTS) and "Decoding Time Stamp" (DTS) shown that those values can be negative and the functionality is not affected: the only condition for the rewind() method to fail is the existence of one single keyframe

Steps to reproduce:
1. Install and enable xuggler from the control panel
2. Upload the attached videos to the Document Library in the control panel
3. See log
4. Reload the document library page in the control panel
5. See log again

Observed result:

• Log shows some exceptions when thumbnail generation is triggered

[com.liferay.portlet.documentlibra[email protected]77a807f2]19:28:06,509 WARN [main][LiferayVideoThumbnailConverter:118] Unable to create thumbnail from specified frame. Will generate thumbnail from the beginning.
18:28:06,860 ERROR [liferay/document_library_video_processor-1][VideoProcessorImpl:378] java.util.concurrent.ExecutionException: com.liferay.portal.kernel.process.ProcessException: java.lang.RuntimeException: Error while seeking file
java.util.concurrent.ExecutionException: com.liferay.portal.kernel.process.ProcessException: java.lang.RuntimeException: Error while seeking file
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at com.liferay.portal.kernel.concurrent.NoticeableFutureConverter.get(NoticeableFutureConverter.java:66)
at com.liferay.portlet.documentlibrary.util.VideoProcessorImpl._generateThumbnailXuggler(VideoProcessorImpl.java:356)
....
at com.liferay.portlet.documentlibrary.messaging.VideoProcessorMessageListener.generate(VideoProcessorMessageListener.java:32)
....
at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask.run(ThreadPoolExecutor.java:667)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.liferay.portal.kernel.process.ProcessException: java.lang.RuntimeException: Error while seeking file
at com.liferay.portlet.documentlibrary.util.VideoProcessorImpl$LiferayVideoThumbnailProcessCallable.call(VideoProcessorImpl.java:727)
at com.liferay.portlet.documentlibrary.util.VideoProcessorImpl$LiferayVideoThumbnailProcessCallable.call(VideoProcessorImpl.java:683)
at com.liferay.portal.kernel.process.local.LocalProcessLauncher.main(LocalProcessLauncher.java:122)
Caused by: java.lang.RuntimeException: Error while seeking file
at com.liferay.portlet.documentlibrary.util.LiferayConverter.rewind(LiferayConverter.java:817)
at com.liferay.portlet.documentlibrary.util.LiferayConverter.rewind(LiferayConverter.java:797)
at com.liferay.portlet.documentlibrary.util.LiferayVideoThumbnailConverter.doConvert(LiferayVideoThumbnailConverter.java:123)
at com.liferay.portlet.documentlibrary.util.LiferayVideoThumbnailConverter.convert(LiferayVideoThumbnailConverter.java:51)
at com.liferay.portlet.documentlibrary.util.VideoProcessorImpl$LiferayVideoThumbnailProcessCallable.call(VideoProcessorImpl.java:724)
... 2 more
18:28:06,861 ERROR [liferay/document_library_video_processor-1][VideoProcessorImpl:475] com.liferay.portal.kernel.exception.SystemException: com.liferay.portal.kernel.exception.SystemException: java.io.FileNotFoundException: /home/dsanz/projects/trunk/server/tomcat/temp/liferay/document_thumbnail/21973.1.0.jpg (No existe el archivo o el directorio)
com.liferay.portal.kernel.exception.SystemException: com.liferay.portal.kernel.exception.SystemException: java.io.FileNotFoundException: /home/dsanz/projects/trunk/server/tomcat/temp/liferay/document_thumbnail/21973.1.0.jpg (No existe el archivo o el directorio)
at com.liferay.portlet.documentlibrary.util.VideoProcessorImpl._generateThumbnailXuggler(VideoProcessorImpl.java:391)
....
at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask.run(ThreadPoolExecutor.java:667)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.liferay.portal.kernel.exception.SystemException: java.io.FileNotFoundException: /home/dsanz/projects/trunk/server/tomcat/temp/liferay/document_thumbnail/21973.1.0.jpg (No existe el archivo o el directorio)
at com.liferay.document.library.kernel.store.BaseStore.addFile(BaseStore.java:107)
....
at com.liferay.document.library.kernel.store.DLStoreUtil.addFile(DLStoreUtil.java:115)
at com.liferay.document.library.kernel.util.DLPreviewableProcessor.addFileToStore(DLPreviewableProcessor.java:222)
at com.liferay.portlet.documentlibrary.util.VideoProcessorImpl.storeThumbnailImages(VideoProcessorImpl.java:298)
at com.liferay.portlet.documentlibrary.util.VideoProcessorImpl._generateThumbnailXuggler(VideoProcessorImpl.java:381)
... 11 more
Caused by: java.io.FileNotFoundException: /home/dsanz/projects/trunk/server/tomcat/temp/liferay/document_thumbnail/21973.1.0.jpg (No existe el archivo o el directorio)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at com.liferay.document.library.kernel.store.BaseStore.addFile(BaseStore.java:102)
... 20 more

This stacktrace denotes that the thumbnail file could not be generated so it's not possible to persist it in the DL Store.

• Only videos having more than 1 key frame have a thumbnail. No file which name starts with "sample_1kf" has thumbnail

Expected result:

• No exceptions
• All videos have thumbnail

Technical notes:

• Attached videos have the following naming convention:

  sample_<number of keyframes>kf_pts(+|-)0_dts(+|-)0_<duration in seconds>_sec
  

  Therefore, each file name indicates the number of key frames in the video, whether the pts > 0 or not, whether the dts > 0 or not and the video duration.

• For 6.2.x, the exception is a bit different because we don't capture the Future errors. So the effect in logs can only be observed if thumbnail generation is not delegated to a separate thread (dl.file.entry.preview.fork.process.enabled=false)
• It's possible to know the video keyframes with the tool ffprobe included in the ffmpeg unix package:

  $> ffprobe -show_frames sample_1kf_dts-0_pts-0_3sec.mp4 > data.txt
  

  Then, data.txt can be examined to look for the text "key_frame=1"
  You can count the keyframes using the following command

  $> ffprobe -show_frames sample_1kf_dts-0_pts-0_3sec.mp4 | grep "key_frame=1" | wc -l