Video Frames and Preview Management

This module handles the thumbnails (frames) and previews (git-animation) of videos. Its main task is to cache, scale and provide them to the GUI.

Attention

Frame and preview scaling has not been implemented yet.

Definitions in Context of Videos

frame:

One frame extracted from a music video stored as a picture.

thumbnail:

One video frame that is used as image to represent the video in the UI. File format is JPEG, the file names begin with the prefix frame-. In context of HTML video-tags, this thumbnail is used as poster

preview:

A short WebP-animation consisting of several frames of the video. This animation will can be played when the cursor hovers above the video. The file names begin with the prefix preview.

Relevant Database Entries

The thumbnail and preview data is part of the video entry in the MusicDB Database. The thumbnail and preview part consists of the following entries:

framesdirectory

thumbnailfile

previewfile
framesdirectory:

The path to a frame relative to the thumbnails and previews root directory set in the MusicDB Configuration. To access a scaled version of the artwork, the scale as suffix can be used.

Video Frames Path structure

The video frames root directory can be configured in the MusicDB Configuration file. Everything related to video frames takes place in this directory. To use the artwork inside a web frontend, the HTTPS server needs access to this directory.

Relative to the frames root directory are the frames paths stored in the database. For each video a sub directory exists. Source-frames and all scaled frames as well as WebP animations are stored in this sub directory.

The name of the frames directory for a video, consists of the artist name and video name: $Artistname/$Videoname. This guarantees unique file names that are human readable at the same time.

Inside this sub directory the following files exists:

frame-$i.jpg:

The source video frames. $i is a continuing number with two digits starting by 1. For one video, multiple frames will be extracted. The exact number of frames can be defined in the configuration file. The file format is JPEG. The samples are collected uniform distributed over the video length.

frame-$i ($s×$s).jpg:

A scaled version of frame-$i. $s represents the scaled size that can be configured in the configuration file. The separator is a multiplication sign × (U+00D7) to make the name more human readable. Multiple scales are possible. For example, the name of the 5th frame scaled down to a size of max 100px would be frame-5 (100×100).jpg

preview.webp:

A preview of the video as animation. All source frames available as JPEG are combined to the GIF animation. The amount of frames can be configured, as well as the animation length. The frames are uniform distributed over the animation length.

preview ($s×$s).webp:

A scaled version of the preview animation.

The sub directory name for each video gets created by the method CreateFramesDirectoryName(). This method replaces “/” by an Unicode division slash (U+2215) to avoid problems with the filesystem.

All new creates files and directories were set to the ownership [musicdb]->username:[musicdb]->groupname and gets the permission rw-rw-r-- (+x for directories)

Attention

Existing frames and previews will not be overwritten. In case the settings change, video frames and previews need to be removed manually before recreating them.

HTTPS Server Configuration for Video Frames

Web browsers has to prefix the path with videoframes/. So, the server must be configured. The resulting path will then be for example "videoframes/$framesdirectory/$previewfile".

Scaling Video Frames

Scales that shall be provides are set in the MusicDB Configuration as list of edge-lengths. For example, to generate 50×27 and 150×83 versions of a frame, the configuration would look like this: scales=50x27, 150x83 The scaled frames get stored as progressive JPEGs to get a better responsiveness for the WebUI.

Usually videos do not have a ration of 1:1. If the aspect ration of a frame differs from the desired thumbnail, the borders will be cropped.

Configuration of Video Frames

An example configuration can look like the following one:

[videoframes]
path=/data/musicdb/videoframes  ; Path to the sub directories for videos
frames=5                        ; Grab 5 frames from the video
scales=50x27, 150x83            ; Provide scales of 50px and 150px width with aspect ration of 16/9
previewlength=3                 ; Create GIF-Animations with 3 second loop length

Under these conditions, a 150×83 pixels preview animation of a video “Sonne” from “Rammstein” would have the following absolute path: /data/musicdb/videoframes/Rammstein/Sonne/preview (150×83).webp. Inside the database, this path is stored as Rammstein - Sonne. Inside the HTML code of the WebUI the following path would be used: Rammstein/Sonne/preview (150×83).webp.

Algorithm for Creating Video Frames

To update the frames cache the following steps are done:

  1. Create a sun directory for the frames via CreateFramesDirectory()

  2. Generate the frames from a video with GenerateFrames()

  3. Generate the previews from the frames with GeneratePreviews()

  4. Update database entry with the directory name in the database via SetVideoFrames()

VideoFrames Class

class musicdb.mdbapi.videoframes.VideoFrames(config, database)[source]

This class implements the concept described above. The most important method is UpdateVideoFrames() that generates all frames and previews for a given video.

Parameters

  • config – MusicDB configuration object

  • database – MusicDB database

Raises

  • TypeError – if config or database are not of the correct type

  • ValueError – If one of the working-paths set in the config file does not exist

ChangeThumbnail(video, timestamp)[source]

This method creates a thumbnail image files, including scaled a version, from a video. The image will be generated from a frame addressed by the timestamp argument.

To generate the thumbnail, ffmpeg is used in the following way:

ffmpeg -y -ss $timestamp -i $video["path"] -vf scale=iw*sar:ih -vframes 1 $videoframes/$video["framesdirectory"]/thumbnail.jpg

video and timestamp are the parameters of this method. videoframes is the root directory for the video frames as configured in the MusicDB Configuration file.

The scale solves the differences between the Display Aspect Ratio (DAR) and the Sample Aspect Ratio (SAR). By using a scale of image width multiplied by the SAR, the resulting frame has the same ratio as the video in the video player.

The total length of the video gets determined by GetPlaytime() If the time stamp is not between 0 and the total length, the method returns False and does nothing.

When there is already a thumbnail existing it will be overwritten.

Parameters

  • video – A video entry that shall be updated

  • timestamp (int) – Time stamp of the frame to select in seconds

Returns

True on success, otherwise False

CreateAnimation(framepaths, animationpath)[source]

This method creates a WebP animation from frames that are addresses by a sorted list of paths. Frame paths that do not exists or cannot be opened will be ignored. If there already exists an animation addressed by animation path, nothing will be done.

The list of frame paths must at least contain 2 entries.

Parameters

  • framepaths (list(str)) – A list of relative frame paths that will be used to create an animation

  • animationpath (str) – A relative path where the animation shall be stored at.

Returns

True when an animation has been created or exists, otherwise False

CreateFramesDirectory(artistname, videoname)[source]

This method creates the directory that contains all frames and previews for a video. The ownership of the created directory will be the music user and music group set in the configuration file. The permissions will be set to rwxrwxr-x. If the directory already exists, only the attributes will be updated.

Parameters

  • artistname (str) – Name of an artist

  • videoname (str) – Name of a video

Returns

The name of the directory.

CreateFramesDirectoryName(artistname, videoname)[source]

This method creates the name for a frames directory regarding the following schema: $Artistname/$Videoname. If there is a / in the name, it gets replaced by ∕ (U+2215, DIVISION SLASH)

Parameters

  • artistname (str) – Name of an artist

  • videoname (str) – Name of a video

Returns

valid frames sub directory name for a video

GenerateFrames(dirname, videopath)[source]

This method creates all frame files, including scaled frames, from a video. After generating the frames, animations can be generated via GeneratePreviews().

To generate the frames, ffmpeg is used in the following way:

ffmpeg -ss $time -i $videopath -vf scale=iw*sar:ih -vframes 1 $videoframes/$dirname/frame-xx.jpg

videopath and dirname are the parameters of this method. videoframes is the root directory for the video frames as configured in the MusicDB Configuration file. time is a moment in time in the video at which the frame gets selected. This value gets calculated depending of the videos length and amount of frames that shall be generated. The file name of the frames will be frame-xx.jpg where xx represents the frame number. The number is decimal, has two digits and starts with 01.

The scale solves the differences between the Display Aspect Ratio (DAR) and the Sample Aspect Ratio (SAR). By using a scale of image width multiplied by the SAR, the resulting frame has the same ratio as the video in the video player.

The total length of the video gets determined by GetPlaytime()

When there are already frames existing, nothing will be done. This implies that it is mandatory to remove existing frames manually when there are changes in the configuration. For example when increasing or decreasing the amount of frames to consider for the animation. The method will return True in this case, because there are frames existing.

Parameters

  • dirname (str) – Name/Path of the directory to store the generated frames

  • videopath (str) – Path to the video that gets processed

Returns

True on success, otherwise False

GeneratePreviews(dirname)[source]

This method creates all preview animations (.webp), including scaled versions, from frames. The frames can be generated via GenerateFrames().

In case there is already a preview file, the method returns True without doing anything.

Parameters

dirname (str) – Name/Path of the directory to store the generated frames

Returns

True on success, otherwise False

ScaleFrame(dirname, framenumber)[source]

This method creates a scaled version of the existing frames for a video. The aspect ration of the frame will be maintained. In case the resulting aspect ratio differs from the source file, the borders of the source frame will be cropped in the scaled version.

If a scaled version exist, it will be skipped.

The scaled JPEG will be stored with optimized and progressive settings.

Parameters

  • dirname (str) – Name of the directory where the frames are stored at (relative)

  • framenumber (int) – Number of the frame that will be scaled

Returns

Nothing

ScaleThumbnail(dirname)[source]

This method creates a scaled version of the existing thumbnail for a video. The aspect ration of the frame will be maintained. In case the resulting aspect ratio differs from the source file, the borders of the source frame will be cropped in the scaled version.

If a scaled version exist, it will be overwritten.

The scaled JPEG will be stored with optimized and progressive settings.

Parameters

dirname (str) – Name of the directory where the frames are stored at (relative)

Returns

Nothing

SetVideoFrames(videoid, framesdir, thumbnailfile=None, previewfile=None)[source]

Set Database entry for the video with video ID videoid. Using this method defines the frames directory to which all further paths are relative to. The thumbnail file addresses a static source frame (like frame-01.jpg), the preview file addresses the preview animation (usually preview.webp).

If thumbnailfile or previewfile is None, it will not be changed in the database.

This method checks if the files exists. If not, False gets returned an no changes will be done in the database.

Example

retval = vf.SetVideoFrames(1000, "Fleshgod Apocalypse/Monnalisa", "frame-02.jpg", "preview.webp")
if retval == False:
    print("Updating video entry failed.")
Parameters

  • videoid (int) – ID of the video that database entry shall be updated

  • framesdir (str) – Path of the video specific sub directory containing all frames/preview files. Relative to the video frames root directory

  • thumbnailfile (str, NoneType) – File name of the frame that shall be used as thumbnail, relative to framesdir

  • previewfile (str, NoneType) – File name of the preview animation, relative to framesdir

Returns

True on success, otherwise False

UpdateVideoFrames(video)[source]

  1. Create frames directory (CreateFramesDirectory())

  2. Generate frames (GenerateFrames())

  3. Generate previews (GeneratePreviews())

Parameters

video – Database entry for the video for that the frames and preview animation shall be updated

Returns

True on success, otherwise False