MusicDBExtern¶

This class handles the upload of music files to external storages. Before most of the methods can be used, the mountpoint of the external storage must be set using SetMountpoint(). The default mount poiunt is "/mnt". There are three main tasks this class implements:

Initializing a Storage
Updating a Storage
Handling Toxic Environments

Initializing a Storage¶

Each storage must have a directory configured in the main MusicDB-Configuration, that holds some states. This directory can be created and initialized using the InitializeStorage(). It is possible to check if the storage was already initialized using the method IsStorageInitialized().

Example

database = MusicDatabase("./music.db")
config   = MusicDBConfig("./musicdb.ini")
extern   = MusicDBExtern(config, database)
extern.SetMountpoint("/mnt")

# Initialize mounted storage if not done yet
if not extern.IsStorageInitialized():
    extern.InitializeStorage()

After initializing, a directory is created and a bare config.ini is copied from the MusicDB share directory. The name of the directory, the config-source-file and the state-file-names can be configured in the MusicDB-Config file. It is recommended to use a hidden directory for the states on the external storage. MusicDB must have write access to the directory and its files.

Storage Configuration¶

The storage configuration can be used to adapt to the environment other software or devices require to use the music that will be stored on it. More details about Handling Toxic Environments can be found in the related subsection.

Template for such a storage configuration:

[meta]
version=3

[constraints]
; define the allowed charset:
;  * default: use the default chars that are used by the source (utf-8)
;  * FAT:     utf-8 without the chars forbidden by the FAT-Spec
charset=FAT

; max path length (0 = infinity length)
pathlen=256

; some really bad players like the one in my car do not allow other fileformats than mp3
; Set forcemp3 to True to convert all "foreign" encoded files to crappy mp3s
forcemp3=True

[paths]
; path for the music relative to the mountpoint - usually /
;  * /:      root directory of the external device
;  * /MUSIC: used by some over engineered firmware
musicdir=/

[mp3tags]
; Remove all unnecessary ID3 frames and use the values from the database
optimize=True

; prescale the album cover (optimize must be True otherwise this option will be ignored)
; None:    No prescaling
; False:   No prescaling
; {X}x{Y}: Scale to XxY
prescale=240x240

; if optimize=True, remove artwork if true
noartwork=False

; Do not use the modern 2.4.0 version
forceID3v230=False

[m4atags]
; Update tags to database entries - EXPERIMENTAL!
optimize=True

The State-File¶

The state-file mostly called songmap is a Comma Separated Value (csv) file. It contains a row for each song on the storage and is used to identify the song. This is mandatory because the path on the external storage may differ from the paths of the music collection. This can happen for example by transcoding or renaming to a FAT compatible path.

Each row has the following columns:

Original file path (relative)

File path on the external storage (relative)

The csv file must have the following dialect:

delimiter: ,

escapechar: \

quotechar: "

quoting: csv.QUOTE_NONNUMERIC

Example

"Rammstein/2001 - Mutter/03 Sonne.m4a","Rammstein/2001 - Mutter/03 Sonne.mp3"

This file gets generated by the method WriteSongmap() and can be read by ReadSongmap(). Usually the user should never touch this file.

Updating a Storage¶

Updating an external storage device like a mp3-player or a SD-Card, the UpdateStorage() method can be used.

Example

database = MusicDatabase("./music.db")
config   = MusicDBConfig("./musicdb.ini")
extern   = MusicDBExtern(config, database)
extern.SetMountpoint("/mnt")

# Update storage if it is valid
if extern.IsStorageInitialized():
    extern.UpdateStorage()

Handling Toxic Environments¶

A toxic environment is a device that has some limitations and constraints the exported music has to fulfill. For example, my car can only read mp3 files, has a path length limit of 256 characters and can only access a FAT filesystem. My mp3-player does not have that many constraints, but slows down if the album covers are too large and have to be scaled down the mp3 players screen resolution. Those limitations can be handled by MusicDBExtern.

There are several methods to handle toxic environments. They can be activated in the config file that will be generated when the storage gets initialized.

The following methods will be applied if activated in the config:

ReducePathLength() if the pathlength is limited

musicdb.lib.fileprocessing.Fileprocessing.ConvertToMP3() if only mp3-files are allowed

musicdb.lib.fileprocessing.Fileprocessing.OptimizeMP3Tags() to scale artwork and make proper ID3 tags

musicdb.lib.fileprocessing.Fileprocessing.OptimizeM4ATags() make proper meta tags for m4a files

FixPath() to handle unicode in paths

Warning

When optimizing M4A-Tags, the album artwork gets lost. This is a bug in ffmpeg. I did not find any good workarounds yet.

MusicDBExtern Class¶

class musicdb.mdbapi.extern.MusicDBExtern(config, database)[source]¶

Parameters

config – MusicDB configuration object
database – MusicDB database

Raises

TypeError – when config or database not of type MusicDBConfig or MusicDatabase

CheckForDependencies() → bool[source]¶

Checks for dependencies required by this module. Those dependencies are ffmpeg and id3edit <https://github.com/rstemmer/id3edit>_.

If a module is missing, the error message will be printed into the log file and also onto the screen (stderr)

Returns: True if all dependencies exist, otherwise False.

CopyNewSongs(songmap, extconfig)[source]¶

This method handles the songs that are new to the collection and not yet copied to the external storage. The process is split into two tasks:

Generate path names for the new files on the external storage

Copy the songs to the external storage

The copy-process itself is done in another method CopySong(). In future, the CopySong method shall be called simultaneously for multiple songs.

Parameters: songmap – A list of tuples representing the external storage state.
Returns: songmap with the new state of the storage. The dstpath-column is set for the copied songs.

CopySong(relsrcpath, reldstpath, extconfig)[source]¶

In this method, the copy process is done. This method is the core of this class. The copy process is done in several steps:

Preparation of all paths and configurations

Create missing directories

Transcode, optimize, copy file

It also creates the Artist and Album directory if they do not exist. If a song file already exists, the copy-process gets skipped.

Parameters

relsrcpath (str) – relative source path to the song that shall be copied
reldstpath (str) – relative destination path. Its extension may be changed due to transcoding.
extconfig – Instance of the external storage configuration

Returns

On success the updated reldstpath is returned. It may differ from the parameter due to transcoding the file. Otherwise None is returned.

FixPath(string, charset)[source]¶

This method places characters that are invalid for charset by a valid one.

Replaces ?<>\:*|" by _

Replaces äöüÄÖÜß by aouAOUB

Warning

Obviously, this method is incomplete and full of shit. It must and will be replaced in future.

Example

self.FixPath("FAT/is f*cking/scheiße.mp3")
# returns "FAT/is f_cking/scheiBe.mp3"

Parameters

string (str) – string that shall be fixed
charset (str) – until now, only "FAT" is considered. Other sets will be ignored

Returns

A string that is valid for charset

InitializeStorage()[source]¶

This method creates the state-directory inside the mountpoint. Then a template of the storage configuration gets copied inside the new creates state-directory

Returns: True on success, else False

IsStorageInitialized()[source]¶

This method checks for the state-directory and the storage configuration file inside the state directory. If both exists, the storage is considered as initialized and True gets returned.

Returns: True if storage is initialized, otherwise False

ReadSongmap(mappath)[source]¶

This method reads the song map that maps relative song paths from the collection to relative paths on the external storage.

Parameters: mappath (str) – absolute path to the songmap
Returns: None if there is no songmap yet. Otherwise a list of tuples (srcpath, dstpath) is returned.

ReducePathLength(path)[source]¶

This method reduces a path length to hopefully fit into a path-length-limit. The reduction is done by removing the song name from the path. Everything left is the directory the song is stored in, the song number and the file extension.

Example

self.ReducePathLength("artist/album/01 very long name.mp3")
# returns "artist/album/01.mp3"
self.ReducePathLength("artist/album/1-01 very long name.mp3")
# returns "artist/album/1-01.mp3"

Parameters: path (str) – path of the song
Returns: shortend path as string if successfull, None otherwise

RemoveOldSongs(songmap, extconfig)[source]¶

Remove all songs that have a destination-entry but no source-entry in the songmap. This constellation means that there is a file on the storage that does not exist in the music collection.

Parameters

songmap – A list of tuple representing the external storage state.
extconfig – Instance of the external storage configuration.

Returns

The updated songmap without the entries of the files that were removed in this method

SetMountpoint(mountpoint='/mnt')[source]¶

Sets the mountpoint MusicDBExtern shall work on. If the mountpoint does not exists, False gets returned. The existence of a mountpoint does not guarantee that the device is mounted. Furthermore the method does not check if the mounted device is initialized - this can be done by calling IsStorageInitialized().

Parameters: mountpoint (str) – Path where the storage that shall be worked on is mounted
Returns: True if mountpoint exists, False otherwise.

UpdateSongmap(songmap, mdbpathlist)[source]¶

This method updates the songmap read with ReadSongmap(). Therefore, new source-paths will be added, and old one will be removed. The new ones will be tuple of (srcpath, None) added to the list. Removing songs will be done by replacing the sourcepath with None. This leads to a list of tuple, with each tuple representing one of the following states:

(srcp, dstp) Nothing to do: Source and Destination files exists

(srcp, None) New file in the collection that must be copied to the external storage

(None, dstp) Old file on the storage that must be removed

Parameters

songmap – A list of tuples representing the external storage state. If None, an empty map will be created.
mdbpathlist – list of relative paths representing the music colletion.

Returns

The updated songmap gets returned

UpdateStorage()[source]¶

This method does the whole update process. It consists of the following steps:

Prepare envrionment like determin configfiles and opening them.
Get all song paths from the Music Database
ReadSongmap() - Read the current state of the external storage
UpdateSongmap() - Update the list with the current state of the music collection
RemoveOldSongs() - Remove old songs from the external storage that are no longer in the collection
CopyNewSongs() - Copy new songs from the collection to the storage. Here, transcoding will be applied if configured. See Handling Toxic Environments
WriteSongmap() - Writes the new state of the external storage device

Returns: None

WriteSongmap(songmap, mappath)[source]¶

Writes all valid entries of songmap into the state-file. This method generates the new state of the external storage.

A valid entry has a source and a destination path.

Parameters

songmap – A list of tuples representing the external storage state.
mappath (str) – Path to the state-file

Returns

None