||2 months ago|
M U Z I K F A K T R Y by 12bytes.org
- Folder Structure
- Error Levels
- Tagging Operations
- Duplicates & Hashes
- Repairing Files
- Rotate Files
- COMMAND LINE OPTIONS
Muzik Faktry is a user-friendly, menu driven, Bash shell script for various GNU/Linux based operating systems. The script is primarily intended for working with lossless music files and transcoding them to the FLAC format and/or performing various operations and integrity checks on them before adding them to your collection. Producing clean, compact, error free FLAC files is the primary goal of the script. All of the tasks performed by Muzik Faktry are non-destructive (lossless), meaning there is no reduction in the quality of the audio.
The audience for Muzik Faktry might range from the audiophile to the casual listener. In either case the script is designed to be easy to use while still providing the user with a plethora of options.
Muzik Faktry is nearing the final stages of development, however there are very likely bugs that still need to be squashed. If you find any please report them on the Muzik Faktry (https://codeberg.org/12bytes.org/muzikfaktry) code repository or leave a comment on the Muzik Faktry web page (https://12bytes.org/projects/muzik-faktry/).
For additional information see:
- Muzik Faktry: Processing music files on Linux: https://12bytes.org/projects/muzik-faktry/
- The default configuration file: /config/default.conf
- Batch processing
- Album splitting
- Extract, write and strip metadata
- Tag to file name, file name to tag
- Decompress multiple lossless formats and encode to FLAC
- Comprehensive integrity and metadata checking
- Generate image and/or video spectrograms of the audio frequencies
- Trim silence
- Volume normalization (ReplayGain)
- Optimization of file size
- Store and verify CRC hashes, find duplicate files
- Repair some specific problems
- Multiple configuration profiles
Bash v4.x or greater is required.
Several dependencies required by Muzik Faktry are already installed with most mainstream Linux-based desktop operating systems. The script will notify you of if any dependencies which are missing.
Important dependencies which may not be installed include:
- mediainfo (required) sources: https://github.com/MediaArea/MediaInfo
- shellcheck (highly recommended) sources and binaries: https://github.com/koalaman/shellcheck/releases
- shntool (required) sources: http://shnutils.freeshell.org/shntool/
For normalizing gain (volume) across albums or tracks, only metaflac is required (part of the FLAC package), however rsgain is highly recommended, followed by loudgain.
While optional, shellcheck is highly recommended since it is use to check the syntax of the active configuration file.
If a package is not available in your package repository you may drop precompiled binaries in the /bin folder and Muzik Faktry will use them. Make sure the files are executable:
$ chmod +x <file>
Unpack the archive somewhere where you have read, write and execute permissions, perhaps in your /home directory, and make the script executable:
$ chmod +x muzikfaktry.sh
The default configuration file is /config/default.conf. If you wish to make any changes to the configuration then make a copy of the file, giving it a new and descriptive name, and edit the copy. Upon running the script you will be prompted to choose a configuration file to load if there is more than one.
It is of particular importance to understand that, although the format of default.conf closely resembles an INI file (option=value), the configuration file is effectively part of the main script and therefore proper Bash syntax is crucial. It is for this reason that the installation of the shellcheck package is strongly recommended.
Most anything you want to know about the configuration options should be contained in the file.
IT IS STRONGLY SUGGESTED TO WORK ON COPIES OF YOUR SOURCE FILES. Muzik Faktry does not backup source files which it modifies. For example, when a compressed file is decompressed, the source file is deleted. The only exception is when splitting an album file in which case the the album and CUE sheet files are moved to the /backup directory.
The only file format that Muzik Faktry encodes to is FLAC, however the Decompress task is able to handle many lossless formats to get you going.
Other than the ability to read metadata for a wide variety of formats, there is no support for the MP3 format.
During the course of operations your input will be solicited, often in the form of a question. A typical reply for a yes or no type question might require a "Y" or "N" key press (lower case). Appended to the end of such a question might be "[Y/n]" or "[y/N]" where the default choice is indicated by the upper case letter. To select the default choice you may of course press the corresponding key (lower case), or the Enter/Return key, or most any other key.
Files which are discarded, either by the user or automatically, are moved to a sub-folder of the /discard directory.
To begin a session, copy (not move!) your source files to the /working directory, then open a terminal and cd to the /muzikfaktry directory and run the script:
Personally i like to open my file manager to the /working directory and move it to one side of my screen, then open a terminal and move it to the other side. This allows me to watch what's happening in the /working directory as files are processed.
/backup : album files are moved here after splitting /bin : precompiled dependencies may be dropped here if not installed /config : holds one or more configuration files /discard/failed_crc : files which failed CRC hash verification /discard/metadata_issues : files with repairable metadata issues /discard/minor_issues : files with misc. repairable issues (other than metadata) /discard/serious_issues : typically these are files which failed a test /discard/unrepairable : files which are not likely to be repairable /discard/user_discard : files which were discarded by the user /finished : finished files ready to be added to your collection /holding : general purpose convenience folder (not used by the script) /logs : session log files /metadata : metadata files created using the 'Save Metadata' task /spectro : PNG spectrograms created using the 'Spectrogram' task /working : files being actively worked on
Tasks should generally be run in order for each task that is necessary. A typical scenario might be where you start with FLAC tracks or an album file and a CUE sheet in which case you might want to consider the following tasks:
1. Set File Permissions : if necessary 2. Split An Album : if necessary 3. Save Metadata : if you wish to reuse the original metadata 4. Tags To File : optional 5. Format File Names : optional 6. Decompress : required to take full advantage of the Integrity Check task 7. Integrity Check : suggested file integrity check 8. Spectrogram : suggested unless you know the source files are of good quality 9. Trim Silence : if necessary 10. Encode : to FLAC 11. File To Tags : (or Restore Metadata) to write missing metadata 12. Write Tags : optional 13. Save Metadata : optional, if you wish to reuse the potentially modified metadata 14. Normalize Gain : recommended 15. Optimize : recommended if you want the most compact files 16. Metadata Check : suggested, this checks both file integrity and metadata 17. Duplicates & Hashes : optional, find duplicate files, testing if files have been altered, update CRC hash
Each time a task is run you will be asked if you want to batch process all files in the /working directory. If you are new to Muzik Faktry it is suggested to avoid batch processing until you become familiar with it.
It is suggested to work with one file type at a time to avoid confusion.
When splitting album files the album must be accompanied by a CUE file of the same name. For a file name of "album.flac" both "album.cue" and "album.flac.cue" are acceptable. Although multiple albums can be split in batch mode, consider splitting only one at a time in order to avoid file name collisions. Gap files (*.pregap.wav) are automatically deleted.
Decompressing compressed files will result in the loss of all metadata such as artist, title, album art, etc.. Text metadata can be saved by running the Save Metadata task prior to decompression, then written back using the the Restore Metadata task after encoding the file. If some or all of the metadata you require is contained in the file name (artist, title, etc.), perhaps after running the Tag To File Name and Format File Names tasks, it can be written back later using the File Name To Tag task.
If the source file is known to be a good FLAC file and you only want to change something such as the compression ratio, the Encode task will non-destructively re-encode the file using the settings defined in the configuration file.
The Normalize Gain task uses one of three programs to write ReplayGain information to the files depending on what programs are installed. The script will look for rsgain first, followed by loudgain, and finally metaflac (a component of the FLAC package). Writing ReplayGain information is non-destructive and the information can be removed at any time using the Strip Metadata task.
The Integrity Check and Metadata Check tasks are largely responsible for flagging files with problems. The Metadata Check task is the same as the Integrity Check task with extra metadata tests appended to it. Other than the optional Duplicates & Hashes task, the Metadata Check task is the final task that should be run before adding the files to your collection.
In addition to informational messages, tasks can report 5 different error levels when a problem is detected with a file:
"ERROR" level errors generally indicate that the file was not able to be processed by an executable. This likely due to a damaged file, however specifying incorrect options for an executable in the configuration file could also cause this error. These files are moved to the /discard/unrepairable directory.
"WARNING" level errors generally indicate that the file failed a test based on the criteria specified in the configuration file or that which is hard-coded in the script. These files are moved to the /discard/serious_issues directory.
"NOTICE" level errors indicate problems which are repairable using another task. Such files may have header issues, a problematic file name, etc.. These files are moved to the /discard/minor_issues directory.
"METADATA" level errors indicate problems which are repairable using another task. Such issues may indicate missing tags, excess metadata, etc.. These files are moved to the /discard/metadata_issues directory.
"CRC" level errors indicate the file failed a CRC hash check meaning that it was modified in some way since the previous hash check. These files are moved to the /discard/failed-crc directory.
A file which produced no errors but which is discarded by the user will be moved to the /discard/user_discard directory.
Tags can be read from many formats but may only be written to FLAC files. Any of the tags defined in the Vorbis I format specification may be written (see: https://xiph.org/vorbis/doc/v-comment.html).
Most tagging tasks should be self-explanatory with the possible exception of the Save Metadata and Restore Metadata tasks. The Save Metadata task will attempt to read the values for each tag specified in the configuration file for all files in the /working directory and write this information to text files in the /metadata directory, each file having the same name as the base file name of music file (the extension is ignored). These files are overwritten each time the task is run with files having the same name as the metadata files. This information can then be written to FLAC files using the Restore Metadata task. One possible use for this is to transfer metadata across different file formats, such as when upgrading from the MP3 to the FLAC format.
A predefined Muzik Faktry comment tag with version and date information may be written to your files in batch mode. This could be useful should you wish to reprocess your music files when a newer version of the script is released with a different feature set.
The purpose of the Spectrogram task is to create a visual representation of the audio frequency spectrum which can be used to help determine whether the audio is actually lossless or whether it was up-scaled/up-sampled from a lossy format, such as when some genius transcodes an MP3 to FLAC. More information is available on the web, including on the Muzik Faktry page at https://12bytes.org/projects/muzik-faktry/.
Spectrogram are stored as PNG image files in the /spectro directory if the Spectrogram task is run in run batch mode. The file names will be the same as the audio file with an added PNG extension. If batch mode is not active then each image is opened for previewing as it is generated and no image files are written to disk. Additionally you may view an audio/video spectrogram of the file.
Duplicates & Hashes
The Duplicates & Hashes task performs different operations depending on the selected mode. Regardless of the operating mode, any newly found file will have its name and CRC hash added to the checksums.txt file. If the file will be altered then a copy of checksums.txt named checksums.txt.bak is made overwriting the last backup. This task is always run in batch mode. Beyond that the operating modes perform the following:
Verify CRC: Verifies CRC hashes to find any files which have been unintentionally altered in any way, such as by "bit rot" (see: https://en.wikipedia.org/wiki/Data_degradation), unintentional metadata changes, etc..
Update CRC: Updates any file hashes in the checksums.txt file which have changed. This mode is useful if the files have been intentionally altered, such as a change to the metadata.
Find Duplicates: 1) Find potential duplicates using fuzzy file name matching. File names are stripped to a minimum in order to eliminate any potentially unimportant information including numbers, dots, hyphens, spaces and bracket pairs. 2) Compare file sizes to find any exact size matches. 3) Compare CRC hashes to find any bit-identical files.
If a file requires repairing it may be best to locate another copy. Problems such as low sample or bit rates, poor encoding methods, damaged audio or poor sound quality cannot be repaired. That said, there are some trivial issues that can be resolved such as stripping extra RIFF chunks, making headers canonical and sector boundary issues if you're writing tracks to CD. Note that files which report an "ERROR" level error when running one of the integrity check tasks are likely not repairable.
When repairing a file it will first be decompressed to WAV format if necessary. Choose only the necessary repair task(s) based upon the error messages that resulted from a prior Integrity Check task since subjecting the file to a repair option which is not fitting of the problem may irreparably damage the file.
Repairing WAVE Extensible format files (0xfffe) with non-canonical headers will result in a broken file that cannot be decoded, therefore the canonical header check is automatically disabled for this format. This issue is resolved if the file is transcoded to the FLAC format.
Though not used by Muzik Faktry, you may find the wavfix tool helpful for repairing issues with WAV files if you don't intend to encode them to FLAC or if FLAC encoding fails (see: https://github.com/agfline/wavfix).
Evoking this choice in the Main Menu will accomplish the following:
- Prompt to move any files in /working to the /finished directory. These files should be ready to add to your collection.
- If the /working directory is empty and there are files in the /holding or /discard/metadata_issues or /discard/minor_issues directories, you will be prompted to move them to the /working directory.
COMMAND LINE OPTIONS
Run Muzik Faktry with '-h' to see the available options:
$ ./muzikfaktry.sh -h
As i started listening to music on a PC i became more attuned to the audio quality and the metadata that was included with the files. At the time i was using that other OS and listening to MP3's, and so i built a tool chain for processing my music which checked for errors, added ReplayGain information, etc.. I was happy with the process i created but then i moved to GNU/Linux and wanted to recreate my tool chain the best i could, thus MP3 Factory was born which was the original name of this script. Eventually i figured out that the MP3 format is largely obsolete crap and difficult to work with and so i started focusing on producing lossless files in the FLAC format which led to the re-branding of the script. Eventually i dropped virtually all support for the dying MP3 format.
Muzik Faktry has since evolved into a fairly comprehensive tool that greatly exceeded both my original expectations and the vastly inferior tool chain i constructed on Windows, plus it's been a great project for learning the many intricacies of Bash shell scripting. Whether Bash is an appropriate language to code such a script is certainly debatable, however it's currently the only Linux capable scripting language i'm familiar with.