BioMaj - Use case : Write a post-process

Use case : Write a post-process

Even though BioMAJ is shipped with some post-processes, you might need to write your owns.

Here are some points that will help you in the process :

BioMAJ can run scripts or binaries
The executables must be placed in the directory BIOMAJ_ROOT/conf/process
You have access to some environment variables that are set by BioMAJ. The exhaustive list is available in the user guide, but the most useful are :

$datadir : root directory for all production directories
$dirversion : production directory
The current location of the files in the post-process stage can be obtained by : $datadir/$dirversion/future_release
The downloaded raw data is available in the directory : $datadir/$dirversion/future_release/flat

Basic interaction : BioMAJ retrieves the return value at the end of the program. If no value is returned or the value is null, the execution of the program is considered to be successful. If any other value is received, an error is returned and the worklow is stopped.
Advanced interaction : You can specify file dependencies if your messages have a specific structure :

$PP_DEPENDENCYabsolute_path_of_your_file : Specifies that this file must be watched by BioMAJ. As a result, when checking the integrity of the repository, if the file is somehow missing, the process will be rerun.
$PP_VOLATILE_DEPENDENCYabsolute_path_of_your_file : Specifies a file that must be deleted at the end of the workflow.

An example of a script that concatenates several files into one, and adds the resulting file as a dependency :

workdir=$datadir/$dirversion/future_release/flat

cd $workdir;
for file in $@

do
files=$files" "$file

done
cat $files > $workdir/output ;
echo $PP_DEPENDENCE$workdir/output;