Sisyphe

Sisyphe is a generic Golang recursive folder analyser terminal application

Requirements

Tested with Golang 1.17

Works on Linux/OSX/Windows

Mount a corpus folder and :

docker-compose up -d
docker exec -it  ./main.go -n corpusname -c corpuspath

Install it

1. Download the latest Sisyphe version
2. Just do : go build .
3. ... that's it.

Help

./app --help Will output help

Options

--help                          Output usage
-n, --corpusname <name>         Corpus name
-c, --config-dir <path>         Configuration folder path
-w, --counting-word             Counting word on pdf
-o, --output-dir <dirname>      Output directory where results are written
-q, --quiet                     Silence output
-r, --restore                   Read only unprocessed files

How it works ?

Just start Sisyphe on a folder with any files in it.

go run *.go -n corpusname ~/Documents/customfolder/corpus

go run *.go -n corpusname -c ~/Documents/customfolder/corpusResources ~/Documents/customfolder/corpus

Sisyphe is now working in background with all your computer thread. Just take a coffee and wait , it will prevent you when it's done :)

The results of sisyphe are present @ sisyphe/out/{timestamp}-corpusname/ (errors,info,duration..)

For a control panel & full binded app, go to Sisyphe-monitor sisyphe has a server that allows to control it and to obtain more information on its execution. Simply run the server with npm run server to access these features

Modules

There is a list of default modules (focused on xml & pdf).

Those URL NEED to be updated when merge branch will be ok.

• FILETYPE Will detect mimetype,extension, corrupted files..
• PDF Will get info from PDF (version, author, meta...)
• XML Will check if it's wellformed, valid-dtd's, get elements from balises ...
• XPATH Will generate a complete list of xpaths from submitted folder
• OUT Will export data to json file & ElasticSearch database