* Commands are given to storage daemon over a socket
* Implemented as a queue (commands are being queued)
* Multiple worker threads handle the commands.
* Based on policies the storage daemon will know what todo
What is the main purpose of the storage daemon ¶
processing of new or modified files on OSIS (V1) ¶
* copy new or modified file on OSIS to one or more of the storagepools available depending on the policy.
* While copying do optionally compression & encryption if necessary based on policy.
* Calculate the CRC while doing copying.
* Put a link on OSIS server towards the copied file on primary storagepool.
* If a slave OSIS server is present talk to daemon there and make sure a link is create there as well. (V2)
o Create a link to an existing file on storagepool (for slave operation)
delete OSIS file (V1.5) ¶
* Delete a file on the storagepool(s)
* Remove link and delete the metadata in the lucene DB.
Check health of OSIS file (V2) ¶
* Check health of file on storagepools.
* Multiple levels of checks are possible (for now basic & extended)
o In basic mode only size & file exists need to be checked,
o In extended mode, check if the file exists and read the file and verify against CRC (if encryption was used, create also CRC after encryption and/or compression)
+ more levels will be defined and included in later versions
Repair OSIS file (V1.5) ¶
* Make sure appropriate number of file copies exists on the storagepools (according to the policy), if not copy back.
redundant configuration of OSIS ¶
If installed in redundant mode every handled file on primary node will also be handled on slave.
Primary (V1) ¶
* copy file to storagepool(s) while respecting the policies
* optionally encrypt
* optionally compress
* calculate CRC
* create symlink between primary OSIS & primary storagepool
* convert supported formats to text and populate local lucene for full text search
* put all metadata in lucene
* tell slave to process document as well (as parameters all metada, full text (if supported doc), …)
Slave (V2) ¶
* create symlink between slave OSIS & primary storagepool?
* populate full text lucene as well as metadata lucene
* written in java
project1/coder1: working prototype storage daemon functionality:
* see [url removed, login to view] (should be ready before 24-8-2006 evening)