distributed-blockchain-based-secure-file-dedupe

• Searching is Distributed, Block and Access List for each upload is unique and it is stored in a single VM to reduce overhead in the metadata (blockchain based).

• Instructions for 4 nodes (including a storage node) docker swarm setup:

  1. get.docker.com .. install docker on all VMs (run both commands under "This script is meant for the quick & easy installation")

  2. ssh into all VMs with sudo access (or add docker usergroup)

  3. Run "docker swarm init" on the leader node.

  4. Run "docker swarm join-token manager", copy paste the output to all other VMs, so that all other nodes will be added to the swarm as manager.

  5. To confirm all the nodes present in swarm, run "docker node ls"

  6. Create a common network (attachable) among all nodes on the swarm using "docker network create --driver overlay --attachable dedupe"

  7. To confirm, run "docker network ls" on all the VMs and check the scope(==swarm) and driver(==overlay)

  8. Build and push the image "docker-compose build", "docker-compose push".

  9. Pull the image for dedupe nodes separately "docker image pull abhishekvtangod/blockdedupe"

  10. Pull the image for cloud node "docker image pull abhishekvtangod/blockdedupe_cloud"

  11. Run the containers on all VMs.

      • docker container run -d --name node1 --network dedupe --env PYTHONUNBUFFERED=1 --env node_id=1 -p 5001:5000 --hostname localhost abhishekvtangod/blockdedupe
      • docker container run -d --name node2 --network dedupe --env PYTHONUNBUFFERED=1 --env node_id=2 -p 5002:5000 --hostname localhost abhishekvtangod/blockdedupe
      • docker container run -d --name node3 --network dedupe --env PYTHONUNBUFFERED=1 --env node_id=3 -p 5003:5000 --hostname localhost abhishekvtangod/blockdedupe
      • docker container run -d --name cloud --network dedupe --env PYTHONUNBUFFERED=1 -p 5000:5000 --hostname localhost abhishekvtangod/blockdedupe_cloud

  12. Access all the urls from individual ips of VM with appropriate ports(change port mappings according to your ease and avoid confusions).

Pure Blockchain Implementation: https://github.com/lanxeon