Join the social network of Tech Nerds, increase skill rank, get work, manage projects...
 
  • Syncing to S3

    • 0
    • 0
    • 0
    • 0
    • 0
    • 0
    • 0
    • 0
    • 227
    Comment on it

    Syncing Same Folder to S3 Bucket

    Hi Friends,
    Suppose you have some folder structure in your local and you want to upload the same folder structure to your amazon s3 bucket. Then you need to follow these steps.
    Step 1: Set up aws cli in your application with the bucket you want to access. Here is the link given below that will guide you to aws cli setup and few basic commands.
    How to install, configure & use AWS client in ubuntu
    Step 2: Go to your folder's root which you want to sync on s3. after configuring your aws with the specific account

    	>> aws configure
    	AWS Access Key ID [None]: XXXXXXXXXXXXXEXAMPLE
    	AWS Secret Access Key [None]: XXXXXXXXXXXXXXXXXXXXXXXXXSAMPLEKEY
    	Default region name [None]: 
    	Default output format [None]: 
    
    	>> cd root_folder
    


    Step 3: Now just type this syncing command and the same structure folder will move to s3 bucket

    	>> aws s3 sync . s3://your-bucket-name
    	# => Output
    	upload: folder_1/image1.png to s3://your-bucket-name/folder_1/image1.png
    	upload: folder_1/image2.png to s3://your-bucket-name/folder_1/image2.png
    	upload: folder_2/image1.png to s3://your-bucket-name/folder_2/image1.png
    	upload: folder_2/image2.png to s3://your-bucket-name/folder_2/image2.png
    

    Similarly if you want to sync or download the entire s3 bucket on your local you can do just the reverse

    	>> aws s3 sync s3://your-bucket-name . 
    	# => Output
    	download: s3://your-bucket-name/folder_1/image1.png to folder_1/image1.png
    	download: s3://your-bucket-name/folder_1/image2.png to folder_1/image2.png
    	download: s3://your-bucket-name/folder_2/image1.png to folder_2/image1.png
    	download: s3://your-bucket-name/folder_2/image2.png to folder_2/image2.png
    

    For copying one bucket content to another

    	>> aws s3 sync s3://your-bucket-1 s3://your-bucket-2
    	# => Output
    	copy: s3://your-bucket-1/folder_1/image1.png to s3://your-bucket-2/folder_1/image1.png
    	copy: s3://your-bucket-1/folder_1/image2.png to s3://your-bucket-2/folder_1/image2.png
    	copy: s3://your-bucket-1/folder_2/image1.png to s3://your-bucket-2/folder_2/image1.png
    	copy: s3://your-bucket-1/folder_2/image2.png to s3://your-bucket-2/folder_2/image2.png
    

    For entirely syncing the exact copy of the source, there is --delete prefix. By using it, the already existing content in the destination will be removed, if it is not present in the source. Suppose in the destination  s3 bucket, there was already an image with name image5.png, and it is not in the source local folder then it will be removed from s3.

    	>> aws s3 sync . s3://your-bucket-name
    	# => Output
    	upload: folder_1/image1.png to s3://your-bucket-name/folder_1/image1.png
    	upload: folder_1/image2.png to s3://your-bucket-name/folder_1/image2.png
    	upload: folder_2/image1.png to s3://your-bucket-name/folder_2/image1.png
    	upload: folder_2/image2.png to s3://your-bucket-name/folder_2/image2.png
    	delete: s3://your-bucket-name/image5.png
    

    There is another prefix that can be used is --exclude,  it will exclude those files, that are mentioned after it, Parameters to exclude can be a file name, folder name or some extensions. 

    	## To exclude all the contents for folder_2
    	>> aws s3 sync s3://your-bucket-name .  --exclude "*folder_2/*" 
    	# => Output
    	download: s3://your-bucket-name/folder_1/image1.png to folder_1/image1.png
    	download: s3://your-bucket-name/folder_1/image2.png to folder_1/image2.png
    
    	## To exclude all the png images
    	>> aws s3 sync s3://your-bucket-name . --exclude "*.png"
    

     

    Hope you liked it. For more blogs like this. Click here.

 0 Comment(s)

Sign In
                           OR                           
                           OR                           
Register

Sign up using

                           OR                           
Forgot Password
Fill out the form below and instructions to reset your password will be emailed to you:
Reset Password
Fill out the form below and reset your password: