1gsutil mb -p codes-20180829 -c standard -l us-central1 -b ON --pap enforced gs://sushanth-cs-test
Note : This is a Class A operation
1gsutil ls -r -l gs://sushanth-cs-test/**
Below command initiates a parallel upload via rsync and recursively uploads all folder and sub-folders to GCS
1gsutil -m rsync -r -d ./ gs://sushanth-cs-test
Below command initiates a parallel upload excluding the pattern via rsync and recursively uploads all folder and sub-folders to GCS. It excludes folders like node_modules, .git, .cache and ...
1gsutil -m rsync -r -d -x ".*node_modules.*$|.*\.git.*$|.*\.cache.*$|.*Eclipse Java.*$" ./ gs://sushanth-cs-test/**
If you get
Caught non-retryable exception while listing... error like below that because in windows MAX_PATH defaults to 260 characters.
1Caught non-retryable exception while listing file://.\: [WinError 3] The system cannot find the path specified: '.\\3. Java\\Eclipse Java\\eclipse-java-2021-06-R-win32-x86_64\\eclipse\\plugins\\org.eclipse.justj.openjdk.hotspot.jre.full.win32.x86_64_16.0.1.v20210528-1205\\META-INF\\maven\\org.eclipse.justj\\org.eclipse.justj.openjdk.hotspot.jre.full.win32.x86_64\\pom.properties'2CommandException: Caught non-retryable exception - aborting rsync
To increase that limit, you have two options,
Update Windows Registry
1Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled2Type: REG_DWORD3Value: 1
Tried this and it works.
Specify UNC Paths. Prefix paths with "\?\". This changes maximum path names to 32,767 characters.
Did not experiment on this. Good to know there is another option.
If you want to download the entire bucket, you can do like this. It will create a folder with the buckets name in the current folder.
1-- Parallel and Recursive flags are mentioned in below command2gsutil -m cp -r gs://sushanth-cs-test ./
If you want to download to a specific folder, that folder needs to exist in target. Here i have created a new folder called t1
1-- Parallel and Recursive flags are mentioned in below command2gsutil -m cp -r gs://sushanth-cs-test ./t1