{"id":81,"date":"2021-12-21T17:30:25","date_gmt":"2021-12-21T17:30:25","guid":{"rendered":"https:\/\/danielpgleason.com\/?p=81"},"modified":"2021-12-21T17:32:19","modified_gmt":"2021-12-21T17:32:19","slug":"perforce-commit-master-bash-script","status":"publish","type":"post","link":"https:\/\/danielpgleason.com\/index.php\/2021\/12\/21\/perforce-commit-master-bash-script\/","title":{"rendered":"Perforce Commit Master Bash Script"},"content":{"rendered":"\n<p>So I was recently commissioned to do some work on writing up a perforce installation script and also a script to create backups to install from s3 tarballs. This is the end result of the bash script I created. There are some issues with regards to chmod permissions on the system. However, the script is fully functional. <\/p>\n\n\n\n<p>Here is a list of commands:<\/p>\n\n\n\n<p>.\/p4d_install.sh -c 1 &#8211; Uninstalls everything that the script had installed previously<\/p>\n\n\n\n<p> .\/p4d_install.sh  -r -b s3:\/\/bucket-url -p superuserpassword &#8211; Installs perforce from an s3 bucket tarball, it will grab the latest tarball and use this for installation. <\/p>\n\n\n\n<p> .\/p4d_install.sh -p superuserpassword &#8211; will install perforce without a backup, clean installation<\/p>\n\n\n\n<div class=\"hcb_wrap\"><pre class=\"prism line-numbers lang-bash\" data-file=\"p4d_install.sh\" data-lang=\"Bash\"><code>#!\/bin\/bash\n: &#39; @Author Daniel Gleason\n\t@Email danielpgleason@icloud.com\n\t@Script_Name p4d_install.sh\n\t@Date 12\/11\/2021\n\n\t-- Developer Notes --\n\tMost if not all of the permissions of the perforce files are owned by the perforce user\n\tUse sudo -u perforce CMD_LINE for most commands\n\tp4dctl commands:\n\tsudo -u perforce p4dctl stop master\n\tsudo -u perforce p4dctl start master\n\tsudo -u perforce p4dctl status master\n&#39;\nset -e\n\n\n# Prints error to stderr in red\nfunction print_error {\n\tlocal RED=&#39;\\033[0;31m&#39;\n\tlocal NC=&#39;\\033[0m&#39;\n\tprintf &quot;${RED}$1${NC}\\n&quot; 1&gt;&2;\n}\n\n\n# Prints out the usage of the script\nfunction usage() {\n\tlocal usage=&quot;Usage: $0 [-r &lt;1|0&gt; restore_from_s3 number] [-k aws_access_key str] [-s aws_secret_key str] [-d aws_region str] [-b s3_bucket_path str] [-p perforce_password str] [-c &lt;1|0&gt; clean_install]&quot;;\n\tprint_error $usage\n\texit 1;\n}\n\n\n# Removes any potential installation residue from previous runs of the script\nfunction clean() {\n\tsudo slay perforce 2&gt; \/dev\/null\n\tsudo rm -rf \/perforce\n\tsudo apt-get --purge remove helix-p4d -y\n\tsudo apt-get remove awscli -y\n\tsudo apt-get autoremove -y\n\tsudo rm \/etc\/perforce\/p4dctl.conf.d\/master.conf 2&gt; \/dev\/null\n\tsudo userdel perforce -f 2&gt; \/dev\/null\n\tsudo delgroup perforce 2&gt; \/dev\/null\n}\n\n\n# Get inputs\nwhile getopts &quot;r:k:s:d:b:p:c:&quot; flag; do\n\tcase &quot;${flag}&quot; in\n\t\tr)\n\t\t\texport restore=${OPTARG}\n\t\t\t;;\n\t\tk)\n\t\t\texport AWS_ACCESS_KEY_ID=${OPTARG}\n\t\t\t;;\n\t\ts)\n\t\t\texport AWS_SECRET_ACCESS_KEY=${OPTARG}\n\t\t\t;;\n\t\td)\n\t\t\texport AWS_DEFAULT_REGION=${OPTARG}\n\t\t\t;;\n\t\tb)\n\t\t\texport S3_BUCKET_PATH=${OPTARG}\n\t\t\t;;\n\t\tp)\n\t\t\texport PERFORCE_PASSWORD=${OPTARG}\n\t\t\t;;\n\t\tc)\n\t\t\texport CLEAN=1\n\t\t\tclean\n\t\t\t;;\n\t\t*)\n\t\t\tusage\n\t\t\t;;\n\tesac\ndone\n\n\nfunction install_p4d() {\n\tlocal ubuntu_distro=`lsb_release --codename --short` # focal, xenial, etc\n\twget -qO - https:\/\/package.perforce.com\/perforce.pubkey | sudo apt-key add -\n\tsudo rm \/etc\/apt\/sources.list.d\/perforce.list 2&gt; \/dev\/null # Delete perforce apt file if it already eixsts, ignore the error output.\n\techo &quot;deb http:\/\/package.perforce.com\/apt\/ubuntu $ubuntu_distro release&quot; | sudo tee -a \/etc\/apt\/sources.list.d\/perforce.list\n\tsudo apt-get update\n\tsudo apt-get install helix-p4d -y\n}\n\n\nfunction configure_p4d() {\n\tsudo mkdir \/perforce 2&gt; \/dev\/null\n\tsudo \/opt\/perforce\/sbin\/configure-helix-p4d.sh -n master -p ssl:1666 -r \/perforce -u perforce -P $PERFORCE_PASSWORD\n\tsudo chown -R perforce.perforce \/perforce\n\tsudo usermod -aG perforce ubuntu # Add ubuntu user to perforce group\n\tsudo -u perforce mkdir \/perforce\/depots\n\t# All depot data will be located in \/perforce\/depots\n\tp4 configure set master#server.depot.root=\/perforce\/depots\n\tsudo chmod -R 777 \/perforce\/\n\tsudo chmod -R 770 \/perforce\/depots\n}\n\n\n# Just some sueful tools I like to use\nfunction install_prerequisites() {\n\tsudo apt-get install awscli -y\n\tsudo apt-get install slay -y\n\tsudo apt-get install mlocate -y\n}\n\n\n# Produces credentials file in ~\/.aws\/credentials and config file in ~\/.aws\/config used by aws cli\nfunction setup_aws_cli() {\n\taws configure set aws_access_key_id $AWS_ACCESS_KEY_ID\n\taws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY\n\taws configure set default.region $AWS_DEFAULT_REGION\n}\n\n\nfunction restore_from_s3_backup() {\n\techo &#39;Restoring from s3 backup&#39;\n\tsudo mkdir \/perforce\/backups 2&gt; \/dev\/null\n\tsudo chown perforce.perforce \/perforce\/backups\n\tsudo -u perforce p4dctl stop master\n\t\n\t# Download from s3 into \/perforce\/backups\/\n\tpushd \/perforce\/backups\/\n\t\tlocal most_recent_checkpoint_s3=$(aws s3 ls $S3_BUCKET_PATH | sort | tail -n 1 | awk &#39;{print $4}&#39;)\n\t\tlocal backup_dir=`pwd`\n\t\tsudo -u perforce aws s3 cp $S3_BUCKET_PATH\/$most_recent_checkpoint_s3 .\n\t\tsudo -u perforce tar -xf $most_recent_checkpoint_s3\n\t\tsudo -u perforce rsync -a perforce\/depots\/ \/perforce\/depots\/ # Move the depot files over\n\t\t\n\t\t# Get checkpoint file we just extracted from the tarball\n\t\tpushd perforce\/backups\/\n\t\t\tlocal most_recent_checkpoint_file=`ls -t master.ckp.*.gz | head -1`\n\t\tpopd\n\n\t\t# Restore the checkpoint\n\t\tpushd \/perforce\/root\/\n\t\t\tsudo rm -rf \/perforce\/root\/db.*\n\t\t\tsudo -u perforce p4d -r . -jr $backup_dir\/perforce\/backups\/$most_recent_checkpoint_file\n\t\tpopd\n\n\t\t# Remove extracted files to keep things clean\n\t\tsudo rm -rf perforce\/\n\tpopd\n\n\t# Add P4PORT to \/etc\/perforce\/p4dctl.conf.d\/master.conf\n\tsudo sed -i &#39;\/P4SSLDIR\\s.*=\\s.*ssl\/a \\\\tP4PORT=ssl:1666&#39; \/etc\/perforce\/p4dctl.conf.d\/master.conf\n\n\t# Change permissions of db files\n\tsudo chmod -R 600 \/perforce\/root\/db.*\n\n\t# Generate new ssl certs\n\tpushd \/perforce\/root\/\n\t\tsudo chmod -R 0700 ssl\/\n\t\tsudo rm ssl\/privatekey.txt ssl\/certificate.txt\n\t\tP4SSLDIR=ssl sudo -Eu perforce p4d -Gc\n\tpopd\n\t\n\t# Start p4d\n\tsudo -u perforce p4dctl start master\n\n\t# Run p4 trust on the new server setup. \n\tp4 trust -f -y\n}\n\n\nfunction main() {\n\t\n\tif [ ! $CLEAN ]\n\tthen\n\t\tinstall_prerequisites\n\tfi\n\n\tif [ $PERFORCE_PASSWORD ]\n\tthen\n\t\tinstall_p4d\n\t\tconfigure_p4d\n\tfi\n\t\n\tif [ $AWS_ACCESS_KEY_ID ] && [ $AWS_SECRET_ACCESS_KEY ] && [ $AWS_DEFAULT_REGION ]\n\tthen\n\t\tsetup_aws_cli\n\tfi\n\n\tif [ $restore ] && [ $restore -eq 1 ]\n\tthen\n\t\tif [ $S3_BUCKET_PATH ]\n\t\tthen\n\t\t\techo Bucket Path: $S3_BUCKET_PATH\n\t\t\trestore_from_s3_backup\n\t\telse\n\t\t\tprint_error &quot;Error: restore flag must be 1 and s3_bucket_path must be set to restore from a backup.&quot;\n\t\t\tusage\n\t\tfi\n\tfi\n}\n\n\nmain<\/code><\/pre><\/div>\n\n\n\n<p>This next script is used in conjunction with the script above to create backups. You can run this script on a cron task and it will create a tarball and create a new perforce checkpoint and upload it to s3. It does not have functionality to setup an aws cli configuration. You will need to do that yourself, or give the aws instance the permissions it needs. I also hard coded the s3 backup location, you will need to change this to a variable and use getopts or just change the hard coded path for the bucket url. <\/p>\n\n\n\n<div class=\"hcb_wrap\"><pre class=\"prism line-numbers lang-bash\" data-file=\"create_backup.sh\" data-lang=\"Bash\"><code>#!\/bin\/bash\n: &#39; @Author Daniel Gleason\n\t@Email danielpgleason@icloud.com\n\t@Script_Name create_backup.sh\n\t@Date 12\/13\/2021\n&#39;\n\n\nfunction get_most_recent_checkpoint() {\n    pushd \/perforce\/backups\/ &gt; \/dev\/null\n        echo `ls -t master.ckp.*.gz | head -1`\n    popd &gt; \/dev\/null\n}\n\n\nfunction make_backup() {\n    pushd \/perforce &gt; \/dev\/null\n        mkdir backups 2&gt; \/dev\/null\n        sudo chown -R perforce.perforce backups\/\n        pushd backups\/\n            # Creates a checkpoint in \/perforce\/backups using the p4 database in \/perforce\/root and the journal in \/perforce\/root\n            sudo -u perforce p4d -r \/perforce\/root -J \/perforce\/root\/journal -z -jc `pwd`\/master\n\n            # Create tarball of depots and the checkpoint\n            local backup_name=&quot;$(date +&quot;%m.%d.%Y.%I.%M.%p&quot;)_backup.tar.gz&quot;\n            local most_recent_checkpoint=$(get_most_recent_checkpoint)\n            sudo -u perforce tar -Ppzcf $backup_name \/perforce\/depots\/ \/perforce\/backups\/$most_recent_checkpoint\n            aws s3 cp `pwd`\/$backup_name &quot; S3_LOCATION &quot;\n        popd &gt; \/dev\/null\n    popd &gt; \/dev\/null\n}\n\n\nmake_backup\n<\/code><\/pre><\/div>\n","protected":false},"excerpt":{"rendered":"<p>So I was recently commissioned to do some work on writing up a perforce installation script and also a script to create backups to install from s3 tarballs. This is the end result of the bash script I created. There are some issues with regards to chmod permissions on the system. However, the script is &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/danielpgleason.com\/index.php\/2021\/12\/21\/perforce-commit-master-bash-script\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Perforce Commit Master Bash Script&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-81","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/danielpgleason.com\/index.php\/wp-json\/wp\/v2\/posts\/81"}],"collection":[{"href":"https:\/\/danielpgleason.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/danielpgleason.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/danielpgleason.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/danielpgleason.com\/index.php\/wp-json\/wp\/v2\/comments?post=81"}],"version-history":[{"count":2,"href":"https:\/\/danielpgleason.com\/index.php\/wp-json\/wp\/v2\/posts\/81\/revisions"}],"predecessor-version":[{"id":83,"href":"https:\/\/danielpgleason.com\/index.php\/wp-json\/wp\/v2\/posts\/81\/revisions\/83"}],"wp:attachment":[{"href":"https:\/\/danielpgleason.com\/index.php\/wp-json\/wp\/v2\/media?parent=81"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/danielpgleason.com\/index.php\/wp-json\/wp\/v2\/categories?post=81"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/danielpgleason.com\/index.php\/wp-json\/wp\/v2\/tags?post=81"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}