labunix's blog

labunixのラボUnix

awscliでS3に静的コンテンツを作ってパブリック公開の許可と拒否を試す。

■awscliでS3に静的コンテンツを作ってパブリック公開の許可と拒否を試す。
 dpkgだと1.11.13、ローカルのパス指定実行だと1.16.106の環境。
 ユーザとかグループとか書名付きURLは今回は触れない。

$ aws --version
aws-cli/1.11.13 Python/3.5.3 Linux/4.9.0-8-amd64 botocore/1.4.70

$ ./.local/bin/aws --version
aws-cli/1.16.106 Python/2.7.13 Linux/4.9.0-8-amd64 botocore/1.12.96

■バケットの作成と削除

$ aws s3 mb s3://myw3
make_bucket: myw3

$ aws s3 ls
2019-03-04 00:11:16 myw3

$ aws s3 rb s3://myw3
remove_bucket: myw3

■コンテンツを用意する。

$ sudo apt-get install -y txt2html
$ echo 'Hello World' | txt2html | tee index.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title></title>
<meta name="generator" content="HTML::TextToHTML v2.51"/>
</head>
<body>
<p>Hello World</p>

</body>
</html>

■バケットを作成してアップロード、ダウンロード

$ aws s3 mb s3://myw3
make_bucket: myw3

$ aws s3 ls
2019-03-04 00:21:00 myw3

$ aws s3 cp index.html s3://myw3
upload: ./index.html to s3://myw3/index.html

$ aws s3 ls s3://myw3
2019-03-04 00:21:44        286 index.html

$ aws s3 cp s3://myw3/index.html index.html2
download: s3://myw3/index.html to ./index.html2

$ ls index.html* | awk '{print "md5sum "$1}' | sh
2a1e0a19087fed1b10a970d9a04c3e40  index.html
2a1e0a19087fed1b10a970d9a04c3e40  index.html2

$ aws s3 website s3://myw3/ --index-document index.html

■バケットACLを使って公開する。

$ w3m -dump https://s3-ap-northeast-1.amazonaws.com/myw3/index.html?versionId=null | html2text | head -1
<?xml version="1.0" encoding="UTF-8"?> AccessDeniedAccess

$ aws s3api put-bucket-acl --bucket myw3 --grant-read uri=http://acs.amazonaws.com/groups/global/AllUsers

$  w3m -dump https://s3-ap-northeast-1.amazonaws.com/myw3/index.html?versionId=null | html2text | head -1
Hello World

$ aws s3api get-bucket-acl --bucket myw3 --query 'Grants[]' --output text
READ
GRANTEE	Group	http://acs.amazonaws.com/groups/global/AllUsers

$ aws s3api put-bucket-acl --acl private --bucket myw3

$ aws s3api get-bucket-acl --bucket myw3 --query 'Grants[]' --output text | \
    awk '{if(NF>2){gsub("[a-z-]*","dummyuser",$2)}}{if(NF>3){gsub("[a-f0-9]","x",$3)}}{print $0}'
FULL_CONTROL
GRANTEE dummyuser xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx CanonicalUser

■オブジェクトACLを使って公開する。

$ aws s3api put-object-acl --acl public-read --bucket myw3 --key index.html

$ aws s3api get-object-acl --bucket myw3 --key index.html --query 'Grants[].Grantee[].[URI,Type]' --output text
None	CanonicalUser
http://acs.amazonaws.com/groups/global/AllUsers	Group

$ w3m -dump https://s3-ap-northeast-1.amazonaws.com/myw3/index.html?versionId=null | html2text | head -1
Hello World

$ aws s3api put-object-acl --acl private --bucket myw3 --key index.html

$ w3m -dump https://s3-ap-northeast-1.amazonaws.com/myw3/index.html?versionId=null | html2text | head -1
<?xml version="1.0" encoding="UTF-8"?> AccessDeniedAccess

■バケットポリシーを使って公開する。
 元が空なので、公式ドキュメントを参考に。

$ aws s3api get-bucket-policy --bucket myw3
An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicy operation: The bucket policy does not exist

 バケットポリシーの例
 https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/dev/example-bucket-policies.html

$ sed -i -e 's/examplebucket/myw3/g' myw3.json ;cat myw3.json
{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"AddPerm",
      "Effect":"Allow",
      "Principal": "*",
      "Action":["s3:GetObject"],
      "Resource":["arn:aws:s3:::myw3/*"]
    }
  ]
}

$ aws s3api put-bucket-policy --bucket myw3 --policy file://$(pwd)/myw3.json
$ aws s3api get-bucket-policy --bucket myw3 --output text | jq . --indent 4
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AddPerm",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::myw3/*"
        }
    ]
}

$ w3m -dump https://s3-ap-northeast-1.amazonaws.com/myw3/index.html?versionId=null | html2text | head -1
<?xml version="1.0" encoding="UTF-8"?> AccessDeniedAccess

$ w3m -dump https://s3-ap-northeast-1.amazonaws.com/myw3/index.html | html2text | head -1
Hello World

$ sed -i -e 's/Allow/Deny/g' myw3.json ;cat myw3.json
{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"AddPerm",
      "Effect":"Deny",
      "Principal": "*",
      "Action":["s3:GetObject"],
      "Resource":["arn:aws:s3:::myw3/*"]
    }
  ]
}

$ aws s3api put-bucket-policy --bucket myw3 --policy file://$(pwd)/myw3.json

$ aws s3api get-bucket-policy --bucket myw3 --output text | jq . --indent 4
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AddPerm",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::myw3/*"
        }
    ]
}

$ w3m -dump https://s3-ap-northeast-1.amazonaws.com/myw3/index.html | html2text | head -1
<?xml version="1.0" encoding="UTF-8"?> AccessDeniedAccess

■お片付けとしてのバケット削除

$ aws s3 rb s3://myw3
remove_bucket failed: s3://myw3 An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: The bucket you tried to delete is not empty

$ aws s3 rb s3://myw3 --force
delete: s3://myw3/index.html
remove_bucket: myw3

$ aws s3 ls