{"id":3024,"date":"2015-03-23T10:00:34","date_gmt":"2015-03-23T01:00:34","guid":{"rendered":"http:\/\/www.skyarch.net\/blog\/?p=3024"},"modified":"2015-03-23T11:34:04","modified_gmt":"2015-03-23T02:34:04","slug":"%e3%80%90pukiwiki%e8%a8%98%e6%b3%95%e3%81%8c%e5%8b%95%e4%bd%9c%e3%81%97%e3%81%aa%e3%81%84%e3%81%ae%e3%81%a7%e6%8a%95%e7%a8%bf%e6%8c%ab%e6%8a%98%e3%80%91%e3%83%8b%e3%83%95%e3%83%86%e3%82%a3%e3%82%af","status":"publish","type":"post","link":"https:\/\/www.skyarch.net\/blog\/%e3%80%90pukiwiki%e8%a8%98%e6%b3%95%e3%81%8c%e5%8b%95%e4%bd%9c%e3%81%97%e3%81%aa%e3%81%84%e3%81%ae%e3%81%a7%e6%8a%95%e7%a8%bf%e6%8c%ab%e6%8a%98%e3%80%91%e3%83%8b%e3%83%95%e3%83%86%e3%82%a3%e3%82%af\/","title":{"rendered":"\u30cb\u30d5\u30c6\u30a3\u30af\u30e9\u30a6\u30c9\u30b9\u30c8\u30ec\u30fc\u30b8\u3067fluentd(td-agent)\u3092\u4f7f\u7528\u3059\u308b\u65b9\u6cd5."},"content":{"rendered":"

AWS\u3067\u306f\u30aa\u30fc\u30c8\u30b9\u30b1\u30fc\u30eb\u306e\u90fd\u5408\u306a\u3069\u3082\u3042\u308a\u3001\u30ed\u30b0\u3092fluentd(td-agent)\u3092\u7528\u3044\u3066Amazon S3\u3078\u7f6e\u304f\u4e8b\u304c\u6d41\u884c\u3063\u3066\u3044\u307e\u3059\u306d\u3002<\/p>\n

\u3057\u304b\u3057\u3001AWS\u3068\u540c\u3058\u624b\u9806\u3067td-agent\u3092\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3057\u3066\u3082\u3001\u30cb\u30d5\u30c6\u30a3\u30af\u30e9\u30a6\u30c9\u30b9\u30c8\u30ec\u30fc\u30b8\u3067\u306f\u5229\u7528\u3067\u304d\u307e\u305b\u3093\u3002
\n\u539f\u56e0\u306ftd-agent\u306baws-sdk\u304c\u540c\u68b1\u3055\u308c\u3066\u304a\u308a\u3001\u4fdd\u5b58\u5148\u306bAmazonS3\u304c\u6307\u5b9a\u3057\u3066\u3042\u308b\u70ba\u306b\u4ed6\u30af\u30e9\u30a6\u30c9\u3067\u306f\u30a8\u30e9\u30fc\u306b\u306a\u3063\u3066\u3057\u307e\u3044\u307e\u3059\u3002<\/p>\n

\u305d\u3053\u3067\u3001\u672c\u8a18\u4e8b\u3067\u306f\u3053\u306e\u5bfe\u7b56\u65b9\u6cd5\u306b\u3064\u3044\u3066\u8a18\u8f09\u3057\u305f\u3044\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n

OS<\/h1>\n

CentOS6.4
\nApache2.2 td-agent1.1.24<\/p>\n

Apache\u306e\u30ed\u30b0\u5f62\u5f0f\u306f\u6a19\u6e96\u306ecombined
\n\u30ed\u30b0\u4fdd\u5b58\u9818\u57df\u306f\/var\/www_logs\/[\u30d0\u30fc\u30c1\u30e3\u30eb\u30db\u30b9\u30c8\u540d]-access_log.%Y%m%d<\/p>\n

td-agent\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u65b9\u6cd5<\/h1>\n

\u672c\u624b\u9806\u3067\u306ftd-agent1\u7cfb\u3092\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3057\u307e\u3059\u3002<\/p>\n

\u30ea\u30dd\u30b8\u30c8\u30ea\u306e\u8ffd\u52a0<\/h2>\n

\u4ee5\u4e0b\u306e\u5185\u5bb9\u3092\u30bf\u30fc\u30df\u30ca\u30eb\u306b\u305d\u306e\u307e\u307e\u8cbc\u308a\u3064\u3051<\/p>\n

\ncat > \/etc\/yum.repos.d\/td.repo <<EOF\n[treasuredata]\nname=TreasureData\nbaseurl=http:\/\/packages.treasuredata.com.s3.amazonaws.com\/redhat\/$basearch\ngpgcheck=0\ngpgkey=http:\/\/packages.treasuredata.com.s3.amazonaws.com\/redhat\/RPM-GPG-KEY-td-agent\nenable=0\nEOF\n<\/pre>\n
yum\u30b3\u30de\u30f3\u30c9\u306b\u3066\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb<\/h2>\n
\u4ee5\u4e0b\u306e\u5185\u5bb9\u3092\u30bf\u30fc\u30df\u30ca\u30eb\u306b\u305d\u306e\u307e\u307e\u8cbc\u308a\u3064\u3051<\/p>\n
\n yum -y --enablerepo=treasuredata install td-agent\n<\/pre>\n
conf\u8a2d\u5b9a<\/h2>\n
\u6a19\u6e96\u3067\u306f\u4e0d\u8981\u306aconf\u30d5\u30a1\u30a4\u30eb\u304c\u542b\u307e\u308c\u308b\u306e\u3067\u3001\u3053\u3061\u3089\u306e\u524a\u9664\u3068API\u30ad\u30fc\u7b49\u306e\u8a18\u8f09\u3092\u3057\u307e\u3059<\/p>\n
conf\u306e\u5909\u66f4<\/h3>\n
\u4ee5\u4e0b\u306e\u5185\u5bb9\u3092\u30bf\u30fc\u30df\u30ca\u30eb\u306b\u305d\u306e\u307e\u307e\u8cbc\u308a\u3064\u3051<\/p>\n
\n    mkdir \/tmp\/td-agent\n    chown td-agent:td-agent \/tmp\/td-agent\n    mkdir \/var\/log\/td-agent\/buffer\n    chown td-agent:td-agent \/var\/log\/td-agent\/buffer\n    cp -pi \/etc\/td-agent\/td-agent.conf \/etc\/td-agent\/td-agent.conf.org\n    cat > \/etc\/td-agent\/td-agent.conf <<EOF\n    # File input\n    ## read apache logs continuously and tags td.apache.access\n\n      type tail\n      format apache\n      path \/var\/www_logs\/!!!\u30d0\u30fc\u30c1\u30e3\u30eb\u30db\u30b9\u30c8\u540d!!!-access_log.%Y%m%d\n      tag apache.access.!!!\u30d0\u30fc\u30c1\u30e3\u30eb\u30db\u30b9\u30c8\u540d!!!\n      pos_file \/tmp\/td-agent\/!!!\u30d0\u30fc\u30c1\u30e3\u30eb\u30db\u30b9\u30c8\u540d!!!.pos\n\n\n\n      type s3\n      aws_key_id !!!NiftyCloudAccessKey!!!\n      aws_sec_key !!!NiftyCloudSecretAccessKey!!!\n      s3_bucket !!!NiftyCloud\u30d0\u30b1\u30c3\u30c8\u540d!!!\n      s3_endpoint ncss.nifty.com\n      check_apikey_on_start false\n      s3_object_key_format %{path}%{time_slice}%{hostname}_apachelog_%{index}.%{file_extension}\n      path www_logs\/\n      time_slice_format %Y\/%m\/%d\/\n      buffer_path \/var\/log\/td-agent\/buffer\/CloudStorage\n      flush_interval 60s\n\n    EOF\n<\/pre>\n
\u30b5\u30fc\u30d0\u56fa\u6709\u306e\u8a2d\u5b9a\u5909\u66f4<\/h3>\n
\n vi \/etc\/td-agent\/td-agent.conf \n<\/pre>\n
\u4ee5\u4e0b\u306e\u500b\u6240\u3092\u5909\u66f4(\u7f6e\u63db)
\n !!!\u30d0\u30fc\u30c1\u30e3\u30eb\u30db\u30b9\u30c8\u540d!!! \u2192 \u30b5\u30fc\u30d0\u30fc\u3067\u7a3c\u50cd\u3057\u3066\u3044\u308b\u30d0\u30fc\u30c1\u30e3\u30eb\u30db\u30b9\u30c8\u540d\u3002\u8907\u6570\u3042\u308b\u5834\u5408\u306f\u8981conf\u7de8\u96c6
\n !!!NiftyCloudAccessKey!!! \u3000\u3000\u3000\u2192\u8a72\u5f53\u30a2\u30ab\u30a6\u30f3\u30c8\u306e\u30a2\u30af\u30bb\u30b9\u30ad\u30fc\u3092\u8a18\u8f09
\n !!!NiftyCloudSecretAccessKey!!! \u2192\u8a72\u5f53\u30a2\u30ab\u30a6\u30f3\u30c8\u306e\u30b7\u30fc\u30af\u30ec\u30c3\u30c8\u30ad\u30fc\u3092\u8a18\u8f09
\n !!!NiftyCloud\u30d0\u30b1\u30c3\u30c8\u540d!!!\u3000\u3000\u3000\u2192\u4f5c\u6210\u3059\u308b\u30d0\u30b1\u30c3\u30c8\u540d\u3092\u8a2d\u5b9a<\/p>\n
aws-sdk\u306eS3\u56fa\u5b9a\u8a2d\u5b9a\u306e\u5909\u66f4<\/h3>\n
\u4ee5\u4e0b\u306e\u5185\u5bb9\u3092\u30bf\u30fc\u30df\u30ca\u30eb\u306b\u305d\u306e\u307e\u307e\u8cbc\u308a\u3064\u3051\u3001\u5b9f\u884c<\/p>\n
\n    sed -i -e 's\/        @endpoint = config.send(:"#{service_ruby_name}_endpoint")\/        #@endpoint = config.send(:"#{service_ruby_name}_endpoint")n        @endpoint = "ncss.nifty.com"\/g' \/etc\/td-agent\/td-agent.conf\n<\/pre>\n
td-agent\u8d77\u52d5<\/h2>\n
\u4ee5\u4e0b\u306e\u5185\u5bb9\u3092\u30bf\u30fc\u30df\u30ca\u30eb\u306b\u305d\u306e\u307e\u307e\u8cbc\u308a\u3064\u3051\u3001\u5b9f\u884c<\/p>\n
\n \/etc\/init.d\/td-agent start\n<\/pre>\n
\u52d5\u4f5c\u78ba\u8a8d<\/h2>\n
\u4ee5\u4e0b\u306e\u30b3\u30de\u30f3\u30c9\u3067\u30ed\u30b0\u3092\u78ba\u8a8d\u3057\u3001
\n\u4e0b\u8a18\u30ed\u30b0\u304c\u3042\u308b\u4e8b\u3092\u78ba\u8a8d<\/p>\n
\n tail \/var\/log\/td-agent\/td-agent.log\n<\/pre>\n[\u8d77\u52d5\u65e5\u6642]\u3000[info]: following tail of \/var\/www_logs\/[\u672c\u65e5\u306e\u30ed\u30b0\u30d5\u30a1\u30a4\u30eb]\n
\u4f8b\uff1a
\n 2015-03-12 11:28:55 +0900 [info]: following tail of \/var\/www_logs\/fluentdtest.xxx.xxx-access_log.20150312<\/p>\n
\u4ee5\u4e0a\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n
\u30b9\u30c8\u30ec\u30fc\u30b8\u306b\u5165\u3063\u305f\u30ed\u30b0\u3092\u5404\u7a2e\u30ed\u30b0\u89e3\u6790\u30c4\u30fc\u30eb\u306b\u5165\u308c\u3066\u78ba\u8a8d\u3057\u305f\u308a\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u308b\u306f\u305a\u3067\u3059\uff01<\/p>\n
\u306a\u304a\u3001\u624b\u9806\u9014\u4e2d\u306e\u30d0\u30b1\u30c3\u30c8\u540d\u3092\u5909\u66f4\u3059\u308b\u3053\u3068\u3067\u3001IDCF\u30af\u30e9\u30a6\u30c9\u306e\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u30b9\u30c8\u30ec\u30fc\u30b8\u3084\u3001\u3055\u304f\u3089\u306eBASE Storage\u306b\u3082\u5bfe\u5fdc\u3067\u304d\u307e\u3059\uff01
\n\u8208\u5473\u306e\u3042\u308b\u4eba\u306f\u4e00\u5ea6\u30c8\u30e9\u30a4\u3057\u3066\u307f\u3066\u304f\u3060\u3055\u3044\uff01<\/p>\n
\u306a\u306b\u304b\u3042\u308c\u3070\u30b3\u30e1\u30f3\u30c8\u7b49\u3070\u3093\u3070\u3093\u9802\u3051\u308c\u3070\u5b09\u3057\u3044\u3067\u3059\uff01<\/p>\n","protected":false},"excerpt":{"rendered":"
AWS\u3067\u306f\u30aa\u30fc\u30c8\u30b9\u30b1\u30fc\u30eb\u306e\u90fd\u5408\u306a\u3069\u3082\u3042\u308a\u3001\u30ed\u30b0\u3092fluentd(td-agent)\u3092\u7528\u3044\u3066Amazon S3\u3078\u7f6e\u304f\u4e8b\u304c\u6d41\u884c\u3063\u3066\u3044\u307e\u3059\u306d\u3002 \u3057\u304b\u3057\u3001AWS\u3068\u540c\u3058\u624b\u9806\u3067td-agent\u3092\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3057\u3066\u3082\u3001\u30cb\u30d5\u30c6\u30a3\u30af\u30e9\u30a6\u30c9\u30b9…<\/p>\n","protected":false},"author":43,"featured_media":789,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_locale":"ja","_original_post":"3024","footnotes":""},"categories":[34,21],"tags":[57,36],"class_list":{"0":"post-3024","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-web","8":"category-21","9":"tag-niftycloud","10":"tag-tips","11":"ja"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/3024","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/users\/43"}],"replies":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/comments?post=3024"}],"version-history":[{"count":20,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/3024\/revisions"}],"predecessor-version":[{"id":3078,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/3024\/revisions\/3078"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/media\/789"}],"wp:attachment":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/media?parent=3024"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/categories?post=3024"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/tags?post=3024"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}