{"id":5023,"date":"2015-12-09T10:55:24","date_gmt":"2015-12-09T01:55:24","guid":{"rendered":"http:\/\/www.skyarch.net\/blog\/?p=5023"},"modified":"2015-12-09T11:42:06","modified_gmt":"2015-12-09T02:42:06","slug":"5023","status":"publish","type":"post","link":"https:\/\/www.skyarch.net\/blog\/5023\/","title":{"rendered":"fluentd\u306b\u3066\u751f\u30ed\u30b0\u3092\u8ee2\u9001\u3059\u308b\u65b9\u6cd5"},"content":{"rendered":"

\u30b5\u30fc\u30d0\u30fc\u304c\u30b9\u30b1\u30fc\u30eb\u3059\u308b\u306e\u3067\u3001\u30ed\u30b0\u3092\u30b5\u30fc\u30d0\u5185\u306b\u6301\u305f\u305b\u305f\u304f\u306a\u3044\uff01<\/p>\n

json\u5f62\u5f0f\u3060\u3068\u898b\u306b\u304f\u3044 or json\u5f62\u5f0f\u3060\u3068\u6271\u3044\u306b\u304f\u3044\uff01
\n\u2192\u3060\u304b\u3089\u751f\u30ed\u30b0\u306e\u307e\u307es3\u306b\u8ee2\u9001\u3057\u305f\u3044\uff01\uff01\uff01<\/p>\n

\u3068\u3044\u3063\u305f\u5834\u5408\u306e\u65b9\u6cd5<\/p>\n

 <\/p>\n

1\uff0efluentd\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb<\/h3>\n

\u81f3\u3063\u3066\u57fa\u672c\u7684\u306a\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u65b9\u6cd5\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n

\u203bruby\u306f\u65e2\u306b\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u6e08\u307f\u3068\u3057\u307e\u3059
\ncurl -L http:\/\/toolbelt.treasuredata.com\/sh\/install-redhat-td-agent2.sh | sh<\/p>\n

mkdir \/tmp\/td-agent
\nchown td-agent:td-agent \/tmp\/td-agent
\nmkdir \/var\/log\/td-agent\/buffer
\nchown td-agent:td-agent \/var\/log\/td-agent\/buffer<\/p>\n

 <\/p>\n

2.fluentd\u306econf\u3092\u66f8\u304d\u63db\u3048\u307e\u3059<\/h3>\n

\u3053\u3053\u3067\u306f\/var\/www_logs\/\u3078\u5410\u304b\u308c\u3066\u3044\u308bapache\u306e\u30a2\u30af\u30bb\u30b9\u30ed\u30b0\u3092\u751f\u8ee2\u9001\u3059\u308b\u5834\u5408\u3092\u8a18\u8f09\u3057\u307e\u3059\u3002<\/p>\n

format\u3092none\u306b\u3059\u308b\u4e8b\u304c\u30ad\u30e2\u3067\u3059
\n\/var\/log\/messages\u7b49\u3092\u8ee2\u9001\u3057\u305f\u3044\u5834\u5408\u306f\u5404\u81ea\u66f8\u304d\u63db\u3048\u3066\u304f\u3060\u3055\u3044<\/p>\n

 <\/p>\n

vi\u00a0\/etc\/td-agent\/td-agent.conf<\/p>\n

<source>
\ntype tail
\nformat none
\npath \/var\/www_logs\/test.com-access_log.%Y%m%d
\ntag rawlog.test.com
\npos_file \/tmp\/td-agent\/rawlog.test.com.pos
\n<\/source>
\n<match rawlog.**>
\ntype s3
\naws_key_id [\u30a2\u30af\u30bb\u30b9\u30ad\u30fc]\naws_sec_key [\u30b7\u30fc\u30af\u30ec\u30c3\u30c8\u30ad\u30fc]\ns3_bucket [\u30d0\u30b1\u30c3\u30c8]\ncheck_apikey_on_start false
\ns3_object_key_format %{path}%{time_slice}%{hostname}raw_apachelog<\/em>%{index}.%{file_extension}
\npath www_logs\/
\ntime_slice_format %Y\/%m\/%d\/
\nbuffer_path \/var\/log\/td-agent\/buffer\/CloudStorage_raw
\nflush_interval 600s
\n<\/match><\/p>\n

 <\/p>\n

3. fluentd\u8d77\u52d5<\/h4>\n

\/etc\/init.d\/td-agent start<\/p>\n

\u3053\u3053\u3067\u30a8\u30e9\u30fc\u304c\u51fa\u308b\u5834\u5408\u3001\/var\/log\/td-agent\/td-agent.log\u3092\u898b\u308b\u3068\u539f\u56e0\u89e3\u6c7a\u306e\u30d2\u30f3\u30c8\u306b\u306a\u308b\u3068\u601d\u3044\u307e\u3059<\/p>\n

 <\/p>\n

4. s3\u306e\u30d0\u30b1\u30c3\u30c8\u5185\u3092\u78ba\u8a8d\u3057\u3066\u300110\u5206\u3054\u3068\u306b\u8ee2\u9001\u3055\u308c\u3066\u3044\u308b\u4e8b\u3092\u78ba\u8a8d<\/h3>\n

\u30b5\u30fc\u30d0\u5185\u306b\u30ed\u30b0\u304c\u5897\u5927\u3057\u3066\u30c7\u30a3\u30b9\u30af\u5bb9\u91cf\u304c\u8db3\u308a\u306a\u3044\u3088\uff01<\/p>\n

\u306a\u3093\u3066\u53e4\u3044\u6642\u4ee3\u3068\u306f\u3053\u308c\u3067\u30aa\u30b5\u30e9\u30d0\u3067\u304d\u307e\u3059\u3002<\/p>\n

\u30c7\u30a3\u30b9\u30af\u5bb9\u91cf\u3092\u6c17\u306b\u3055\u308c\u308b\u65b9\u306f\u305c\u3072\u691c\u8a0e\u3092\uff01<\/p>\n","protected":false},"excerpt":{"rendered":"

\u30b5\u30fc\u30d0\u30fc\u304c\u30b9\u30b1\u30fc\u30eb\u3059\u308b\u306e\u3067\u3001\u30ed\u30b0\u3092\u30b5\u30fc\u30d0\u5185\u306b\u6301\u305f\u305b\u305f\u304f\u306a\u3044\uff01 json\u5f62\u5f0f\u3060\u3068\u898b\u306b\u304f\u3044 or json\u5f62\u5f0f\u3060\u3068\u6271\u3044\u306b\u304f\u3044\uff01 \u2192\u3060\u304b\u3089\u751f\u30ed\u30b0\u306e\u307e\u307es3\u306b\u8ee2\u9001\u3057\u305f\u3044\uff01\uff01\uff01 \u3068\u3044\u3063\u305f\u5834\u5408\u306e\u65b9\u6cd5   1\uff0efluentd…<\/p>\n","protected":false},"author":43,"featured_media":5040,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_locale":"ja","_original_post":"5023","footnotes":""},"categories":[20,29,19],"tags":[],"class_list":{"0":"post-5023","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-aws","8":"category-linux","9":"category-19","10":"ja"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/5023","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/users\/43"}],"replies":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/comments?post=5023"}],"version-history":[{"count":8,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/5023\/revisions"}],"predecessor-version":[{"id":5233,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/5023\/revisions\/5233"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/media\/5040"}],"wp:attachment":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/media?parent=5023"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/categories?post=5023"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/tags?post=5023"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}