{"id":9287,"date":"2016-12-02T10:41:54","date_gmt":"2016-12-02T01:41:54","guid":{"rendered":"http:\/\/www.skyarch.net\/blog\/?p=9287"},"modified":"2016-12-02T10:47:43","modified_gmt":"2016-12-02T01:47:43","slug":"%e3%80%90aws-reinvent-2016%e3%80%91%e6%96%b0%e3%82%b5%e3%83%bc%e3%83%93%e3%82%b9%e3%80%8caws-glue%e3%80%8d%e3%81%8c%e7%99%ba%e8%a1%a8%ef%bc%81","status":"publish","type":"post","link":"https:\/\/www.skyarch.net\/blog\/%e3%80%90aws-reinvent-2016%e3%80%91%e6%96%b0%e3%82%b5%e3%83%bc%e3%83%93%e3%82%b9%e3%80%8caws-glue%e3%80%8d%e3%81%8c%e7%99%ba%e8%a1%a8%ef%bc%81\/","title":{"rendered":"\u3010AWS re:Invent 2016\u3011\u65b0\u30b5\u30fc\u30d3\u30b9\u300cAWS Glue\u300d\u304c\u767a\u8868\uff01"},"content":{"rendered":"\"img_20161201_100237\"\n

AWS Glue\u3068\u306f\uff1f<\/h2>\n

re:Invent 2016 DAY3 keynote \u306b\u3066\u767a\u8868\u3055\u308c\u305f\u65b0\u30b5\u30fc\u30d3\u30b9\u3068\u306a\u308a\u307e\u3059\u3002
\n\u30b5\u30fc\u30d3\u30b9\u540d\u306e\u7531\u6765\u306f\u3001\u4e0b\u56f3\u306b\u8a18\u8f09\u306e\u300c\u30e2\u30c0\u30f3\u30c7\u30fc\u30bf\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u300d\uff08AWS\u306e\u8868\u73fe\uff09\u69cb\u7bc9\u306e\u305f\u3081\u306b\u5fc5\u8981\u306a\u5404\u7a2e\u30b5\u30fc\u30d3\u30b9\u306e\u7e4b\u304e\u8fbc\u307f\u3092\u884c\u3046\u969b\u306b\u3001\u7cca \u306e\u3088\u3046\u306b\u52d5\u4f5c\u3059\u308b\u30b5\u30fc\u30d3\u30b9\u3068\u306e\u4e8b\u3067\u3059\u3002
\n\u203b\u4f1a\u5834\u3067\u306f\u88fd\u54c1\u540d\u304c\u767a\u8868\u3055\u308c\u305f\u969b\u306b\u7b11\u3044\u306b\u3064\u3064\u307e\u308c\u305f\u4e8b\u3092\u6ce8\u8a18\u3055\u305b\u3066\u9802\u304d\u307e\u3059<\/p>\n\"img_20161201_100440_stitch\"\n

\u516c\u5f0f\u30ea\u30f3\u30af<\/strong>
\nhttps:\/\/aws.amazon.com\/jp\/glue\/<\/a><\/p>\n

AWS Glue \u306f\u30d5\u30eb\u30de\u30cd\u30fc\u30b8\u30c9\u306e ETL \u30b5\u30fc\u30d3\u30b9\u3067\u3059\u3002
\n\u56f0\u96e3\u3067\u6642\u9593\u306e\u304b\u304b\u308b\u30c7\u30fc\u30bf\u306e\u767a\u898b\u3001\u5909\u63db\u3001\u30de\u30c3\u30d4\u30f3\u30b0\u3001\u304a\u3088\u3073\u30b8\u30e7\u30d6\u30b9\u30b1\u30b8\u30e5\u30fc\u30ea\u30f3\u30b0\u306e\u30bf\u30b9\u30af\u3092\u7c21\u7d20\u5316\u3057\u3001\u81ea\u52d5\u5316\u3057\u307e\u3059\u3002
\n\u4f7f\u3044\u3084\u3059\u3044\u30b3\u30f3\u30bd\u30fc\u30eb\u3067\u30c7\u30fc\u30bf\u3092\u79fb\u52d5\u3059\u308b\u30d7\u30ed\u30bb\u30b9\u3092\u30ac\u30a4\u30c9\u3057\u3001\u30c7\u30fc\u30bf\u30bd\u30fc\u30b9\u306e\u7406\u89e3\u3001\u5206\u6790\u306e\u305f\u3081\u306e\u30c7\u30fc\u30bf\u306e\u6e96\u5099\u3001\u30c7\u30fc\u30bf\u30bd\u30fc\u30b9\u304b\u3089\u5b9b\u5148\u3078\u306e\u4fe1\u983c\u6027\u306e\u9ad8\u3044\u30ed\u30fc\u30c9\u306b\u5f79\u7acb\u3061\u307e\u3059\u3002
\n \u203bETL\u3068\u306f Extract\/Transform\/Load\uff08\u30c7\u30fc\u30bf\u306e\u62bd\u51fa\u3001\u5909\u63db\u3001\u30ed\u30fc\u30c9\uff09\u3092\u884c\u3046\u30c4\u30fc\u30eb - wikipedia<\/p>\n

\u52d5\u4f5c\u624b\u9806<\/h2>\n

Step 1. \u30c7\u30fc\u30bf\u30ab\u30bf\u30ed\u30b0\u306e\u69cb\u7bc9<\/h3>\n\"img_20161201_100247\"\n

AWS\u7ba1\u7406\u30b3\u30f3\u30bd\u30fc\u30eb\u3092\u4f7f\u7528\u3057\u3066\u3001AWS Glue\u3067\u30c7\u30fc\u30bf\u30bd\u30fc\u30b9\u3092\u767b\u9332\u3057\u307e\u3059\u3002 AWS Glue\u306f\u3001JSON\u3001CSV\u3001Parquet\u306a\u3069\u3001\u591a\u304f\u306e\u4e00\u822c\u7684\u306a\u30bd\u30fc\u30b9\u30d5\u30a9\u30fc\u30de\u30c3\u30c8\u3084\u30c7\u30fc\u30bf\u30bf\u30a4\u30d7\u306b\u5bfe\u3057\u3066\u3001\u3042\u3089\u304b\u3058\u3081\u4f5c\u6210\u3055\u308c\u305f\u5206\u985e\u5b50\u3092\u4f7f\u7528\u3057\u3066\u30c7\u30fc\u30bf\u30bd\u30fc\u30b9\u3092\u30af\u30ed\u30fc\u30eb\u3057\u3001\u30c7\u30fc\u30bf\u30ab\u30bf\u30ed\u30b0\u3092\u69cb\u7bc9\u3057\u307e\u3059\u3002\u307e\u305f\u3001\u72ec\u81ea\u306e\u30af\u30e9\u30b7\u30d5\u30a1\u30a4\u30a2\u3092\u8ffd\u52a0\u3057\u305f\u308a\u3001AWS Glue\u30b3\u30df\u30e5\u30cb\u30c6\u30a3\u304b\u3089\u30af\u30e9\u30b7\u30d5\u30a1\u30a4\u30a2\u3092\u9078\u629e\u3057\u3066\u30af\u30ed\u30fc\u30eb\u306b\u8ffd\u52a0\u3059\u308b\u3053\u3068\u3082\u3067\u304d\u307e\u3059\u3002(google\u7ffb\u8a33)<\/p>\n

Step 2. \u30c7\u30fc\u30bf\u5909\u63db\u30eb\u30fc\u30eb\u3092\u30b3\u30fc\u30c9\u3067\u8a18\u8f09<\/h3>\n\"img_20161201_100336\"\n

\u30c7\u30fc\u30bf\u30bd\u30fc\u30b9\u3068\u30bf\u30fc\u30b2\u30c3\u30c8\u3092\u9078\u629e\u3059\u308b\u3068\u3001AWS Glue\u306fPython\u30b3\u30fc\u30c9\u3092\u751f\u6210\u3057\u3066\u30bd\u30fc\u30b9\u304b\u3089\u30c7\u30fc\u30bf\u3092\u62bd\u51fa\u3057\u3001\u30bf\u30fc\u30b2\u30c3\u30c8\u30b9\u30ad\u30fc\u30de\u3068\u4e00\u81f4\u3059\u308b\u3088\u3046\u306b\u30c7\u30fc\u30bf\u3092\u5909\u63db\u3057\u3001\u30bf\u30fc\u30b2\u30c3\u30c8\u306b\u30ed\u30fc\u30c9\u3057\u307e\u3059\u3002\u81ea\u52d5\u751f\u6210\u30b3\u30fc\u30c9\u306f\u3001\u30c7\u30fc\u30bf\u3084\u30cf\u30fc\u30c9\u30a6\u30a7\u30a2\u306e\u4e0d\u826f\u306a\u3069\u306e\u4e00\u822c\u7684\u306a\u30a8\u30e9\u30fc\u306e\u30b1\u30fc\u30b9\u3092\u51e6\u7406\u3057\u307e\u3059\u3002\u3053\u306e\u30b3\u30fc\u30c9\u306f\u3001\u304a\u6c17\u306b\u5165\u308a\u306eIDE\u3092\u4f7f\u7528\u3057\u3066\u7de8\u96c6\u3057\u3001\u72ec\u81ea\u306e\u30b5\u30f3\u30d7\u30eb\u30c7\u30fc\u30bf\u3067\u30c6\u30b9\u30c8\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002\u307e\u305f\u3001\u4ed6\u306eAWS Glue\u30e6\u30fc\u30b6\u30fc\u3068\u5171\u6709\u3057\u3066\u3044\u308b\u30b3\u30fc\u30c9\u3092\u30d6\u30e9\u30a6\u30ba\u3057\u3066\u3001\u81ea\u5206\u306e\u4ed5\u4e8b\u306b\u5f15\u304d\u8fbc\u3080\u3053\u3068\u3082\u3067\u304d\u307e\u3059\u3002(google\u7ffb\u8a33)<\/p>\n

Step 3. \u30b9\u30b1\u30b8\u30e5\u30fc\u30eb\u306e\u4f5c\u6210\u30fb\u5b9f\u884c<\/h3>\n\"img_20161201_100355\"\n

AWS Glue\u306e\u67d4\u8edf\u306a\u30b9\u30b1\u30b8\u30e5\u30fc\u30e9\u3092\u4f7f\u7528\u3057\u3066\u3001\u5b9a\u671f\u7684\u306b\u3001\u30c8\u30ea\u30ac\u306b\u5fdc\u3058\u3066\u3001\u307e\u305f\u306fAWS Lambda\u30a4\u30d9\u30f3\u30c8\u306b\u5bfe\u5fdc\u3057\u3066\u30d5\u30ed\u30fc\u3092\u5b9f\u884c\u3067\u304d\u307e\u3059\u3002 AWS Glue\u306fETL\u30b8\u30e7\u30d6\u3092Apache Spark\u30ce\u30fc\u30c9\u306b\u81ea\u52d5\u7684\u306b\u914d\u5e03\u3059\u308b\u305f\u3081\u3001\u30c7\u30fc\u30bf\u91cf\u304c\u5897\u3048\u308b\u306b\u3064\u308c\u3066ETL\u306e\u5b9f\u884c\u6642\u9593\u304c\u4e00\u5b9a\u306b\u4fdd\u305f\u308c\u307e\u3059\u3002 AWS Glue\u306f\u30b8\u30e7\u30d6\u306e\u5b9f\u884c\u3092\u9069\u5207\u306a\u9806\u5e8f\u3067\u8abf\u6574\u3057\u3001\u5931\u6557\u3057\u305f\u30b8\u30e7\u30d6\u3092\u81ea\u52d5\u7684\u306b\u518d\u8a66\u884c\u3057\u307e\u3059\u3002 AWS Glue\u306f\u3001\u6642\u9593\u901a\u308a\u306b\u30b8\u30e7\u30d6\u3092\u5b8c\u4e86\u3057\u3001\u30b3\u30b9\u30c8\u3092\u6700\u5c0f\u9650\u306b\u6291\u3048\u308b\u305f\u3081\u306b\u5fc5\u8981\u306a\u30a4\u30f3\u30d5\u30e9\u30b9\u30c8\u30e9\u30af\u30c1\u30e3\u3092\u30b9\u30b1\u30fc\u30e9\u30d6\u30eb\u306b\u62e1\u5f35\u3057\u307e\u3059\u3002(google\u7ffb\u8a33)<\/p>\n

\u307e\u3068\u3081<\/h2>\n

\u30c7\u30fc\u30bf\u5909\u63db\u30fb\u53d6\u308a\u8fbc\u307f\u306e\u305f\u3081\u306b\u540c\u3058\u3088\u3046\u306a\u51e6\u7406\u3092\u8a18\u8f09\u3057\u305f\u308a\u3001CSV\u30d5\u30a9\u30fc\u30de\u30c3\u30c8\u304c\u5909\u66f4\u3055\u308c\u308b\u305f\u3073\u306b\u30d7\u30ed\u30b0\u30e9\u30e0\u5909\u66f4\u3092\u5b9f\u969b\u306b\u73fe\u5834\u3067\u884c\u3063\u3066\u3044\u308b\u4e8b\u3082\u3042\u308a
\n\u305d\u306e\u3088\u3046\u306a\u51e6\u7406\u306e\u624b\u52a9\u3051\u3092\u884c\u3063\u3066\u304f\u308c\u308b\u3068\u3044\u3046\u4e8b\u3067\u3001\u975e\u5e38\u306b\u697d\u3057\u307f\u306a\u30b5\u30fc\u30d3\u30b9\u3067\u3059(\u307e\u3060coming soon\u3068\u306a\u3063\u3066\u304a\u308a\u307e\u3059)
\nPython\u30b3\u30fc\u30c9\u751f\u6210\u3084\u3001\u30c7\u30fc\u30bf\u304c\u5897\u3048\u3066\u3082\u30b9\u30b1\u30fc\u30e9\u30d6\u30eb\u306b\u5bfe\u5fdc\u3001\u51e6\u7406\u7d50\u679c\u306e\u53ef\u8996\u5316\u3082\u5927\u304d\u306a\u9b45\u529b\u3068\u611f\u3058\u307e\u3057\u305f\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"

AWS Glue\u3068\u306f\uff1f re:Invent 2016 DAY3 keynote \u306b\u3066\u767a\u8868\u3055\u308c\u305f\u65b0\u30b5\u30fc\u30d3\u30b9\u3068\u306a\u308a\u307e\u3059\u3002 \u30b5\u30fc\u30d3\u30b9\u540d\u306e\u7531\u6765\u306f\u3001\u4e0b\u56f3\u306b\u8a18\u8f09\u306e\u300c\u30e2\u30c0\u30f3\u30c7\u30fc\u30bf\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u300d\uff08AWS\u306e\u8868\u73fe\uff09\u69cb\u7bc9\u306e\u305f\u3081\u306b\u5fc5\u8981\u306a\u5404\u7a2e\u30b5…<\/p>\n","protected":false},"author":2,"featured_media":8966,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_locale":"ja","_original_post":"9287","footnotes":""},"categories":[184,81],"tags":[],"class_list":{"0":"post-9287","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-aws-reinvent-2016","8":"category-81","9":"ja"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/9287","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/comments?post=9287"}],"version-history":[{"count":8,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/9287\/revisions"}],"predecessor-version":[{"id":9356,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/9287\/revisions\/9356"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/media\/8966"}],"wp:attachment":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/media?parent=9287"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/categories?post=9287"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/tags?post=9287"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}