{"id":23325,"date":"2022-12-22T10:26:58","date_gmt":"2022-12-22T01:26:58","guid":{"rendered":"https:\/\/www.skyarch.net\/blog\/?p=23325"},"modified":"2022-12-22T10:26:58","modified_gmt":"2022-12-22T01:26:58","slug":"%e3%82%b9%e3%83%a9%e3%83%83%e3%82%b7%e3%83%a5%e5%8c%ba%e5%88%87%e3%82%8a%e3%81%ae%e6%97%a5%e4%bb%98%e3%82%92sql-query%e5%a4%89%e6%8f%9b%e3%81%a7timestamp%e5%9e%8b%e3%81%ab%e8%aa%8d%e8%ad%98%e3%81%95","status":"publish","type":"post","link":"https:\/\/www.skyarch.net\/blog\/%e3%82%b9%e3%83%a9%e3%83%83%e3%82%b7%e3%83%a5%e5%8c%ba%e5%88%87%e3%82%8a%e3%81%ae%e6%97%a5%e4%bb%98%e3%82%92sql-query%e5%a4%89%e6%8f%9b%e3%81%a7timestamp%e5%9e%8b%e3%81%ab%e8%aa%8d%e8%ad%98%e3%81%95\/","title":{"rendered":"\u30b9\u30e9\u30c3\u30b7\u30e5\u533a\u5207\u308a\u306e\u65e5\u4ed8\u3092SQL Query\u5909\u63db\u3067timestamp\u578b\u306b\u8a8d\u8b58\u3055\u305b\u3066\u307f\u305f"},"content":{"rendered":"<p>Glue\u30b8\u30e7\u30d6\u3067\u65e5\u4ed8\u5f62\u5f0f\u306e\u6587\u5b57\u5217\u3092ApplyMapping\u5909\u63db\u3067\u8a8d\u8b58\u3055\u305b\u3088\u3046\u3068\u3057\u305f\u3089\u3046\u307e\u304f\u884c\u304b\u306a\u304b\u3063\u305f\u306e\u3067\u3001\u4ee3\u308f\u308a\u306bSQL Query\u3092\u4f7f\u7528\u3057\u3066\u8a8d\u8b58\u3055\u305b\u3066\u307f\u307e\u3057\u305f\u3002<\/p>\n<h1>\u3084\u308a\u305f\u3044\u3053\u3068<\/h1>\n<ul>\n<li>\u5165\u529b\u30c7\u30fc\u30bf\u306eCSV\u3092Glue\u30b8\u30e7\u30d6\u3067\u5909\u63db\u3057\u3001Glue\u30c6\u30fc\u30d6\u30eb\u306b\u51fa\u529b\u3059\u308b\u3002<\/li>\n<li>Glue\u30c6\u30fc\u30d6\u30eb\u306f\u9069\u5207\u306a\u30c7\u30fc\u30bf\u578b\u306b\u3059\u308b\u3002<\/li>\n<li><code>2022\/01\/01<\/code>\u3068\u3044\u3046\u65e5\u4ed8\u5f62\u5f0f\u3092timestamp\u578b\u3068\u3057\u3066\u5909\u63db\u3059\u308b\u3002<\/li>\n<\/ul>\n<h2>\u5165\u529b\u30c7\u30fc\u30bf<\/h2>\n<p>\u4eca\u56de\u3001Glue\u3067\u51e6\u7406\u3059\u308b\u5165\u529b\u3068\u306a\u308bCSV\u30c7\u30fc\u30bf\u306f\u3053\u306e\u30c7\u30fc\u30bf\u3068\u3057\u307e\u3059\u3002<br \/>\n\u65e5\u4ed8\u306e\u533a\u5207\u308a\u304c\u30cf\u30a4\u30d5\u30f3\u3067\u306f\u306a\u304f\u30b9\u30e9\u30c3\u30b7\u30e5\u306b\u306a\u3063\u3066\u3044\u307e\u3059\u3002<\/p>\n<pre>\nid,name,updatedat\n1,\"\u3059\u3054\u3044 \u305f\u308d\u3046\",\"2020\/01\/15 03:44:40\"\n2,\"\u3082\u306e\u3059\u3054\u3044 \u3058\u308d\u3046\",\"2022\/10\/12 05:12:59\"\n3,\"\u3068\u3093\u3067\u3082\u306a\u3044 \u3055\u3076\u308d\u3046\",\"2022\/03\/15 12:34:56\"\n5,\"\u306f\u3093\u3071\u306a\u3044 \u3054\u308d\u3046\",\"2021\/10\/29 23:55:10\"\n<\/pre>\n<h2>\u51fa\u529b\u30c6\u30fc\u30d6\u30eb<\/h2>\n<p>\u3053\u3093\u306a\u611f\u3058\u306eGlue\u30c6\u30fc\u30d6\u30eb\u306b\u51fa\u3059\u3053\u3068\u3092\u76ee\u6a19\u3068\u3057\u307e\u3059\u3002<\/p>\n<ul>\n<li>\u30c7\u30fc\u30bf\u30b9\u30c8\u30a2\uff1aS3\u30d0\u30b1\u30c3\u30c8<\/li>\n<li>\u30ab\u30e9\u30e0\u4e00\u89a7\uff1a\n<ul>\n<li><code>id<\/code> (int)<\/li>\n<li><code>name<\/code> (string)<\/li>\n<li><code>updatedat<\/code> (timestamp)<\/li>\n<\/ul>\n<\/li>\n<li>\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\uff1a\u306a\u3057<\/li>\n<li>\u4fdd\u5b58\u5f62\u5f0f\uff1aparquet (\u30c7\u30fc\u30bf\u305d\u306e\u3082\u306e\u304c\u30c7\u30fc\u30bf\u578b\u3092\u6301\u3063\u3066\u3044\u308c\u3070\u4f55\u3067\u3082OK)<\/li>\n<\/ul>\n<h1>\u30c6\u30b9\u30c8\u74b0\u5883\u306e\u4f5c\u6210\u30fb\u5b9f\u884c<\/h1>\n<p>\u74b0\u5883\u3092\u7528\u610f\u3059\u308b\u5fc5\u8981\u304c\u3042\u308a\u307e\u3059\u306e\u3067\u3001\u4e0b\u8a18\u306e\u624b\u9806\u3067\u74b0\u5883\u3092\u4f5c\u3063\u3066\u8a66\u3057\u3066\u307f\u307e\u3059\u3002<\/p>\n<h2>\u30bd\u30fc\u30b9\u30c7\u30fc\u30bf\u306e\u914d\u7f6e<\/h2>\n<ul>\n<li>S3\u30d0\u30b1\u30c3\u30c8\uff1a<code>hogehoge-440f-413c-a52b-e7edc6f4ea8f<\/code><\/li>\n<li>\u30c7\u30fc\u30bf\u914d\u7f6e\u5148\uff1a<code>\/usersource\/source.csv<\/code><\/li>\n<\/ul>\n<a href=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/s3bucket.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/s3bucket-300x149.png\" alt=\"\" width=\"300\" height=\"149\" class=\"alignnone size-medium wp-image-23328\" srcset=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/s3bucket-300x149.png 300w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/s3bucket-768x381.png 768w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/s3bucket-728x361.png 728w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/s3bucket.png 992w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>\n<img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/s3bucket.png\" alt=\"\" width=\"992\" height=\"492\" class=\"aligncenter size-full wp-image-23328\" srcset=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/s3bucket.png 992w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/s3bucket-300x149.png 300w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/s3bucket-768x381.png 768w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/s3bucket-728x361.png 728w\" sizes=\"auto, (max-width: 992px) 100vw, 992px\" \/>\n<h2>\u51fa\u529b\u5148Glue\u30c6\u30fc\u30d6\u30eb\u306e\u4f5c\u6210<\/h2>\n<ul>\n<li>\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\uff1a<code>default<\/code><\/li>\n<li>\u30c6\u30fc\u30d6\u30eb\u540d\uff1a<code>usertarget<\/code><\/li>\n<li>\u30c7\u30fc\u30bf\u30b9\u30c8\u30a2\uff1a<code>s3:\/\/hogehoge-440f-413c-a52b-e7edc6f4ea8f\/usertarget\/<\/code><\/li>\n<li>\u30c7\u30fc\u30bf\u30d5\u30a9\u30fc\u30de\u30c3\u30c8\uff1a<code>Parquet<\/code><\/li>\n<li>\u30ab\u30e9\u30e0\uff1a\n<ul>\n<li><code>id<\/code> (int)<\/li>\n<li><code>name<\/code> (string)<\/li>\n<li><code>updatedat<\/code> (timestamp)<\/li>\n<\/ul>\n<\/li>\n<li>\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\uff1a\u306a\u3057<\/li>\n<\/ul>\n<a href=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluetable.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluetable-300x290.png\" alt=\"\" width=\"300\" height=\"290\" class=\"alignnone size-medium wp-image-23329\" srcset=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluetable-300x290.png 300w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluetable-768x742.png 768w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluetable-728x703.png 728w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluetable.png 856w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>\n<h2>AWS Glue Studio\u3067\u306eGlue\u30b8\u30e7\u30d6\u306e\u4f5c\u6210<\/h2>\n<p>\u307e\u305a\u306fCreate Job\u306e\u30bb\u30af\u30b7\u30e7\u30f3\u3067\u30b8\u30e7\u30d6\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002<\/p>\n<ul>\n<li>\u30b8\u30e7\u30d6\u30bf\u30a4\u30d7\uff1aVisual with a source and target<\/li>\n<li>Source\uff1aAmazon S3<\/li>\n<li>Target\uff1aAWS Glue Data Catalog<\/li>\n<\/ul>\n<a href=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/createjob.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/createjob-300x104.png\" alt=\"\" width=\"300\" height=\"104\" class=\"alignnone size-medium wp-image-23330\" srcset=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/createjob-300x104.png 300w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/createjob-1024x354.png 1024w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/createjob-768x266.png 768w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/createjob-728x252.png 728w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/createjob.png 1043w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>\n<p>\u30b8\u30e7\u30d6\u7de8\u96c6\u753b\u9762\u306b\u9077\u79fb\u3057\u3001\u5de6\u5074\u306b\u306f\u51e6\u7406\u306e\u30b0\u30e9\u30d5\u304c\u73fe\u308c\u307e\u3059\u3002<\/p>\n<a href=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/jobgraph.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/jobgraph-187x300.png\" alt=\"\" width=\"187\" height=\"300\" class=\"alignnone size-medium wp-image-23332\" srcset=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/jobgraph-187x300.png 187w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/jobgraph.png 319w\" sizes=\"auto, (max-width: 187px) 100vw, 187px\" \/><\/a>\n<p>\u30b0\u30e9\u30d5\u4e0a\u306e\u300cS3 bucket\u300d\u30ce\u30fc\u30c9\u3092\u30af\u30ea\u30c3\u30af\u3057\u3001\u5404\u7a2e\u8a2d\u5b9a\u3092\u5b9f\u65bd\u3057\u307e\u3059\u3002<\/p>\n<ul>\n<li>Data source properties\u30bf\u30d6\n<ul>\n<li>S3 source type\uff1aS3 location<\/li>\n<li>S3 URL\uff1a<code>s3:\/\/hogehoge-440f-413c-a52b-e7edc6f4ea8f\/usersource\/<\/code><\/li>\n<li>Data format\uff1aCSV<\/li>\n<li>First line of source file contains column headers\uff1a\u30c1\u30a7\u30c3\u30af\u3059\u308b<\/li>\n<\/ul>\n<\/li>\n<li>Output schema\u30bf\u30d6\n<ul>\n<li>id\u306e\u5217\u304c\u52dd\u624b\u306blong\u3067\u8a8d\u8b58\u3055\u308c\u3066\u3044\u308b\u306e\u3067string\u306b\u4fee\u6b63<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<a href=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/applymapping-transform.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/applymapping-transform-300x115.png\" alt=\"\" width=\"300\" height=\"115\" class=\"alignnone size-medium wp-image-23333\" srcset=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/applymapping-transform-300x115.png 300w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/applymapping-transform-768x294.png 768w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/applymapping-transform-728x279.png 728w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/applymapping-transform.png 815w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>\n<p>\u30b0\u30e9\u30d5\u4e0a\u306e\u300cApplyMapping\u300d\u30ce\u30fc\u30c9\u3092\u30af\u30ea\u30c3\u30af\u3057\u3001\u5404\u7a2e\u8a2d\u5b9a\u3092\u5b9f\u65bd\u3057\u307e\u3059\u3002<\/p>\n<ul>\n<li>Transform\u30bf\u30d6\n<ul>\n<li><code>id<\/code> \u2192 <code>id<\/code> (int)<\/li>\n<li><code>name<\/code> \u2192 <code>name<\/code> (string)<\/li>\n<li><code>updatedat<\/code> \u2192 <code>updatedat<\/code> (timestamp)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<a href=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sourceschema-autorecog.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sourceschema-autorecog-300x113.png\" alt=\"\" width=\"300\" height=\"113\" class=\"alignnone size-medium wp-image-23331\" srcset=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sourceschema-autorecog-300x113.png 300w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sourceschema-autorecog-768x288.png 768w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sourceschema-autorecog-728x273.png 728w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sourceschema-autorecog.png 810w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>\n<p>\u30b0\u30e9\u30d5\u4e0a\u306e\u300cData Catalog table\u300d\u30ce\u30fc\u30c9\u3092\u30af\u30ea\u30c3\u30af\u3057\u3001\u5404\u7a2e\u8a2d\u5b9a\u3092\u5b9f\u65bd\u3057\u307e\u3059\u3002<\/p>\n<ul>\n<li>Data target properties\u30bf\u30d6\n<ul>\n<li>Database\uff1a<code>default<\/code><\/li>\n<li>Table\uff1a<code>usertarget<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<a href=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailbasic.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailbasic-176x300.png\" alt=\"\" width=\"176\" height=\"300\" class=\"alignnone size-medium wp-image-23334\" srcset=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailbasic-176x300.png 176w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailbasic-600x1024.png 600w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailbasic-768x1310.png 768w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailbasic-728x1242.png 728w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailbasic.png 802w\" sizes=\"auto, (max-width: 176px) 100vw, 176px\" \/><\/a>\n<p>\u30b8\u30e7\u30d6\u305d\u306e\u3082\u306e\u3092\u8a2d\u5b9a\u3057\u307e\u3059\u3002 (Job details\u30bf\u30d6)<\/p>\n<ul>\n<li>Basic Properties\n<ul>\n<li>Name\uff1a\u9069\u5f53\u306b\u8a2d\u5b9a\u3057\u3066\u304f\u3060\u3055\u3044<\/li>\n<li>IAM role\uff1a\u8a2d\u5b9a\u3059\u308b (\u4eca\u56de\u306e\u624b\u9806\u306b\u306f\u542b\u3081\u3066\u3044\u307e\u305b\u3093\u304c\u3001\u3042\u3063\u305f\u3082\u306e\u3092\u4f7f\u3044\u307e\u3057\u305f)<\/li>\n<li>workers\uff1a2<\/li>\n<li>Job bookmark\uff1aDisable<\/li>\n<li>retries\uff1a0<\/li>\n<li>timeout\uff1a10 (minutes)<\/li>\n<\/ul>\n<\/li>\n<li>Advanced Properties\n<ul>\n<li>Script filename\uff1ajob.py<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<a href=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailadvanced.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailadvanced-300x53.png\" alt=\"\" width=\"300\" height=\"53\" class=\"alignnone size-medium wp-image-23335\" srcset=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailadvanced-300x53.png 300w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailadvanced-768x135.png 768w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailadvanced-728x128.png 728w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/gluejobdetailadvanced.png 794w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>\n<p>\u6700\u5f8c\u306b\u5fd8\u308c\u305a\u306bSave\u3092\u62bc\u3057\u3066\u4fdd\u5b58\u3057\u307e\u3057\u3087\u3046\u3002<\/p>\n<h2>\u30b8\u30e7\u30d6\u3092\u5b9f\u884c\u3059\u308b<\/h2>\n<p>\u5b9f\u969b\u306b\u30b8\u30e7\u30d6\u3092\u5b9f\u884c\u3057\u3066\u307f\u307e\u3059\u3002\u7279\u306b\u30b3\u30fc\u30c7\u30a3\u30f3\u30b0\u3082\u305b\u305a\u3001\u30de\u30cd\u30b8\u30e1\u30f3\u30c8\u30b3\u30f3\u30bd\u30fc\u30eb\u306e\u30d5\u30a9\u30fc\u30e0\u306b\u5f93\u3063\u3066\u3044\u308b\u3060\u3051\u306a\u306e\u3067\u3001\u30b8\u30e7\u30d6\u3082\u554f\u984c\u306a\u304f\u7d42\u4e86\u3057\u307e\u3057\u305f\u3002<\/p>\n<h1>\u7d50\u679c\u306e\u78ba\u8a8d<\/h1>\n<h2>Athena\u3067\u30af\u30a8\u30ea\u3057\u3066\u307f\u308b<\/h2>\n<p>\u51fa\u6765\u4e0a\u304c\u3063\u305f\u30c7\u30fc\u30bf\u3092Athena\u304b\u3089\u30af\u30a8\u30ea\u3057\u3066\u307f\u307e\u3057\u305f\u3002<br \/>\n\u304c\u3001\u4e8b\u524d\u8abf\u67fb\u306e\u901a\u308a\u3001updatedat\u5217\u306etimestamp\u304c\u3046\u307e\u304f\u8a8d\u8b58\u3055\u308c\u3066\u3044\u306a\u3044\u3088\u3046\u3067\u3059\u3002<\/p>\n<pre>\nSELECT * FROM default.usertarget\n<\/pre>\n<pre>\n\"id\",\"name\",\"updatedat\"\n\"1\",\"\u3059\u3054\u3044 \u305f\u308d\u3046\",\n\"2\",\"\u3082\u306e\u3059\u3054\u3044 \u3058\u308d\u3046\",\n\"3\",\"\u3068\u3093\u3067\u3082\u306a\u3044 \u3055\u3076\u308d\u3046\",\n\"5\",\"\u306f\u3093\u3071\u306a\u3044 \u3054\u308d\u3046\",\n<\/pre>\n<h1>SQL Query\u5909\u63db\u3067\u65e5\u4ed8\u6587\u5b57\u5217\u3092\u8a8d\u8b58\u3055\u305b\u3066\u307f\u308b<\/h1>\n<p>\u30b9\u30e9\u30c3\u30b7\u30e5\u533a\u5207\u308a\u306e\u65e5\u4ed8\u3092timestamp\u3068\u3057\u3066\u8a8d\u8b58\u3055\u305b\u308b\u305f\u3081\u306b\u3001\u5148\u307b\u3069\u306e\u30b8\u30e7\u30d6\u3092\u3061\u3087\u3063\u3068\u3044\u3058\u308a\u307e\u3059\u3002<\/p>\n<h2>ApplyMaping\u30ce\u30fc\u30c9\u306e\u5909\u66f4<\/h2>\n<p>\u4e0b\u8a18\u306e\u90e8\u5206\u3092\u5909\u66f4\u3057\u3066\u3044\u304d\u307e\u3059\u3002<\/p>\n<ul>\n<li>Node Properties\u30bf\u30d6\n<ul>\n<li>Name\uff1aTransform<\/li>\n<li>Node type\uff1aSQL Query<\/li>\n<li>Node parents\uff1aS3bucket (\u305d\u306e\u307e\u307e)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<a href=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sqlquery-node.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sqlquery-node-300x178.png\" alt=\"\" width=\"300\" height=\"178\" class=\"alignnone size-medium wp-image-23337\" srcset=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sqlquery-node-300x178.png 300w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sqlquery-node.png 669w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>\n<ul>\n<li>Transform\u30bf\u30d6\n<ul>\n<li>Input sources\n<ul>\n<li>S3bucket\uff1a<code>myDataSource<\/code><\/li>\n<\/ul>\n<\/li>\n<li>SQL Query\uff1a<br \/><code>SELECT<br \/>\n        CAST(id AS INT) AS id,<br \/>\n        name AS name,<br \/>\n        to_timestamp(updatedat, 'yyyy\/MM\/dd HH:mm:ss') AS updatedat<br \/>\n    FROM<br \/>\n        myDataSource<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<a href=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sqlquery-transform.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sqlquery-transform-300x183.png\" alt=\"\" width=\"300\" height=\"183\" class=\"alignnone size-medium wp-image-23338\" srcset=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sqlquery-transform-300x183.png 300w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sqlquery-transform.png 668w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>\n<ul>\n<li>Output schema\u30bf\u30d6\n<ul>\n<li><code>id<\/code>\uff1a<code>int<\/code><\/li>\n<li><code>name<\/code>\uff1a<code>string<\/code><\/li>\n<li><code>updatedat<\/code>\uff1a<code>timestamp<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<a href=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sqlquery-outputschema.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sqlquery-outputschema-300x135.png\" alt=\"\" width=\"300\" height=\"135\" class=\"alignnone size-medium wp-image-23339\" srcset=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sqlquery-outputschema-300x135.png 300w, https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2022\/12\/sqlquery-outputschema.png 671w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>\n<h2>\u30b8\u30e7\u30d6\u3092\u518d\u5b9f\u884c\u3059\u308b<\/h2>\n<p>\u30b8\u30e7\u30d6\u5b9f\u884c\u306e\u524d\u306b\u3001\u51fa\u529b\u5148(\u4eca\u56de\u306e\u5834\u5408\u306f<code>s3:\/\/hogehoge-440f-413c-a52b-e7edc6f4ea8f\/usertarget\/<\/code>)\u306b\u5b58\u5728\u3059\u308b\u3001\u524d\u306e\u30b8\u30e7\u30d6\u306e\u51fa\u529b\u3067\u3042\u308bparquet\u30d5\u30a1\u30a4\u30eb\u3092\u524a\u9664\u3057\u3066\u304a\u304d\u307e\u3059\u3002<br \/>\n\u30d5\u30a1\u30a4\u30eb\u306e\u524a\u9664\u304c\u7d42\u308f\u3063\u305f\u3089\u3001\u6539\u3081\u3066\u30b8\u30e7\u30d6\u3092\u5b9f\u884c\u3057\u307e\u3059\u3002<br \/>\n\u4eca\u56de\u306fSQL Query\u306e\u90e8\u5206\u306b\u9650\u3063\u3066\u306f\u30b3\u30fc\u30c9\u3092\u6295\u5165\u3057\u3066\u3044\u307e\u3059\u304c\u3001\u305d\u308c\u4ee5\u5916\u306f\u524d\u306e\u30b8\u30e7\u30d6\u3068\u540c\u3058\u3067\u3059\u306d\u3002\u6700\u521d\u306e\u30b8\u30e7\u30d6\u3088\u308a\u591a\u5c11\u6642\u9593\u304c\u304b\u304b\u3063\u305f\u5370\u8c61\u306f\u3042\u308a\u307e\u3059\u304c\u3001\u7121\u4e8b\u306b\u5b9f\u884c\u3055\u308c\u307e\u3057\u305f\u3002<\/p>\n<h1>\u7d50\u679c\u306e\u518d\u78ba\u8a8d<\/h1>\n<p>\u51fa\u6765\u4e0a\u304c\u3063\u305f\u30c7\u30fc\u30bf\u3092Athena\u304b\u3089\u30af\u30a8\u30ea\u3057\u3066\u307f\u307e\u3057\u305f\u3002<br \/>\n\u524d\u3068\u306f\u9055\u3044\u3001\u65e5\u6642\u306e\u6587\u5b57\u5217\u304c\u6b63\u3057\u304ftimestamp\u578b\u3068\u3057\u3066\u30af\u30a8\u30ea\u51fa\u6765\u3066\u3044\u307e\u3059\u3002<\/p>\n<pre>\nSELECT * FROM default.usertarget\n<\/pre>\n<pre>\n\"id\",\"name\",\"updatedat\"\n\"1\",\"\u3059\u3054\u3044 \u305f\u308d\u3046\",\"2020-01-15 03:44:40.000\"\n\"2\",\"\u3082\u306e\u3059\u3054\u3044 \u3058\u308d\u3046\",\"2022-10-12 05:12:59.000\"\n\"3\",\"\u3068\u3093\u3067\u3082\u306a\u3044 \u3055\u3076\u308d\u3046\",\"2022-03-15 12:34:56.000\"\n\"5\",\"\u306f\u3093\u3071\u306a\u3044 \u3054\u308d\u3046\",\"2021-10-29 23:55:10.000\"\n<\/pre>\n<h1>\u307e\u3068\u3081<\/h1>\n<p>ApplyMapping\u5909\u63db\u3067\u306f\u5bfe\u5fdc\u3067\u304d\u306a\u3044\u65e5\u6642\u5f62\u5f0f\u306e\u5834\u5408\u306b\u4f7f\u3048\u308b\u624b\u6cd5\u306e\u4e00\u3064\u3068\u3057\u3066\u3001SQL Query\u5909\u63db\u306b\u3088\u308b\u30ab\u30b9\u30bf\u30e0\u65e5\u4ed8\u5f62\u5f0f\u306e\u8a8d\u8b58\u65b9\u6cd5\u3092\u3054\u7d39\u4ecb\u3057\u307e\u3057\u305f\u3002<br \/>\n\u5909\u63db\u306b\u306f\u4ed6\u306b\u3082\u3044\u308d\u3044\u308d\u7528\u610f\u3055\u308c\u3066\u3044\u307e\u3059\u3057\u3001\u8907\u6570\u306e\u5165\u529b\u3092\u30de\u30fc\u30b8\u3057\u3066\u5909\u63db\u3092\u66f8\u3051\u308b\u3053\u3068\u3082\u3067\u304d\u307e\u3059\u306e\u3067\u3001\u307f\u306a\u3055\u3093\u3082\u305c\u3072\u8a66\u3057\u3066\u307f\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Glue\u30b8\u30e7\u30d6\u3067\u65e5\u4ed8\u5f62\u5f0f\u306e\u6587\u5b57\u5217\u3092ApplyMapping\u5909\u63db\u3067\u8a8d\u8b58\u3055\u305b\u3088\u3046\u3068\u3057\u305f\u3089\u3046\u307e\u304f\u884c\u304b\u306a\u304b\u3063\u305f\u306e\u3067\u3001\u4ee3\u308f\u308a\u306bSQL Query\u3092\u4f7f\u7528\u3057\u3066\u8a8d\u8b58\u3055\u305b\u3066\u307f\u307e\u3057\u305f\u3002 \u3084\u308a\u305f\u3044\u3053\u3068 \u5165\u529b\u30c7\u30fc\u30bf\u306eCSV\u3092Glue\u30b8\u30e7\u30d6\u3067\u5909\u63db&#8230;<\/p>\n","protected":false},"author":205,"featured_media":23359,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_locale":"ja","_original_post":"https:\/\/www.skyarch.net\/blog\/?p=23325","footnotes":""},"categories":[1192,20,1193],"tags":[369],"class_list":{"0":"post-23325","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-athena","8":"category-aws","9":"category-glue","10":"tag-aws-glue","11":"ja"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/23325","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/users\/205"}],"replies":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/comments?post=23325"}],"version-history":[{"count":16,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/23325\/revisions"}],"predecessor-version":[{"id":23415,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/23325\/revisions\/23415"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/media\/23359"}],"wp:attachment":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/media?parent=23325"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/categories?post=23325"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/tags?post=23325"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}