{"id":8491,"date":"2016-09-27T21:52:51","date_gmt":"2016-09-27T12:52:51","guid":{"rendered":"http:\/\/www.skyarch.net\/blog\/?p=8491"},"modified":"2021-11-02T17:27:40","modified_gmt":"2021-11-02T08:27:40","slug":"getting-to-know-messagepack-as-an-efficient-alternative-to-json","status":"publish","type":"post","link":"https:\/\/www.skyarch.net\/blog\/en\/getting-to-know-messagepack-as-an-efficient-alternative-to-json\/","title":{"rendered":"Getting to know MessagePack as an efficient alternative to JSON"},"content":{"rendered":"<p><a href=\"http:\/\/www.json.org\/\">JSON or JavaScript Object Notation<\/a>\u00a0is now one of the most popular formats\u00a0of transmitting data objects, almost if not entirely replacing XML, because of its \"very\" human-readable, standard text format using the key-value pair structure. \u00a0The language-indepent data format was\u00a0based on a subset of the <a href=\"http:\/\/www.ecma-international.org\/publications\/files\/ECMA-ST\/Ecma-262.pdf\">JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999<\/a>\u00a0and was later popularized by\u00a0<a title=\"Douglas Crockford\" href=\"https:\/\/en.wikipedia.org\/wiki\/Douglas_Crockford\">Douglas Crockford<\/a>\u00a0on around 2001.<\/p>\n<p>Today, JSON is already a standard and is used in many systems, in web, mobile, websockets, etc., even in Relation Database Management Systems and NoSQL databases.<\/p>\n<p>But the innovation did not stop with JSON. Others have created alternatives to minimize the unnecessary characters and the size of the data object\/s. One of them is <a href=\"http:\/\/msgpack.org\/\">MessagePack<\/a>.<\/p>\n<blockquote>\n<h5 class=\"i18n\">It's like JSON.<br \/>\nbut fast and small.<\/h5>\n<p class=\"i18n\">MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it's faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves.<\/p>\n<\/blockquote>\n<p class=\"i18n\">MessagePack, the data is compact and therefore faster to transmit, in any way you want to transmit it, in bytes instead of string. And the beauty of it is that serialization and deserialization is fast.<\/p>\n<h4 class=\"i18n\">Usages of MessagePack<\/h4>\n<p>You can use the MessagePack Specification in almost anywhere you want to use it. Some have used them in together with caching, rpc's, sockets, and data representation across multiple platforms.<br \/>\nOthers have used MessagePack with Redis, with Memcached, and there is also an RPC project based on MessagePack.<\/p>\n<h5>Why use MessagePack?<\/h5>\n<ul>\n<li>It is small in size, compact, and therefore efficient<\/li>\n<li>A little more efficient<\/li>\n<li>You can create application-specific type using its <code>ext<\/code>\u00a0type family<\/li>\n<li>And what you can do with JSON you can do with MessagePack<\/li>\n<\/ul>\n<p><em>Just for the record, MessagePack was not made mainly for consumer-facing API's.<\/em><\/p>\n<h5>What languages support it?<\/h5>\n<p>Basically, MessagePack specification is just encoding data into bytes, so most of, if not all, languages supports it. It's just the matter of implementing it. But don't worry! most of the popular languages already has MessagePack library, <a href=\"http:\/\/msgpack.org\/index.html#languages\">check this out<\/a>.<\/p>\n<p>MessagePack's specification is properly documented in <a href=\"https:\/\/github.com\/msgpack\/msgpack\/blob\/master\/spec.md\">here<\/a>.<\/p>\n<h5>Usage Insight<\/h5>\n<p>Here is a little example of messagepack compared to json, serialization and deserialization,\u00a0using the python <code>msgpack<\/code> and <code>json<\/code> library.<\/p>\n<p>Install the library using <code>pip<\/code>:<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">$ pip install msgpack-python<\/pre>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\n#!\/usr\/bin\/python\r\n\r\n# msgpack-test.py\r\n\r\nimport msgpack\r\nimport timeit\r\n\r\n\r\nresult = msgpack.packb(&#x5B;1, 2, 3])\r\nprint 'output:', result\r\nprint &quot;Avg time x 10e3 msgpack:&quot;, timeit.timeit(&quot;import msgpack; msgpack.packb(&#x5B;1, 2, 3])&quot;, number=10000)\r\nprint &quot;Avg time x 10e3 json:&quot;, timeit.timeit(&quot;import json; json.dumps(&#x5B;1, 2, 3])&quot;, number=10000)\r\n# output: '\\x93\\x01\\x02\\x03'\r\n\r\nresult = msgpack.unpackb(b'\\x93\\x01\\x02\\x03')\r\nprint '\\noutput:', result\r\nprint &quot;Avg time x 10e3 msgpack:&quot;, timeit.timeit(&quot;import msgpack; msgpack.unpackb(b'\\x93\\x01\\x02\\x03')&quot;, number=10000)\r\nprint &quot;Avg time x 10e3 json:&quot;, timeit.timeit(&quot;import json; json.loads('&#x5B;1, 2, 3]')&quot;, number=10000)\r\n# output: &#x5B;1, 2, 3]\r\n\r\nresult = msgpack.unpackb(b'\\x93\\x01\\x02\\x03', use_list=False)\r\nprint '\\noutput:', result\r\nprint &quot;Avg time x 10e3 msgpack:&quot;, timeit.timeit(&quot;import msgpack; msgpack.unpackb(b'\\x93\\x01\\x02\\x03', use_list=False)&quot;, number=10000)\r\n# output: (1, 2, 3)\r\n<\/pre>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">\r\n$ python sample.py\r\noutput: \u2592\r\nAvg time x 10e3 msgpack: 0.129597902298\r\nAvg time x 10e3 json: 0.0673019886017\r\n\r\noutput: &#x5B;1, 2, 3]\r\nAvg time x 10e3 msgpack: 0.17978978157\r\nAvg time x 10e3 json: 0.0450839996338\r\n\r\noutput: (1, 2, 3)\r\nAvg time x 10e3 msgpack: 0.19033408165\r\n<\/pre>\n<h4>Conclusion<\/h4>\n<p>Use MessagePack depending on your needs and if your specification says that size should really matter. Remember that it was not created mainly for consumer-facing API's. For me, I would use MessagePack in sockets, rpc, caching mechanisms and data representation. There are other more ways to utilize MessagePack.<\/p>\n<p>You would want to experiment first to get the grasp of it and decide on how\u00a0to utilize it properly. Research\u00a0more about it too.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>JSON or JavaScript Object Notation\u00a0is now one of the most popular formats\u00a0of transmitting data objects, almost&#8230;<\/p>\n","protected":false},"author":1,"featured_media":8501,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_locale":"en_US","_original_post":"8491","footnotes":""},"categories":[9],"tags":[162,163,164,161],"class_list":{"0":"post-8491","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-dev","8":"tag-data-objects","9":"tag-json","10":"tag-messagepack","11":"tag-serialization","12":"en-US"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/8491","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/comments?post=8491"}],"version-history":[{"count":10,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/8491\/revisions"}],"predecessor-version":[{"id":8503,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/8491\/revisions\/8503"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/media\/8501"}],"wp:attachment":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/media?parent=8491"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/categories?post=8491"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/tags?post=8491"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}