{"id":1404,"date":"2015-03-09T17:20:05","date_gmt":"2015-03-09T17:20:05","guid":{"rendered":"http:\/\/blog.stratio.com\/?p=29"},"modified":"2023-09-20T13:45:54","modified_gmt":"2023-09-20T13:45:54","slug":"spark-mongodb-library","status":"publish","type":"post","link":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/","title":{"rendered":"Spark-MongoDB library"},"content":{"rendered":"<p style=\"text-align: justify;\">Once the Data Sources API has been released, we&#8217;ve wanted to take advantage of these new features and, for this reason, we have developed a\u00a0<strong>Spark-MongoDB<\/strong>\u00a0library. With this new connector we help the growing MongoDB community to simplify the interaction with this datasource via Spark.<br \/>\n<!--more--><\/p>\n<p style=\"text-align: justify;\">This library provides the mechanism for accessing\u00a0<strong>MongoDB<\/strong>\u00a0collections in a structured way from SparkSQL, accesible from Python and Scala API&#8217;s. Since MongoDB is an open-source document database leader among NoSQL databases and is highly used in several projects [<a href=\"http:\/\/www.mongodb.com\/leading-nosql-database\" target=\"_blank\" rel=\"noopener noreferrer\">http:\/\/www.mongodb.com\/leading-nosql-database<\/a>] we find this connection with all the operations permitted by SparkSQL not only useful but necessary.<\/p>\n<p style=\"text-align: justify;\">Our library uses the 2.13.0 MongoDB Java Driver (that supports the newest MongoDB versions). We use the\u00a0<strong>Casbah<\/strong>\u00a0toolkit in order to better integrate our Scala implementation with MongoDB. Thus, the project becomes cleaner and less verbose while allowing for a simpler and more intuitive way of developing.<\/p>\n<p style=\"text-align: justify;\"><strong>SparkSQL<\/strong>\u00a0is being rapidly developed,\u00a0 giving support for reading data from other formats (Apache Hive, Parquet, &#8230;) and the chance of performing many operations with this data. With our library we extend these possibilities by adding other datasource with which the user could combine existing data in other formats.<\/p>\n<p style=\"text-align: justify;\">We are looking forward for the new Spark 1.3 to keep updating and evolving our library.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Once the Data Sources API has been released, we&#8217;ve wanted to take advantage of these new features and, for this reason, we have developed a\u00a0Spark-MongoDB\u00a0library. With this new connector we help the growing MongoDB community to simplify the interaction with this datasource via Spark.<\/p>\n","protected":false},"author":1,"featured_media":119,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[686],"tags":[85],"ppma_author":[795],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v22.9 (Yoast SEO v22.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Spark-MongoDB library - Stratio<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Spark-MongoDB library\" \/>\n<meta property=\"og:description\" content=\"Once the Data Sources API has been released, we&#039;ve wanted to take advantage of these new features and, for this reason, we have developed a\u00a0Spark-MongoDB\u00a0library. With this new connector we help the growing MongoDB community to simplify the interaction with this datasource via Spark.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/\" \/>\n<meta property=\"og:site_name\" content=\"Stratio\" \/>\n<meta property=\"article:published_time\" content=\"2015-03-09T17:20:05+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-09-20T13:45:54+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2015\/03\/Spark.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"730\" \/>\n\t<meta property=\"og:image:height\" content=\"312\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Stratio\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@stratiobd\" \/>\n<meta name=\"twitter:site\" content=\"@stratiobd\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Stratio\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/\"},\"author\":{\"name\":\"Stratio\",\"@id\":\"https:\/\/www.stratio.com\/blog\/#\/schema\/person\/d0377b199cd052b17e15c9ba44c45ab7\"},\"headline\":\"Spark-MongoDB library\",\"datePublished\":\"2015-03-09T17:20:05+00:00\",\"dateModified\":\"2023-09-20T13:45:54+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/\"},\"wordCount\":220,\"publisher\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2015\/03\/Spark.jpg\",\"keywords\":[\"spark\"],\"articleSection\":[\"Product\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/\",\"url\":\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/\",\"name\":\"Spark-MongoDB library - Stratio\",\"isPartOf\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2015\/03\/Spark.jpg\",\"datePublished\":\"2015-03-09T17:20:05+00:00\",\"dateModified\":\"2023-09-20T13:45:54+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#primaryimage\",\"url\":\"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2015\/03\/Spark.jpg\",\"contentUrl\":\"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2015\/03\/Spark.jpg\",\"width\":730,\"height\":312},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.stratio.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Spark-MongoDB library\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.stratio.com\/blog\/#website\",\"url\":\"https:\/\/www.stratio.com\/blog\/\",\"name\":\"Stratio Blog\",\"description\":\"Corporate blog\",\"publisher\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.stratio.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.stratio.com\/blog\/#organization\",\"name\":\"Stratio\",\"url\":\"https:\/\/www.stratio.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.stratio.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/stratio.com\/blog\/wp-content\/uploads\/2020\/06\/stratio-web-logo-1.png\",\"contentUrl\":\"https:\/\/stratio.com\/blog\/wp-content\/uploads\/2020\/06\/stratio-web-logo-1.png\",\"width\":260,\"height\":55,\"caption\":\"Stratio\"},\"image\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/stratiobd\",\"https:\/\/es.linkedin.com\/company\/stratiobd\",\"https:\/\/www.youtube.com\/c\/StratioBD\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.stratio.com\/blog\/#\/schema\/person\/d0377b199cd052b17e15c9ba44c45ab7\",\"name\":\"Stratio\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.stratio.com\/blog\/#\/schema\/person\/image\/bb38888f58c2bb664646155f78ae6ccc\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e3387ad00609f34a56d6796400eb8191?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e3387ad00609f34a56d6796400eb8191?s=96&d=mm&r=g\",\"caption\":\"Stratio\"},\"description\":\"Stratio guides businesses on their journey through complete #DigitalTransformation with #BigData and #AI. Stratio works worldwide for large companies and multinationals in the sectors of banking, insurance, healthcare, telco, retail, energy and media.\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Spark-MongoDB library - Stratio","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/","og_locale":"en_US","og_type":"article","og_title":"Spark-MongoDB library","og_description":"Once the Data Sources API has been released, we've wanted to take advantage of these new features and, for this reason, we have developed a\u00a0Spark-MongoDB\u00a0library. With this new connector we help the growing MongoDB community to simplify the interaction with this datasource via Spark.","og_url":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/","og_site_name":"Stratio","article_published_time":"2015-03-09T17:20:05+00:00","article_modified_time":"2023-09-20T13:45:54+00:00","og_image":[{"width":730,"height":312,"url":"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2015\/03\/Spark.jpg","type":"image\/jpeg"}],"author":"Stratio","twitter_card":"summary_large_image","twitter_creator":"@stratiobd","twitter_site":"@stratiobd","twitter_misc":{"Written by":"Stratio","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#article","isPartOf":{"@id":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/"},"author":{"name":"Stratio","@id":"https:\/\/www.stratio.com\/blog\/#\/schema\/person\/d0377b199cd052b17e15c9ba44c45ab7"},"headline":"Spark-MongoDB library","datePublished":"2015-03-09T17:20:05+00:00","dateModified":"2023-09-20T13:45:54+00:00","mainEntityOfPage":{"@id":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/"},"wordCount":220,"publisher":{"@id":"https:\/\/www.stratio.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#primaryimage"},"thumbnailUrl":"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2015\/03\/Spark.jpg","keywords":["spark"],"articleSection":["Product"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/","url":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/","name":"Spark-MongoDB library - Stratio","isPartOf":{"@id":"https:\/\/www.stratio.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#primaryimage"},"image":{"@id":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#primaryimage"},"thumbnailUrl":"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2015\/03\/Spark.jpg","datePublished":"2015-03-09T17:20:05+00:00","dateModified":"2023-09-20T13:45:54+00:00","breadcrumb":{"@id":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#primaryimage","url":"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2015\/03\/Spark.jpg","contentUrl":"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2015\/03\/Spark.jpg","width":730,"height":312},{"@type":"BreadcrumbList","@id":"https:\/\/www.stratio.com\/blog\/spark-mongodb-library\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.stratio.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Spark-MongoDB library"}]},{"@type":"WebSite","@id":"https:\/\/www.stratio.com\/blog\/#website","url":"https:\/\/www.stratio.com\/blog\/","name":"Stratio Blog","description":"Corporate blog","publisher":{"@id":"https:\/\/www.stratio.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.stratio.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.stratio.com\/blog\/#organization","name":"Stratio","url":"https:\/\/www.stratio.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.stratio.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/stratio.com\/blog\/wp-content\/uploads\/2020\/06\/stratio-web-logo-1.png","contentUrl":"https:\/\/stratio.com\/blog\/wp-content\/uploads\/2020\/06\/stratio-web-logo-1.png","width":260,"height":55,"caption":"Stratio"},"image":{"@id":"https:\/\/www.stratio.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/stratiobd","https:\/\/es.linkedin.com\/company\/stratiobd","https:\/\/www.youtube.com\/c\/StratioBD"]},{"@type":"Person","@id":"https:\/\/www.stratio.com\/blog\/#\/schema\/person\/d0377b199cd052b17e15c9ba44c45ab7","name":"Stratio","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.stratio.com\/blog\/#\/schema\/person\/image\/bb38888f58c2bb664646155f78ae6ccc","url":"https:\/\/secure.gravatar.com\/avatar\/e3387ad00609f34a56d6796400eb8191?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e3387ad00609f34a56d6796400eb8191?s=96&d=mm&r=g","caption":"Stratio"},"description":"Stratio guides businesses on their journey through complete #DigitalTransformation with #BigData and #AI. Stratio works worldwide for large companies and multinationals in the sectors of banking, insurance, healthcare, telco, retail, energy and media."}]}},"authors":[{"term_id":795,"user_id":1,"is_guest":0,"slug":"stratioadmin","display_name":"Stratio","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/e3387ad00609f34a56d6796400eb8191?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/posts\/1404"}],"collection":[{"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/comments?post=1404"}],"version-history":[{"count":6,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/posts\/1404\/revisions"}],"predecessor-version":[{"id":13861,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/posts\/1404\/revisions\/13861"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/media\/119"}],"wp:attachment":[{"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/media?parent=1404"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/categories?post=1404"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/tags?post=1404"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=1404"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}