{"id":11634,"date":"2018-08-13T08:06:29","date_gmt":"2018-08-13T08:06:29","guid":{"rendered":"http:\/\/stblog.lunaeme.com\/?p=11634"},"modified":"2023-09-20T13:34:32","modified_gmt":"2023-09-20T13:34:32","slug":"wild-data-part-two-transfer-style","status":"publish","type":"post","link":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/","title":{"rendered":"Wild Data Part 2: Transfer Style"},"content":{"rendered":"

This is the second post of our Wild Data series<\/a>. In this post, we are going to expose how to transfer style from one image to another.<\/a> Here, the most interesting point is to know that we won\u2019t use a neural network to classify a set of classes as usual. I mean, we don\u2019t need to train a network for a specific approach. Transfer style is based on pre-trained networks such as it could be a VGG19<\/a> trained with ImageNet <\/a>(one million of images). Thus, a good understanding of transfer style will help you to better understand how convolutional neural networks works for vision. Let\u2019s go on!<\/p>\n

<\/p>\n

Transfer Style<\/h2>\n

The main idea\/solution is very intuitive:<\/p>\n

1. Let\u2019s extract content<\/strong> from an original photo.
\n2. Let\u2019s extract style<\/strong> from a reference photo.
\n3. Now combine content and style together to get a new \u201cstyled\u201d result.<\/p>\n

\"\"<\/p>\n

How do we really get these results? We start from a noise image, and in each epoch in which the algorithm combines both images, we are simply taking advantage of backpropagation to minimize two defined cost function values (content and style). Since we have a total loss function that depends on the combined image, the optimization algorithm will tell us how to change our noise image, in order to obtain a combination image to make the loss a little smaller.<\/p>\n

\"\"<\/p>\n

You may wonder why we use the results of the middle layers of a neural network to classify previously trained images to calculate our losses in style and content. That\u2019s why, the network has to understand the image to perform the function for which it was built and trained<\/strong>. Thus, among all the layers between the input and the output, the network is doing transformations to convert the image pixels into an internal understanding of the image content, and we can use it to extract content and style.<\/p>\n

In most cases, what is more typically used is gradient descent: by using first-order methods, on which backpropagation can be applied, or by using second-order methods, based on the hessian<\/a> or an approximation of it (for instance, information from the second derivative). In addition, we could apply numerical methods to do this without the need to calculate the exact hessian (because it has a quadratic complexity and is very complicated). Besides, it should be pointed out that each method will have a different image result.<\/p>\n

\"\"<\/p>\n

One of the most fascinating concepts about deep learning is that each layer gets a data representation focused on the objective of the problem to be solved<\/strong>. Here, the network as a whole generates an idea derived from different faces. Thereby, lower convolutional layers capture low-level image features (for instance, edges, textures and colours) while higher convolutional layers capture more and more complex details, such as face features and, at the end, different faces.<\/p>\n

As you can see, last layers contains these internal understandings of the content of the images. If we go through the previous neural network with two different images of faces, this internal understanding will be very close because the activations have been similar. Therefore, euclidean distance<\/a> between the intermediate representations will be lower, and this is what will be used to bring the noise image closer and closer to the reference image in each epoch.<\/p>\n

Before, we have tested what an ice Balrog might look like. Likewise, we are going to continue with what would happen if we combined Khaleesi’s face with a white walker\u2019s face. Don’t worry, no spoilers of Game of Thrones here:<\/p>\n

\"\"<\/p>\n

Epic fail! it is all due to the way in which the style was transferred through gram matrix <\/a>(we could use other methods as well), by using correlation in the way of multiplication between different weight values in the filters. Therefore, information about the semantics of the image was lost<\/strong>, because of retaining just basic components of the texture\/ambiance\/appearance, in contrast, we cannot transfer the blue color of the eyes.<\/p>\n

In conclusion, transfer style is always interesting to better understand the nuts and bolts of convolutional neural networks, and it might be interesting in topics related with data augmentation. But, this approach probably will not be enough to address these problems and often, the results are… too artistic, although it is true you could ensemble them with others methods to improve the results. We could also introduce the awesome GANs, which deserve a whole new post. And…just one more thing, keep in mind this quote from Hacker Moon<\/a> that I really like: \u201cIf you really think about it, all human evolution is really nothing more than abstracting problems and automating solutions to earlier problems, which leads to new problems and new solutions in a never ending cycle\u201d.<\/p>\n

If you want to learn more about Deep Learning<\/a>, Data Augmentation<\/a> and Transfer Style, take a look at this Data Science Meetup<\/a> we organized at Campus Madrid:<\/p>\n

– Video<\/a>
\n– Presentation<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Transfer Style allows to use the inner understanding of an already trained Convolutional Neural Network to transfer style from one picture to another.<\/p>\n","protected":false},"author":1,"featured_media":13544,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[686],"tags":[19],"ppma_author":[795],"yoast_head":"\nWild Data - Transfer Style - Deep Learning - Blog<\/title>\n<meta name=\"description\" content=\"Transfer Style allows to use the inner understanding of an already trained Convolutional Neural Network to transfer style from one picture to another.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Wild Data Part 2: Transfer Style\" \/>\n<meta property=\"og:description\" content=\"Transfer Style allows to use the inner understanding of an already trained Convolutional Neural Network to transfer style from one picture to another.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/\" \/>\n<meta property=\"og:site_name\" content=\"Stratio\" \/>\n<meta property=\"article:published_time\" content=\"2018-08-13T08:06:29+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-09-20T13:34:32+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2018\/08\/Wild-Data-Part-2-Transfer-Style.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1300\" \/>\n\t<meta property=\"og:image:height\" content=\"820\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Stratio\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@stratiobd\" \/>\n<meta name=\"twitter:site\" content=\"@stratiobd\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Stratio\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/\"},\"author\":{\"name\":\"Stratio\",\"@id\":\"https:\/\/www.stratio.com\/blog\/#\/schema\/person\/d0377b199cd052b17e15c9ba44c45ab7\"},\"headline\":\"Wild Data Part 2: Transfer Style\",\"datePublished\":\"2018-08-13T08:06:29+00:00\",\"dateModified\":\"2023-09-20T13:34:32+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/\"},\"wordCount\":829,\"publisher\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2018\/08\/Wild-Data-Part-2-Transfer-Style.jpg\",\"keywords\":[\"Big Data\"],\"articleSection\":[\"Product\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/\",\"url\":\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/\",\"name\":\"Wild Data - Transfer Style - Deep Learning - Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2018\/08\/Wild-Data-Part-2-Transfer-Style.jpg\",\"datePublished\":\"2018-08-13T08:06:29+00:00\",\"dateModified\":\"2023-09-20T13:34:32+00:00\",\"description\":\"Transfer Style allows to use the inner understanding of an already trained Convolutional Neural Network to transfer style from one picture to another.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#primaryimage\",\"url\":\"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2018\/08\/Wild-Data-Part-2-Transfer-Style.jpg\",\"contentUrl\":\"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2018\/08\/Wild-Data-Part-2-Transfer-Style.jpg\",\"width\":1300,\"height\":820},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.stratio.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Wild Data Part 2: Transfer Style\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.stratio.com\/blog\/#website\",\"url\":\"https:\/\/www.stratio.com\/blog\/\",\"name\":\"Stratio Blog\",\"description\":\"Corporate blog\",\"publisher\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.stratio.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.stratio.com\/blog\/#organization\",\"name\":\"Stratio\",\"url\":\"https:\/\/www.stratio.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.stratio.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/stratio.com\/blog\/wp-content\/uploads\/2020\/06\/stratio-web-logo-1.png\",\"contentUrl\":\"https:\/\/stratio.com\/blog\/wp-content\/uploads\/2020\/06\/stratio-web-logo-1.png\",\"width\":260,\"height\":55,\"caption\":\"Stratio\"},\"image\":{\"@id\":\"https:\/\/www.stratio.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/stratiobd\",\"https:\/\/es.linkedin.com\/company\/stratiobd\",\"https:\/\/www.youtube.com\/c\/StratioBD\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.stratio.com\/blog\/#\/schema\/person\/d0377b199cd052b17e15c9ba44c45ab7\",\"name\":\"Stratio\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.stratio.com\/blog\/#\/schema\/person\/image\/bb38888f58c2bb664646155f78ae6ccc\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e3387ad00609f34a56d6796400eb8191?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e3387ad00609f34a56d6796400eb8191?s=96&d=mm&r=g\",\"caption\":\"Stratio\"},\"description\":\"Stratio guides businesses on their journey through complete #DigitalTransformation with #BigData and #AI. Stratio works worldwide for large companies and multinationals in the sectors of banking, insurance, healthcare, telco, retail, energy and media.\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Wild Data - Transfer Style - Deep Learning - Blog","description":"Transfer Style allows to use the inner understanding of an already trained Convolutional Neural Network to transfer style from one picture to another.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/","og_locale":"en_US","og_type":"article","og_title":"Wild Data Part 2: Transfer Style","og_description":"Transfer Style allows to use the inner understanding of an already trained Convolutional Neural Network to transfer style from one picture to another.","og_url":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/","og_site_name":"Stratio","article_published_time":"2018-08-13T08:06:29+00:00","article_modified_time":"2023-09-20T13:34:32+00:00","og_image":[{"width":1300,"height":820,"url":"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2018\/08\/Wild-Data-Part-2-Transfer-Style.jpg","type":"image\/jpeg"}],"author":"Stratio","twitter_card":"summary_large_image","twitter_creator":"@stratiobd","twitter_site":"@stratiobd","twitter_misc":{"Written by":"Stratio","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#article","isPartOf":{"@id":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/"},"author":{"name":"Stratio","@id":"https:\/\/www.stratio.com\/blog\/#\/schema\/person\/d0377b199cd052b17e15c9ba44c45ab7"},"headline":"Wild Data Part 2: Transfer Style","datePublished":"2018-08-13T08:06:29+00:00","dateModified":"2023-09-20T13:34:32+00:00","mainEntityOfPage":{"@id":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/"},"wordCount":829,"publisher":{"@id":"https:\/\/www.stratio.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#primaryimage"},"thumbnailUrl":"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2018\/08\/Wild-Data-Part-2-Transfer-Style.jpg","keywords":["Big Data"],"articleSection":["Product"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/","url":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/","name":"Wild Data - Transfer Style - Deep Learning - Blog","isPartOf":{"@id":"https:\/\/www.stratio.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#primaryimage"},"image":{"@id":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#primaryimage"},"thumbnailUrl":"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2018\/08\/Wild-Data-Part-2-Transfer-Style.jpg","datePublished":"2018-08-13T08:06:29+00:00","dateModified":"2023-09-20T13:34:32+00:00","description":"Transfer Style allows to use the inner understanding of an already trained Convolutional Neural Network to transfer style from one picture to another.","breadcrumb":{"@id":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#primaryimage","url":"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2018\/08\/Wild-Data-Part-2-Transfer-Style.jpg","contentUrl":"https:\/\/www.stratio.com\/blog\/wp-content\/uploads\/2018\/08\/Wild-Data-Part-2-Transfer-Style.jpg","width":1300,"height":820},{"@type":"BreadcrumbList","@id":"https:\/\/www.stratio.com\/blog\/wild-data-part-two-transfer-style\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.stratio.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Wild Data Part 2: Transfer Style"}]},{"@type":"WebSite","@id":"https:\/\/www.stratio.com\/blog\/#website","url":"https:\/\/www.stratio.com\/blog\/","name":"Stratio Blog","description":"Corporate blog","publisher":{"@id":"https:\/\/www.stratio.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.stratio.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.stratio.com\/blog\/#organization","name":"Stratio","url":"https:\/\/www.stratio.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.stratio.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/stratio.com\/blog\/wp-content\/uploads\/2020\/06\/stratio-web-logo-1.png","contentUrl":"https:\/\/stratio.com\/blog\/wp-content\/uploads\/2020\/06\/stratio-web-logo-1.png","width":260,"height":55,"caption":"Stratio"},"image":{"@id":"https:\/\/www.stratio.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/stratiobd","https:\/\/es.linkedin.com\/company\/stratiobd","https:\/\/www.youtube.com\/c\/StratioBD"]},{"@type":"Person","@id":"https:\/\/www.stratio.com\/blog\/#\/schema\/person\/d0377b199cd052b17e15c9ba44c45ab7","name":"Stratio","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.stratio.com\/blog\/#\/schema\/person\/image\/bb38888f58c2bb664646155f78ae6ccc","url":"https:\/\/secure.gravatar.com\/avatar\/e3387ad00609f34a56d6796400eb8191?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e3387ad00609f34a56d6796400eb8191?s=96&d=mm&r=g","caption":"Stratio"},"description":"Stratio guides businesses on their journey through complete #DigitalTransformation with #BigData and #AI. Stratio works worldwide for large companies and multinationals in the sectors of banking, insurance, healthcare, telco, retail, energy and media."}]}},"authors":[{"term_id":795,"user_id":1,"is_guest":0,"slug":"stratioadmin","display_name":"Stratio","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/e3387ad00609f34a56d6796400eb8191?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/posts\/11634"}],"collection":[{"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/comments?post=11634"}],"version-history":[{"count":21,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/posts\/11634\/revisions"}],"predecessor-version":[{"id":13545,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/posts\/11634\/revisions\/13545"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/media\/13544"}],"wp:attachment":[{"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/media?parent=11634"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/categories?post=11634"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/tags?post=11634"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.stratio.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=11634"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}