{"id":1538,"date":"2020-03-17T22:02:44","date_gmt":"2020-03-18T05:02:44","guid":{"rendered":"http:\/\/otherthings.com\/blog\/?p=1538"},"modified":"2020-03-17T22:02:44","modified_gmt":"2020-03-18T05:02:44","slug":"machine-learning-and-creativity","status":"publish","type":"post","link":"https:\/\/otherthings.com\/blog\/2020\/03\/machine-learning-and-creativity\/","title":{"rendered":"Machine Learning and Creativity"},"content":{"rendered":"\n<p>It&#8217;s an interesting time to be an artist. As machine learning becomes part of the toolkit, in different ways for different people, new ideas are shaking loose, and I feel compelled to write about them as a way of wrapping my head around the whole thing.<\/p>\n\n\n\n<figure class=\"wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Music and Machine Learning (Google I\/O&#039;19)\" width=\"474\" height=\"267\" src=\"https:\/\/www.youtube.com\/embed\/pM9u9xcM_cs?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p>The most recent headquake hit me by way of the ML-assisted album <em><a href=\"https:\/\/arstechnica.com\/gaming\/2019\/08\/yachts-chain-tripping-is-a-new-landmark-for-ai-music-an-album-that-doesnt-suck\/\">Chain\u00a0Tripping<\/a><\/em> by post-punk-pop band YACHT.  Here&#8217;s a great <a href=\"https:\/\/www.youtube.com\/watch?v=pM9u9xcM_cs\">Google I\/O talk by bandleader Claire Evans<\/a> that describes just how they made it. (Tl;dr: no, the machines are not coming for your jobs, songwriters! Using ML actually made the process <em>slower:<\/em> it took them three years to finish the album.)  This case is interesting for what it tells us about not just the limitations of current AI techniques, but also the creative process, and what makes people enjoy music. <\/p>\n\n\n\n<p>In music there\u2019s this idea that enjoyment comes from a combination of the familiar and the unexpected. For example, a familiar arrangement of instruments, a familiar playing style, with a surprising melody or bass line. Maybe it works like <a href=\"https:\/\/arxiv.org\/pdf\/1910.04639.pdf\">visual indeterminacy<\/a>: it keeps you interested by keeping you guessing.<\/p>\n\n\n\n<p>As genres go, pop music is particularly information-sparse. What I take from YACHT&#8217;s example is that low level noise\u2014 nearly random arrangements of words and notes\u2014 can produce occasional bursts of semi-intelligible stuff. By manually curating the best of that stuff and arranging it, they pushed the surprise factor well above the threshold of enjoyability for a pop song. And then they provided the familiarity piece by playing their instruments and singing in their own recognizable style. The result: it&#8217;s pretty damn catchy.<\/p>\n\n\n\n<p>So if you like the album, what is it exactly that you like? It sounds to me like what you&#8217;re enjoying is not so much the ML algorithm&#8217;s copious output of melodies and lyrics, but YACHT&#8217;s taste in selecting the gems from within it. So far, so good. But there&#8217;s another piece of this puzzle that makes me question whether this analysis is going deep enough.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><a href=\"https:\/\/www.youtube.com\/watch?v=_yz8QYzcfxI\" target=\"_blank\" rel=\"noreferrer noopener\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"542\" src=\"https:\/\/otherthings.com\/blog\/wp-content\/uploads\/2020\/03\/Screen-Shot-2020-03-13-at-10.29.31-AM-1024x542.png\" alt=\"Screenshot of the video for SCATTERHEAD, showing the lyrics.\" class=\"wp-image-1544\" srcset=\"https:\/\/otherthings.com\/blog\/wp-content\/uploads\/2020\/03\/Screen-Shot-2020-03-13-at-10.29.31-AM-1024x542.png 1024w, https:\/\/otherthings.com\/blog\/wp-content\/uploads\/2020\/03\/Screen-Shot-2020-03-13-at-10.29.31-AM-300x159.png 300w, https:\/\/otherthings.com\/blog\/wp-content\/uploads\/2020\/03\/Screen-Shot-2020-03-13-at-10.29.31-AM-768x406.png 768w, https:\/\/otherthings.com\/blog\/wp-content\/uploads\/2020\/03\/Screen-Shot-2020-03-13-at-10.29.31-AM.png 1138w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption><em>Lyrics from <\/em>SCATTERHEAD:<em>  <\/em><br \/><em>Time flies and I feel \/ but I can&#8217;t hear \/ palm of your eye \/ is it empty, memory?<\/em><\/figcaption><\/figure>\n\n\n\n<p>The first time I watched the video for <em>SCATTERHEAD<\/em>, one lyric fragment jumped out at me: \u201cpalm of your eye\u201d. I&#8217;m not alone: <a href=\"https:\/\/livesessions.npr.org\/artists\/yacht\">NPR Music&#8217;s review<\/a> calls it out specifically as a &#8220;lovely phrase &#8230; which pins the lilting chorus into place&#8221;. But it jumped out at me for a rather different reason: I&#8217;d heard those exact words before.  I immediately recognized them from Joanna Newsom\u2019s 2004 song <em><a href=\"https:\/\/www.youtube.com\/watch?v=KcHjAUhtSrk\">Peach, Plum, Pear<\/a><\/em>. <\/p>\n\n\n\n<figure class=\"wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Joanna Newsom - Peach Plum Pear\" width=\"474\" height=\"356\" src=\"https:\/\/www.youtube.com\/embed\/KcHjAUhtSrk?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><figcaption><em>I have read the right books \/ to interpret your look \/ you\u00a0were\u00a0knocking\u00a0me\u00a0down\u00a0\/\u00a0with\u00a0the\u00a0palm\u00a0of\u00a0your\u00a0eye<\/em><\/figcaption><\/figure>\n\n\n\n<p>At the time, not knowing anything about YACHT&#8217;s process, I assumed they were making an overt, knowing reference to Newsom&#8217;s song. But then I learned how they generated their lyrics: they trained the ML model on the lyrics of their own back catalog <em>plus the entire discography of all of the artists that influenced them<\/em>.  This opens up another plausible explanation: it could be that Newsom was among those influencers, the model lifted her lyric whole cloth, and YACHT simply failed to recognize it. If that&#8217;s the case, it would mean the ML model performed a sort of money-laundering operation on authorship. YACHT gets plausible deniability. Everyone wins.<br \/><\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"alignright is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/i.kym-cdn.com\/photos\/images\/newsfeed\/001\/079\/173\/ed2.png\" alt=\"Web comic by Tumblr user Nedroid (Anthony Clark). \" width=\"170\" height=\"371\"\/><figcaption><em>I did not make this. <\/em><a href=\"https:\/\/knowyourmeme.com\/memes\/i-made-this\"><em>Anthony &#8220;Nedroid&#8221; Clark made it.<\/em><\/a><\/figcaption><\/figure><\/div>\n\n\n\n<p>This sounds like a scathing indictment of YACHT or of ML, but I honestly don&#8217;t mean it that way. It really isn\u2019t that different from what happens in the creative process normally. Humans are notoriously bad at remembering where their own ideas come from. It\u2019s all too common for two people to walk away from a shared conversation, each thinking he came up with a key idea.  For example: witness the recent <a rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\" href=\"http:\/\/theconversation.com\/new-ai-art-has-artists-collaborators-wondering-who-gets-the-credit-112661\" target=\"_blank\">kerfuffle<\/a> about the Ganbreeder images, created by one artist using software developed by another artist, unknowingly appropriated by a third artist who thought he had &#8220;discovered&#8221; it in latent space, and then exhibited and sold in a gallery. So, great, now we have yet one more way that ML can cloud questions of authorship in art.<\/p>\n\n\n\n<p>But maybe authorship isn\u2019t actually as important as we think it is. Growing up in our modern capitalist society, we\u2019ve been trained to value the idea of intellectual property. It&#8217;s baked into how working artists earn their living, and it permeates all kinds of conversations around art and technology. We assume that coming up with an original idea means you own that idea (dot dot dot, profit!) But capitalism is a pretty recent invention, and for most of human history this is not how culture worked. Good ideas take hold in a culture by being shared, repeated, modded and remixed. Maybe there&#8217;s a way forward from here, to a world where culture can be culture, and artists can survive and even thrive, without the need to cordon off and commodify every little thing they do. It&#8217;s a nice dream, at any rate.<\/p>\n\n\n\n<p>At some level this is just me, sticking a toe in the water, as I get ready to add ML to my own toolkit. (It&#8217;s taken me this long to get over my <a href=\"https:\/\/otherthings.com\/blog\/2015\/06\/inceptionism-and-learning-envy\/\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\"initial discomfort (opens in a new tab)\">initial discomfort<\/a> at the very thought of it&#8230;) When I do jump in, we&#8217;ll see how long I can keep my eyes open.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>It&#8217;s an interesting time to be an artist. As machine learning becomes part of the toolkit, in different ways for different people, new ideas are shaking loose, and I feel compelled to write about them as a way of wrapping my head around the whole thing. The most recent headquake hit me by way of &hellip; <a href=\"https:\/\/otherthings.com\/blog\/2020\/03\/machine-learning-and-creativity\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Machine Learning and Creativity<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[162,197,20],"tags":[67,198],"class_list":["post-1538","post","type-post","status-publish","format-standard","hentry","category-art","category-machine-learning","category-rants","tag-music","tag-yacht"],"_links":{"self":[{"href":"https:\/\/otherthings.com\/blog\/wp-json\/wp\/v2\/posts\/1538","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/otherthings.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/otherthings.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/otherthings.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/otherthings.com\/blog\/wp-json\/wp\/v2\/comments?post=1538"}],"version-history":[{"count":11,"href":"https:\/\/otherthings.com\/blog\/wp-json\/wp\/v2\/posts\/1538\/revisions"}],"predecessor-version":[{"id":1556,"href":"https:\/\/otherthings.com\/blog\/wp-json\/wp\/v2\/posts\/1538\/revisions\/1556"}],"wp:attachment":[{"href":"https:\/\/otherthings.com\/blog\/wp-json\/wp\/v2\/media?parent=1538"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/otherthings.com\/blog\/wp-json\/wp\/v2\/categories?post=1538"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/otherthings.com\/blog\/wp-json\/wp\/v2\/tags?post=1538"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}