Mike Kestemont & Folgert Karsdorp

The interest for artificial or machine-driven intelligence is currently going through a marked renaissance in a variety of fields. Apart from the impressive advances which have recently been booked in computer vision, language technology is also benefiting hugely from a family of models that is now colloquially known as 'deep learning'. One remarkable application is currently attracting much attention: through the application of so-called recurrent models, it has become possible to have computers generate, fully independently, text in a variety of genres and styles. While the semantic coherence of such texts should not be over-estimated, the grammatical correctness and stylistic qualities of such artificially written texts are often surprisingly convincing.

In this talk, we will report an experiment where we have applied this technology in the domain of hiphop lyrics, an established subgenre of present-day popular music that is known for its explicit content and typical scansion style. Last summer, we experimented with this technology at a major music festival in The Netherlands, allowing us to gather quality judgments from a large, anonymous crowd of festival goers. The methodology of this experiment was a Turing test-like setup, where users had to distinguish between synthetic hiphop and authentic rap lyrics. In this talk, we will discuss how easily participants could distinguish human-generated and computer-generated text and which (linguistic) factors seem to have influenced their decision making.