Methodology: Picked 11 random tracks from random youtube regional weekly charts (10-20mil views each), and 11 random AI tracks from Suno monthly trending, also in random regions. I don’t trust you being ignorant of charts to be good enough factor to shield you from their influence, hence the random regions part. I’ve tried to make it a little more fair by skipping tracks in both sets that contain elements I think would’ve been dead giveaways, like diegetic switches, heavy metal hooks, etc. Also, all tracks are intentionally downsampled to 120kbps to try and hide some of the artifacts inherent to AI music. After all, I don’t think I did that good of a job picking the tracks, tbh, certainly not in favour of AI, but let the result decide.
Not being able to understand the lyrics was a big impediment.
Not being familiar with/fond of the genres was a big impediment.
given those two limitations I think I may have inadvertently selected for intra-song diversity and that may favour AI generation since it’s a lot more work for musicians to change sounds in the middle of a song.
I’m very curious to see the results. Sorry I didn’t get to it sooner.
I’m not exactly sure how to fairly rate this, so let’s try this - for every place in the rating give +11 to -11 score, if it’s AI at the top half flip the sign, same for real tracks at the bottom half, for a possible scores of -132 to +132, and random shuffle expected to average at 0, and giving most value to those at the ends and less to those in-between. Which gives the result of -11+10+9-8+7-6+5+4+3-2-1-1+2-3-4+5+6+7-8+9-10+11 = +24, which equates to 156/264, or 59.1% AI-detection™ score
Some sidenotes: I’m surprised P ended up that high, the amount of autotune in it sounds like AI speech synthesizer with articulation cranked all the way 11 where it just makes random weird sounds. D in the middle is also unexpected, given that it’s one of the few English songs and the lyrics are about AI supremacy. I’ve considered removing it and B from the set though, just because they sound way too different from the rest, but at that point I already skipped tons of AI tracks and had to stop and settle on something. N was my favorite of human songs. Despite it’s relative simplicity, it’s probably the only one that I found pleasant to listen to.
I despair for society’s musical inclinations.
Can’t agree more. Don’t get me wrong, AI music sucks big time, but it sets the bar high enough that most human musicians can’t even pass, and those that do are unlikely to be in mainstream since success in the industry is mostly unrelated to actual talent.
Sorry I didn’t get to it sooner
No worries, thanks for participating! Would be interesting to hear your thoughts on the results
I’m surprised P ended up that high, the amount of autotune in it sounds like AI speech synthesizer with articulation cranked all the way 11 where it just makes random weird sounds
59.1% AI-detection™ score
A small note: that I wasn’t trying to decide whether something was AI, I was trying to evaluate its creativity. So I actually thought A was real, but that it was extremely generic… I also suspected D was AI but trying to judge on creativity, I was unimpressed with the lyrics (not that creative) but I found the soundscape it created interesting. That’s a place where my (un)familiarity with the genre was a liability. I also found the pitch of “you” in “who are you” nrar the end to be creative, like in the wrong key but in a good way.
Also I only listened to each track once, then decided where to put it in the list. Like “top”, “bottom”, “near bottom but not worst”. And I placed them as I went. If I went through the list again I’d probably swap some but that would have taken way longer and I decided it was more important to be done. I probably should have done some spot checks at the end to make sure things ended up where I wanted.
I’ll do you one better. I’m actually curious what’d the result be so I’ve thrown together a little survey.
https://drath.ru/ai/
Methodology: Picked 11 random tracks from random youtube regional weekly charts (10-20mil views each), and 11 random AI tracks from Suno monthly trending, also in random regions. I don’t trust you being ignorant of charts to be good enough factor to shield you from their influence, hence the random regions part. I’ve tried to make it a little more fair by skipping tracks in both sets that contain elements I think would’ve been dead giveaways, like diegetic switches, heavy metal hooks, etc. Also, all tracks are intentionally downsampled to 120kbps to try and hide some of the artifacts inherent to AI music. After all, I don’t think I did that good of a job picking the tracks, tbh, certainly not in favour of AI, but let the result decide.
IVQJHOUPLGCFDNERTSMBKA
Great work. A few thoughts re: methodology.
I’m very curious to see the results. Sorry I didn’t get to it sooner.
Nice. Let me dig up which was what.
I is AI
V is Real
Q is Real
J is AI
H is Real
O is AI
U is Real
P is Real
L is Real
G is AI
C is AI
F is Real
D is AI
N is Real
E is Real
R is AI
T is AI
S is AI
M is Real
B is AI
K is Real
A is AI
I’m not exactly sure how to fairly rate this, so let’s try this - for every place in the rating give +11 to -11 score, if it’s AI at the top half flip the sign, same for real tracks at the bottom half, for a possible scores of -132 to +132, and random shuffle expected to average at 0, and giving most value to those at the ends and less to those in-between. Which gives the result of -11+10+9-8+7-6+5+4+3-2-1-1+2-3-4+5+6+7-8+9-10+11 = +24, which equates to 156/264, or 59.1% AI-detection™ score
Some sidenotes: I’m surprised P ended up that high, the amount of autotune in it sounds like AI speech synthesizer with articulation cranked all the way 11 where it just makes random weird sounds. D in the middle is also unexpected, given that it’s one of the few English songs and the lyrics are about AI supremacy. I’ve considered removing it and B from the set though, just because they sound way too different from the rest, but at that point I already skipped tons of AI tracks and had to stop and settle on something. N was my favorite of human songs. Despite it’s relative simplicity, it’s probably the only one that I found pleasant to listen to.
Can’t agree more. Don’t get me wrong, AI music sucks big time, but it sets the bar high enough that most human musicians can’t even pass, and those that do are unlikely to be in mainstream since success in the industry is mostly unrelated to actual talent.
No worries, thanks for participating! Would be interesting to hear your thoughts on the results
A small note: that I wasn’t trying to decide whether something was AI, I was trying to evaluate its creativity. So I actually thought A was real, but that it was extremely generic… I also suspected D was AI but trying to judge on creativity, I was unimpressed with the lyrics (not that creative) but I found the soundscape it created interesting. That’s a place where my (un)familiarity with the genre was a liability. I also found the pitch of “you” in “who are you” nrar the end to be creative, like in the wrong key but in a good way.
Also I only listened to each track once, then decided where to put it in the list. Like “top”, “bottom”, “near bottom but not worst”. And I placed them as I went. If I went through the list again I’d probably swap some but that would have taken way longer and I decided it was more important to be done. I probably should have done some spot checks at the end to make sure things ended up where I wanted.
Fantastic, I’ll try to listen today