A side effect of the YouTube Music Key service is that the Play Music app for Android/iOS and the desktop site started to show relevant YouTube music videos for the songs you are playing. There's a "watch video" button placed on top of the album art and a "start video" menu option (it's called "watch video" in the iPhone app).
Google Play Music for Android shows the video inside the mobile app, so it doesn't launch the YouTube app. Google Play Music for iOS opens the YouTube app.
In the desktop site, the YouTube player is overlaid. Press Esc or click outside the player to hide it.
I've checked a long list of songs and the "start video" option is available for many popular songs that have music videos. You can also find YouTube videos when searching for artists and albums.