As part of the whole cataloguing my music collection and making it publicly available thing, I've had to come up with a workflow for taking a physical album and converting it into a digital equivalent. That means recording details like album name, artist, cover art, media type (vinyl record, CD, digital etc.), and a tracklist (which includes song title, track number, duration, and which side of a record it's on).

Now, the simplest option would have been to just use some of the existing public APIs out there, of which there are a few. Most popular music services do expose their databases in some way, so you can request data from places like Spotify, Deezer, Last.fm, and many more. Heck, even iTunes has a public API^[1], which I was pleasantly surprised to learn about (look at Apple, being all open and sharing for once). Then there are the independent and/or open-source alternatives from the likes of Discogs, MusicBrainz, and theAudioDB which are crowd-sourced and pretty resilient.

Despite the plethora of options, it still felt a bit limited. There are some aspects to using an existing API that would definitely be beneficial – such as not having to store/host all the cover art – but they all come with caveats. A big one is the specificity of information. Ultimately, this data is being used for my personal collection, so it should be as accurate to the actual physical items as possible. If artists only ever released a single version of an album, on a single medium, then this would be easily done with an API query, but (luckily, IMO) they don't. A single album may have over a dozen releases, all with subtle variations in tracklists, cover art, etc. I already use Discogs and Last.fm extensively, so I can say with experience that even on these incredibly robust services finding a specific album version is not always easy. In fact, that experience means I know it'll be simpler in the long run to just capture the necessary data manually.

And that comes with some extra benefits. First of all, I can mould the data into the shape I need. Want to divide a tracklist into the sides of a record the songs actually appear on? Easy. Need to include a hex code for the vinyl colouration along with the album name? Done. Prefer to display albums in the order purchased? I can add a field for that. Ultimately – like most things with the web – having total control gives me much more flexibility and creative freedom.

Plus, no one can take this data away from me. With a third-party API, I'll be forever tethered to the whims of a company or group of people I have no control over. If Last.fm suddenly go out of business, my site's content would disappear with it. Controlling my own data may be more low-level upkeep in terms of data entry, but in terms of long-term maintenance, it should be much more efficient – yet another win for IndieWeb principles 👍

The Process

Okay, so that's the why of the matter, how about the how. From an infrastructure perspective, it's pretty simple: I use Craft CMS already, so I created a new content type and that becomes my "database". One of the reasons I love Craft so much is how easy it is to generate unusual data templates. Sure, it's a blogging engine at heart, but if I want to create a Music template with a table input for tracks; checkbox input for media type; colour picker for records; and an artist field that automatically records links between albums with the same artists, well, that's all super simple. Shove a new API endpoint on top using the native Element API plugin, and I've got a custom, cross-linked database that's fully accessible to any app I want in about an hour of setup (most of which is API design).

That takes care of data storage, and Craft's inbuilt Assets manager handles the cover art uploads and file structure, so all that's left is data capture. Here's where I learned a few little tricks.

For the most part, data capture is just manual. Artist and album title are quick text strings, so I just type those in^[2]. Medium is a few toggle options, and the record colour is a best-guess type of deal. The sticking points are the tracklist and the album art.

Tracklists

Tracklists are one of those elements of data capture which is just tedious, and I did strongly consider using a third-party API to pull it in. Luckily, Last.fm provides tracklists for most albums on their site and because HTML is great, that means you can copy and paste them. Of course, a tracklist on Last.fm is full of extra columns I don't want or need, such as global scrobble count. I also found literally copying and pasting brought a bunch of unnecessary formatting and layout with it, which I then had to strip out.

Enter the Firefox plugin Download Table as CSV (also available on Chrome). Thankfully, the team at Last.fm have done a great job on their website, which means tracklist tables are actually <table> elements, as they should be. So with the plugin installed, all I have to do is right-click a tracklist and press "save as CSV"; this downloads the data into a CSV file, which I open, delete any extraneous columns, and then copy and paste directly into a table input within Craft. It takes less than 30 seconds and, for most albums, that's the end of the process. Some have extra tracks or slight regional differences, but those edits are quick and pretty painless once the bulk of the data capture is in place.

🚨 UPDATE: Incredibly, the latest version of Firefox appears to have broken this plugin; I've switched to using Table Data Export (TDE), but would still recommend the former if it ever gets fixed, as the UX is much better. If you do end up using TDE, the key class for Last.fm is chartlist; run that on any album page and it'll extract the track info perfectly 👌

Cover Art

That only leaves the album art. I'd love to say I have a super-streamlined method for finding decent quality cover art images, but honestly, it's a hit and miss process. Given that I'm normally already on Last.fm, that tends to be the first option. You can click to view related images from any album page and in a lot of cases that gives me a good enough version, but not always (again, some albums have different art for different releases, and others just are too low resolution to use).

💡 Pro Tip: In modern browsers Last.fm will serve WebP files (you love to see it 👏). That's great, but WebP is still not fully supported, so this may not suit every use-case. However, you can just change the file extension in the URL (e.g. to ".jpeg") and 9/10 times it'll load an identical version but in the format you need 👍

The second best option is the crowdsourced Album Art Exchange. Unsurprisingly, a community built around album art is a pretty good bet for finding, well, album art, and if anything the quality tends to be too good. Better yet, due to the fringe nature of these types of communities, a lot of rarer and more niche artists, genres, and albums are well catered for – though I've still had a few with zero results. The downside is that their search functionality is not exactly brilliant, and it can take quite a while to trawl through results even on a good connection, thanks to the heavy load of dozens of high-res thumbnails. Oh, and you do need to register to remove watermarks.

Finally, we have the hail Mary passes! First up is Amazon, of all places. You have to get a bit fancy with dev tools to get past the annoying magnification overlay, but in general preview images are of decent quality. However, rarer album variations, albums that are no longer for active sale (plus any deliberately released through indie channels), or more obscure artists are often missing. If that's the case, Spotify is basically your last option. There's no easy way to get cover art out of the Spotify app, but if you log in via their web player you can pull the image URL out of the page source, load it up in a new tab, and save it that way. Hardly ideal, but still a viable option.

So far, I've had to use variations of all of the above to get decent enough images (and there are still some I would prefer to find in better resolution), but at least once I have one I can just upload it to Craft and forget about it. Plus, then I get to write the alt text description for it, which I've found surprisingly fun and challenging in equal measure. How the hell do you accurately describe something like The Offspring's cover art for Let the Bad Times Roll in a succinct manner 😂