The release of Craft 5 has seen one of the largest shifts in content architecture patterns that the CMS has ever had. The so-called "entrification" of Matrix fields is a huge change, both in terms of how you should think about content modelling in Craft, and how that content is stored and referenced at an architectural level. It unlocks a bunch of very exciting functionality, and for people starting out fresh it will streamline quite a lot of content decisions in beneficial ways. But if you already have a mature Craft installation, with hundreds or thousands of content entries, it poses a little bit of a problem. Doubly so if (like many Craft admins) you've already been patching together more "advanced" Matrix functionality using popular plugins like Neo or Supertable.

Thankfully, Pixel & Tonic (the makers of Craft CMS) were well aware of the friction that such a big change could make for those looking to upgrade, and I've been very impressed (once again!) by how well the default upgrade experience handled most of the complexity. Understandably, however, it does take the path of least resistance. That means Matrix fields are converted to brand new Entry types, with verbose names to ensure they won't overlap with existing models. No complaints there, as Craft has never been fussy about rewriting data models, so you can hop in after the upgrade and change these to more human-readable values very easily.

However, the same steps are taken for the child fields within the Matrix field as well. Under the old model, Matrix fields used pseudo-fields, which behaved exactly like regular Entry fields but were actually one-off UI configurations that couldn't be shared or reused. This was a real pain point of working with Matrix, as it frequently meant reinventing fields that you were already using elsewhere, and then having to remember to update each variant if your content model changed. Now that all Matrix blocks are simply Entries, like any Entry model they use regular ol' full-fat fields, which means those one-off UI configurations have to be converted as well. This is a moment in the upgrade process that I wish P&T had given us a little more control.

Don't get me wrong, a fully automated path was always going to be a preferable option for some people, but I'd have loved to have been able to say "hey, for this content type, let me map Matrix Field A to existing Entry Field Z", and have the content migration automatically handled within the upgrade itself. Unfortunately, that isn't an option^[1], and instead Matrix fields are cloned to brand new Entry fields with often hilariously long names. If you were using nested Matrix entries, then these get even more ridiculous^[2]; one of my examples wound up as "Critique - Critique - Rewatch - Datetime", for instance 😂

If you're like me and were using nested Matrix fields in a variety of places and ways, the result is a bit of a mess. Whilst the upgrade ran very smoothly, both Entry types and fields proliferated considerably, the latter particularly so. I jumped from having around 80 fields (an amount already calling out for some maintenance, containing more than a couple of relict, deprecated fields from old data models) to closer to 170, many of which were clearly duplicates.

That duplication effect was felt all the more keenly as a result of Craft 5 also bundling the new ability to infinitely reuse fields within a content model^[3], which is a joyous addition, but once again changes best practices around field design completely. If I were starting a Craft project today, I'd look at that functionality and realise that I should be creating reusable fields as a priority. Rather than having a dozen, specifically named Plain Text fields, for instance, I should have two: one for short-form, inline text content (titles, labels, names, etc.) and one for long-form content (addresses, quotes and other text blocks, descriptions etc.). So considering something like a "Person" content model – which might have a name field, address block, and a few fields for contact information – where I would have previously created half a dozen bespoke fields, I could now get away with that singular, short-form Plain Text field repeated half a dozen times.

Taken in combination, between the field proliferation of Matrix entrification and the fact that many more pre-existing fields can now be marked as redundant, I realised that I needed to spend some time streamlining and modifying my existing content models. I won't go into all of the details here, but will say that the process remains ongoing, and that I've been very happy with some of the changes and efficiencies it has allowed.

Unfortunately, it has also highlighted how bad Craft is at these kinds of internal data migrations. It has a wonderful suite of tools for external migrations and importing data, especially if you include the first-party Feed Me plugin which makes extracting content from a variety of file types and API structures incredibly easy, but for internal architectural changes there isn't much guidance or supporting functionality. However, there are definitely some methods that you can use, and having spent the last couple of weeks deep in these particular weeds, I figured I should write up the ones that I've found useful.

Note: No matter which tactic you try, always test it on a local Craft install before trying it on your live database; always create a database backup before making any content model modifications; and always test the data changes rigorously to ensure content has been transferred correctly! I'd also recommend doing each field migration in two steps: first clone the data into the new field, test that the migration has worked, and only then delete the old field from the Entry type.

Warning: I've run into a few issues with conditional logic, some of which resulted in complete data loss or forced rollbacks of certain field migrations. If you're migrating a field that is used as a trigger for other fields to be shown or hidden, remove that logic before moving any data around, deleting the field, or renaming the field. I'd personally recommend stripping out all conditional logic from an Entry type before doing anything else, and only reinstating it once you're certain that all field migrations are complete, just to be safe.

Feed Me

Let's start with the most obvious (and most recommended) tool for this kind of job: Feed Me. Yes, this is a plugin, but it's made and maintained by Pixel & Tonic, so you should think of it as an optional bolt-on piece of core functionality, rather than a typical third-party extension. If you take a quick look around the official Discord, Stack Exchange, or GitHub discussions, you'll also see that is far and away the most commonly suggested solution for data manipulation within Craft, though doing so basically relies on twisting some of its features to work in ways they don't really appear to be intended for.

At the core, Feed Me is designed for importing data. It can ingest just about any kind of web feed; connect directly to XML and JSON APIs; or even pick apart CSV files. But you can't simply point it at an existing Entry type or Section and ask it to move some data around. I really, really wish you could, as I think that would solve 99% of my issues with Craft's lack of innate functionality in this area, but sadly it isn't a (currently) supported option.

Compounding that issue, as of writing (May 2024), Feed Me doesn't support nested Matrix fields at all. So even if you can get your existing Entry data into a valid ingestion format, if you want to modify data more than one-level deep, you simply can't. I imagine this will be rectified in the future, but right now it makes Feed Me a particularly bad fit for post-Craft 5 migration work, as the areas that are probably the most in need of tweaking and touch-ups are Matrix fields.

However, if you aren't using nested Matrix entries, Feed Me is still a solid option. You can either export your existing data (any Entry view in Craft 5 has an Export button at the bottom right, and Feed Me can handle any of the available file types) or create an API endpoint using the Element API plugin – another first-party piece of bolt-on functionality. The docs for Feed Me are solid, and the process is pretty straightforward, albeit a little involved.

Migration Scripts

Craft does have native, core support for migrating data around the system, it's just not built into the actual UI. Instead, it takes the shape of migration scripts, which must be called from a CLI, so you will need SSH access (at a minimum) to trigger them on your live server. I have to admit, as a front end developer with only rudimentary PHP knowledge, I'm not very comfortable with migration scripts, nor have I really dared use them.

That said, the process does seem manageable, and is well outlined in the official documentation. There's a handy CLI command for scaffolding a new migration, craft migrate/create, and another for running any current migrations stored in the relevant folder: craft migrate/up. Again, the docs are the best place to learn more about how this works and what you can do, but if you're comfortable working with PHP and have a good understanding of Craft's own data structures and internal helper functions, writing a script to move content between fields should be a relatively quick task.

If you're a little unsure, I can also recommend Adding Content with Content Migrations from NYStudio107; whilst this is more about content ingestion, it gives you all the building blocks you would need to understand how you might modify a migration script to allow for content manipulation – or you could always use this in a similar way to that discussed above for Feed Me, only with more direct control and (I believe) nested Matrix support.

Bulk Editing

Finally, there is the option that I've found myself leaning on the most. It's horribly hacky, still fairly time consuming, and highlights a few little bugs in the system that will hopefully be squashed in the future, but it allowed me to wrangle my content into a much cleaner state and remove over a hundred fields overall, so I reckon it's kinda great. Let's talk about bulk editing.

This is another under-hyped new feature of Craft 5 – which means everything I'm about to say is specific to v5 and beyond^[4] – but gives us a whole bunch of new content-editing superpowers! Much like the Export button, any Entry view should have a new Edit button in the table footer. Press that, and the currently visible fields will become editable. Well, almost.

Some third-party field types don't support bulk editing (though hopefully support is in the works). The biggest issue right now is CKEditor, which I imagine is unlikely to ever get true support. So if you're needing to migrate data to or from a CKEditor field then I'm afraid this technique simply won't work, and you're probably best off with a migration script or a fully manual conversion process (😬). (Though see below on Redactor ➡ CKEditor.)

A few native fields are also a bit glitchy. Category fields, for instance, appear to be editable, but in my experience never actually save any edits made (especially deletions), so I would categorise them as unsupported too (pun mildly intended 😏). I haven't tested Tag fields, but I wouldn't be surprised if they had the same issues. Strangely, the new Alternative Text field for Assets is also unsupported, though I figured this might be an oversight, raised a GitHub issue, and it looks like it will be fixed in the next release.

Those exceptions aside, the bulk edit feature allows for some very quick content changes. It makes the dreaded manual conversion of fields much faster and easier (I've done this for a few Lightswitch fields, for instance), but also – crucially – it makes that same process automatable. There are a few ways you could go about doing this – browser-based Macro plugins like TamperMonkey spring to mind – but the "simplest" (so long as you have a decent understanding of both JavaScript and HTML) is to use the browser's own developer console to run JavaScript functions that manipulate the DOM. So, for basic field-to-field cloning, here are the steps I've been taking:

1. Add the new field to the Entry type;
2. Rename your old field (and handle) to something obvious – I tend to just stick "Old" in front of both;
   1. If using a new reuseable field type, you'll also want to update the name and handle there as well^[5].
3. Save the Entry type, and go back to the Entries view;
4. I would then recommend creating a new custom source, targeting only those entries that you need to migrate. For me, that typically means selecting the specific Entry type, then filtering further by checking if the old field has a value;
   1. It is also beneficial to filter on whether the new field has a value, though whether you do that within the custom source or the view filters themselves is up to you;
   2. You can also select the relevent old/new fields as the Default Table Columns, and remove any others;
   3. It is easiest if the old field is shown in the first editable column, with the new field directly after it, as this makes DOM traversal much simpler.
5. On your new Entries view, hit the Edit button to enter bulk editing mode, and then open up the browser's developer tools;
   1. These are typically present in both the main browser menu (often top-right) or by right-clicking anywhere and selecting some variation of "Inspect Element", depending on the browser in use.
6. Make sure the Console tab is selected in the developer tools window, and then paste in or create the necessary JavaScript function, before executing it;
7. You should see the data cloned from one field to the other. Give it a quick check over (watch out for things like special characters and white space, though I've not had any issues myself) and then press the red Save button in the footer.
   1. Warning: if you have more than one page of entries to edit, don't try to move to the next page whilst still in bulk edit mode, as this reverts all of the changes you've made. Save first, then re-enter the editing mode and pick the page you need;
8. Repeat these steps until all of your entries have been edited, then check that the data has been copied across thoroughly;
9. Once you're happy that the migration has been successful, go back to your Entry type and remove the old field, making any final adjustments to the new field's name, layout, or conditional logic as needed.

Now, I realise that I've slightly hand-waved my way through step #6 in that list; it may as well read "now draw the rest of the owl" 🦉 If you're not familiar with editing web pages in this way, this is probably a bit of a weird mental model to run into, but the tl;dr involved is that a browser's developer tools allow you to make local only edits to any web page. You can delete parts of the page, add your own sections, edit copy or content (this is how people make those viral – but fake – screenshots of celebrity tweets or ridiculous company product announcements 😉), or manipulate interactive elements, which is exactly what we're doing. For the most part, this kind of local editing is a parlour trick, but with form elements there's no fundamental difference between a script entering a value and a human doing so, which means we can automate the copying of content from one field to another. In its simplest form, for two plain text fields, the function you'd want would look something like this:

# Clone from text field to text field
[].forEach.call(
  document.querySelectorAll("td[data-title='Old Field Name'] input[type='text']"), 
  function(e) { 
    const current = e.value;
    const sibling = e.parentElement.nextSibling.querySelector("input[type='text']");
    sibling.value = current;
  }
);

Again, if you're not familiar with JavaScript, the browser DOM (the structure of the web page), or DOM traversal, that might not make much sense. Here are the steps, broken down (note: this is not in the same order as the code, but instead in the order of how you'd think this through):

1. Find all instances of an <input> element with a type attribute of "text", that are children of a <td> table cell element with a data-title attribute of "Old Field Name" (you'd want to change this to your field name), using the querySelectorAll function to output that as an array;
2. For each matched element, first store its current value (whatever is contained within the <input> i.e. the field's content) as a variable called current;
3. Then use DOM traversal to find the next <input> element;
   1. Specifically, we're finding the element that contains our initial <input>, then moving along to the next sibling, before searching within that sibling for the first text <input>. You may need to modify this e.g. if moving from a number field, the type attribute would be "number", or if migrating a longform content section you'd need to target <textarea> rather than <input>.
4. Finally, update that second <input> element's value with the saved variable, current;
5. Repeat until all elements have been processed.

This is a very minimal example, but you don't have to stop with just cloning data, or taking one piece of data at a time. For instance, with Date fields you might have both date and time inputs available, so if you're moving one Date field to another, you'd want to grab both bits of data. That's a bit more involved, so you'd need something like this:

# Clone date field to date field
[].forEach.call(
  document.querySelectorAll("td[data-title='Old Date Field'] .datetimewrapper"), 
  function(e) { 
    const date = e.querySelector(".datewrapper input[type='text']").value;
    const time = e.querySelector(".timewrapper input[type='text']").value;
    const sibling = e.parentElement.nextSibling.querySelector(".datetimewrapper");
    const sibDate = sibling.querySelector(".datewrapper input[type='text']");
    const sibTime = sibling.querySelector(".timewrapper input[type='text']");
    sibDate.value = date;
    sibTime.value = time;
  }
);

Or what if you're wanting to manipulate the content between two different field types. I had an old text field that I wanted to standardise to a Date field, so wrote a small function that converted the text string (e.g. "May 2020") into a valid date format, filling in the missing data with something that I would know was a dummy/default value (e.g. setting time to "12:34"):

# Convert text to date field
[].forEach.call(
  document.querySelectorAll("td[data-title='Old Date Field'] input[type='text'].nicetext"), 
  function(e) { 
    const current = e.value;
    const dateElem = e.parentElement.nextSibling.querySelector(".datewrapper input");
    const timeElem = e.parentElement.nextSibling.querySelector(".timewrapper input");
    const monthNames = [
        'January', 'February', 'March', 'April', 'May', 'June',
        'July', 'August', 'September', 'October', 'November', 'December'
    ];
    const [monthName, year] = [...current.split(' ')];
    const month = monthNames.findIndex(name => name === monthName) + 1;
    const date = `01/${month}/${year}`;
    dateElem.value = date;
    timeElem.value = "12:34";
  }
);

Each function took a little trial and error to get right, and some field types are irritatingly hard to properly interact with via code – looking at you, Lightswitch! – but you can test out things like DOM selection and traversal in the Console window with document.querySelector and, if something does go wrong, nothing is actually updated or overwritten until you hit the Save button, so you can experiment as much as you need.

One slight quirk that is worth noting is that the entries you see in your custom view will not stay the same when you click the Edit button (and if they do stay the same, the order will likely change). I'm not sure why this is, but bulk edit seems to apply its own set of order-by filters, rather than respecting the current entry list. This is why I recommend filtering out entries with a valid value in the new field, as this will stop you having to repeat a bunch of entries as it shuffles its way through. If you can't filter it out (some fields don't allow for filtering 🤷‍♀️), you could make the overwrite behaviour conditional on the <input> element (or whatever you're targeting) being empty, but in practice I found overwriting was always more convenient.

On the whole, though, I've found this method of partially-manual migration to be very effective and surprisingly fast. With 100 entries per page in a regular Entry view, you can get through thousands in minutes. A little tip there is that the Console window will copy your previous command, ready to be re-executed again, if you press the ⬆ arrow key (as is common for CLI windows), so you can get into a pretty solid rhythm of: Press Edit ➡ Up Arrow ➡ Enter ➡ Scroll to Check ➡ Press Save ➡ Repeat.

One Final Trick

What if you don't want to migrate between similar fields, but would rather convert a specific field from one field type to another? For instance, several third-party field types look unlikely to make the jump to Craft 5, and whilst some (such as Neo and Supertable) have very clear paths for conversion, others aren't as simple. The short answer is that any of the methods discussed above will work in exactly the same way – even bulk editing, with the field type caveats discussed – but there is another, sneakier option available if switching the field type is your primary goal.

I won't lie, what I'm about to suggest is a bit hacky and it only works in very specific circumstances, but when it does work, it's much quicker and easier than anything else I've mentioned. Put simply, Craft has some very robust guardrails in place which compensate wonderfully when you do something that you should probably never do: changing the field type directly on the Field model itself.

If you go to Settings ➡ Fields and then select your Field, you'll see that the Field Type dropdown option is editable. There's a large, orange warning message saying that you might suffer data loss if you change the value – which is very true; test locally! – but you can select any other field type, hit Save, and try your luck. As I say, Craft is pretty robust with this kind of thing, and generally the outcome will be what you would guess.

For instance, if you're migrating from Redactor to CKEditor (like many of us are at the moment), so long as you select similar configuration files, I've found this technique works really well^[6]. Similarly, if you're just jumping between Plain Text and CKEditor, that won't be an issue, though the inverse will obviously strip any formatting you had previously added, so won't be lossless in the same way. Craft does try to highlight which fields are less likely to cause conflicts by flagging some with little ⚠ emojis, but I'd take those recommendations with a strong pinch of salt. For instance, CKEditor ➡ Colour implies that it would work (no warning), but that's clearly nonsense 😂 Still, as a speedy option, this is worth testing, at the very least.

A Hopeful Future

Right now, the state of data migrations and content model changes is a bit messy in Craft. If you compare these kinds of workarounds, hacks, and highly technical options to solutions found in other CMSes, you'll see that it could be much better. Take something like Sanity, which still largely relies on migration files, but uses a much simpler, GraphQL-like syntax, alongside incredible documentation, which makes field-to-field migrations (even with data manipulation) an absolute breeze. Other services even have GUIs that allow you to select two fields and clone (and even transform) the data between them. My hope is that Craft will get similar functionality in the future. In fact, as I've said above, I think Entry-to-Entry functionality is an obvious feature that could be added to Feed Me, so long as the new limitations around Matrix entries can be fixed first.

(If you agree, I've created a GitHub Discussion for the topic on the Feed Me repo – maybe drop it an upvote, comment, or reaction 😉)

In the meantime, bulk editing has allowed me to reduce my total number of fields down to 62! There are still two in that list I would like to remove entirely, but they are CKEditor fields and therefore my options are either migration scripts or manual, neither of which feel worth it. I also have a few duplicate Category fields, but I don't imagine I'll be able to do anything about that until we can either use one Category field multiple times on a given Entry type, or Categories have been entrified into oblivion, so I have to live with them for now. Still, my field list is much improved, and switching to reuseable fields has the added bonus of making searching and finding a field much easier now that most follow a common naming convention – I've gone with examples like "Rich Text: Article" and "Tag: Artists".

Plus, I've finally been able to tidy up some old legacy fields – like the Plain Text date field I've been using on Reviews – and standardise some others across Entry types, all of which make my content and my codebases a little easier to maintain in the long term 🎉