The Joomla 3 to 5 update saga

Image showing the convoluted process of updating Joomla

Joomla, is a popular CMS (Content Management System) for building and maintaining web sites. CMSs do make the "day to day" management of articles, tempates (the layout and look of the site) and giving specific levels of control to the right people contributing to the site very easy.

That said... when it comes time to upgrade the CMS itself... minor updates are usually fine, but major upheavals can cause massive headaches.

Let me regale you of my tale of the seemingly trivial Joomla 3 update to 4 and then 5... which took several attempts, and about a week of concerted effort and sacrifice.

Officially, this is an update... not a "migration"... but since this involved multiple migrations in my particular "update", I have to say that I'm not going to fault anyone for using the most-apt term here. :~) I'm not saying this to be cheeky, this is something any existing or would be administrator would need to know before updating a Joomla 3 site... and there's a lot out there.

Big changes, multiple system upgrades, and transitive versions are a lot to consider.

Joomla 3 has been outdated for quite some time now. In fact, support for it ceased over a year ago. However, at the time of writing, roughly half of all Joomla sites are still running it. Despite the lack of modern compatibility, the security bugs, and of course, slower performance.

"Why?" might you ask?

Because the hard-working volunteers making the free CMS let version 3 reign supreme for a little too long (think years).. then in a desperate move to keep up with all the web-related advancements.... they made massive changes to the fundamental design of version 4. But even that proved to be troublesome....

Suddenly, all the add-ons, extensions, plugins, templates, and modules that we'd long relied upon.... and supplied by third-parties... were utterly incompatible.. and we'd paid good money for some of our "extras".

Worse yet, in many cases.... there simply weren't any viable replacements for these add-ons... now imagine you have 100 or even 10,000 articles/pages of information... embedded into an add-on (not Joomla exactly) that had no replacement or way to extract the data and put it into Joomla's system itself.

So users wait.. in the desperate hope that the each maker of whatever add-ons are installed (and there are thousands to choose from), brings out an update so they can keep their precious info available for a few more years.

Naturally, big (and sudden) changes are not well received by the public at large. Also, in the rush to get Joomla version 4 running, they simply didn't have time to fully flesh out all the details. So 4 became a "transition" version to get to the "real" edition labelled under version 5. Which of course, came out less than a year later... had even higher requirements that deviated from 3 enough to make them mutually exclusive. Joomla 4 was "exclusive enough".

Ultimately, Joomla web administrators had to make a choice....

You can make a system work well for version 3, or you can forsake all version 3 compatibility and use newer PHP versions, newer database systems, and extensions (if they exist). If you're thinking about building a new Joomla-based site... I'd recommend that you go straight to Joomla 5 to avoid all this work.

Is this only an issue for Joomla? Can't I just use something else like Wordpress?

Now I want to be clear here... the perfect web site, web site builder, or CMS... doesn't exist. Everything has it's own "pros and cons". Wordpress is just another CMS like Joomla (albeit a more popular one) and it can just as easily have issues with the need to change over time as Joomla just had.

I'll just use Wix or Squarespace then...

The "proprietary" web site builders like Wix, Squarespace, or photography-specific ones like SmugMug/ShootProof are often "all in one" solutions.. meaning that you can't separate from them once your site has been built. You stop paying, and your site basically dies. You can't migrate the site their system built onto another server either. Your site doesn't stand on it's own. You have to host the site on one of their services... so you can't move it to a different computer. But that's not the worst of it...

Want something different that's not on offer?

Depending on what you want, and the service you use... you might not be able to and in the worst-case-scenario, you have to start from scratch again. Renee and I both tried a variety of these and yes, they make some things beginner friendly.. but there's always something that doesn't work, can't change (like the template once you've built it) and often have limited (or no) capability to do, or add functions beyond the "set menu" later.

With the proprietary site building systems... you do things their way, with all of their limitations... or not at all... oh and did I mention they charge you an additional subscription fee for the builder service too? That's not always a bad thing, since in fairness, I bought my Joomla template, the page editor, and my gallery plugins... and you don't have to do that with the proprietary stuff.

However, also in fairness, my payments (other than hosting/domain name stuff) are one-offs. I may not get updates after a while, but my site keeps going with the last available version of all my plugins until I either choose to upgrade or until the web itself doesn't support it anymore (think years).

Everything in the proprietary stuff is an ongoing monthly payment... but this ensures support because they have to keep everything easy to manage at all times... for all their customers... so if this article scared you off... then maybe the likes of Wix and Squarespace are for you, and the extra expense is worth it. (Don't worry, I don't judge you harshly for taking a path that suits you... if it does indeed, suit you).

So why choose a free CMS like Joomla, Wordpress, Drupal, etc in 2024 and beyond?

A lot of people think CMS's are largely dead because they seem like they were more popular between the early 2000s and mid 2010s. That's not true. Sure, there's more competition now, from Squarespace, Wix and Shopify (also technically, subscription CMSs) but the situation at the end of 2024 is as follows:

  • Wordpress leads the way with it's inherent adaptability and insane catalogue of plugins, estimates saying that it runs 40% of sites on the Internet. Everyday users can build simple sites, but there's enough capability there build some fairly complex stuff with add-ons if needed. It's basically the "Jack of all" option here.
  • Wix and Squarespace offer drag-and-drop simplicity for people who want pretty-but-simple sites with litte or no site building experience. That has a lot of appeal to many.
  • Shopify is for business/e-commerce and is amazing for that stuff.. but it's a "Subscription CMS" that's a total tax write-off in many countries. (So all good stuff then) but it's poorly suited to a typical non-shop site.
  • Drupal CMS offers advanced features and customizability for coders and businesses to create really complex sites, but will be overkill for the average user. So power-users or someome willing to pay a pro coder to "pave the way" would likely choose this one.
  • Joomla is somewhere closer to the Wordpress "mid ground" with a great "flexibility without too much complexity ratio".... despite my whining about this once-in-20 years "update debacle".

Free CMSs are great. They only require a hosting platform (server or service), with the domain name as a subscription. Being free, they don't charge money for the software itself. That said, the templates and plugins you add to them might be a different story but they can make your site look every bit as good as Wix/Squarespace, offer far greater versatility, and save you time in the setup. All, so you can get to creating the content for your site instead.

However, while non-commercial systems are flexible (far more than Wix/Squarespace) they do not come with "CMS support" outside of the forums community. You have to learn to do this stuff yourself. Wix and Squarespace-like offerings probably have a team of folks ready to basically hold your hand through the entire process... you're paying their salaries afterall.

However, for personal sites, non-profits, and small businesses, where additional "subscriptions" being paid each and every month without fail might be an avoidable expense you can do without. It's for this reason, interest groups (everything from photography, book clubs, social organizations for old people, even IT forums often run CMS systems.. and for me Joomla (for it's flaws) still offers more benefits than drawbacks for me, and many of my clients. Although I have helped people with other CMSs and Wix/Squarespace too.

But back to the Joomla migration at hand....

A picture selecting the backup function in cPanel, a common interface used by many hosting services.

Now is one of those times you need to backup absolutely everything.

Whether you're using a hosting service (which many use the cPanel interface as shown above with a one-click full backup button so you have no excuses) or a local server... which can be a little more involved, the process is ultimately the same:

Naturally, the first thing you should do is a complete backup of your site. That means:

  • All server configuration files (php, mysql, apache.. and any certificates, license codes, VPN settings, scripts, and other bits such as the notes of the versions of each software that you know works with everything else).
  • Your whole site's public directory (where the files, images, and other data are made accessible to be served up to the public).
  • Database(s) and the username/passwords for them.
  • Any (possibly now unavailable) install files for your plug-ins.. just so you can resurrect your old site to how it was... if you absolutely need to... or migrate it to a test/upgrade server. Through trial and error, you'll know which combinations of plugins (and their specific versions) are sure to work then :-) Of course, the Joomla extension manager is a huge help here too. It has all the current versions listed in relatively convenient location.

What you need to know about migrating from 3 to 5

Now, I'm going to be frank. There are a lot of addons and extensions that work perfectly well in Joomla 3 that even to this day have no update to any newer Joomla version, and probably never will. While Joomla may be open source and "free", some extensions are quite expensive pieces of software. Conversely, other add-ons are completely free of charge. Which has it's own issues.

As fellow users of the popular free K2 blog and multimedia extension know all too well... when people code for free, they're not beholden to anyone... and they understandably prioritise the work that pays the bills.

That said, despite the best intentions... if they promise an update for years on end with no update or ETA in sight, then I do feel they're promising more than they can deliver. Phrases on their forum like "Joomla likes backward compatibility the way vampires like the sun...." (a quote from the K2 forum) are not valid excuses for misleading people into waiting for software that hasn't come. Also... backwards compatibility? I think, misses the point of the Joomla upgrade in the first place. The changes were made to reflect the times, and insisting on making the transition easier by holding onto vast tracts of outdated and insecure code, all to facilitate an easy update for people that are not directly involved with the Joomla development process itself... is unrealistic to say the least.

However, Fotis (of K2) I thank you for making an amazing piece of software that made J3 so incredible, and your generousity for sharing it so kindly with the world. I'm not targeting you specifically, it's just such a shame I had to sacrifice the K2 pages I worked so hard on... to the holy altar of Joomla 4 and beyond.

So what does this mean?

If you can't export the data from the add on into Joomla itself prior to migration/update... that data will be lost because from my own experience, I've learned that you must absolutely, unequivocally, and completely remove each and every third-party extension before you even think of transitioning to Joomla 4 on a live site.

Yes, that's exactly like uninstalling every app you ever installed after the Windows/Mac OS/IOS/Android setup ...before being able to upgrading to the new version.

How is that better than the standard Microsoft way of "Repent, Reflect, Reboot, Retry, Re-update, Re-verse that, Reformat, Reinstall, Re-update (endlessly) and ultimately... Re-consider your Microsoft Choices? ;~)

Well... you can still migrate your native Joomla article data... and if you have a complex site of interconnected information, that's not a trivial advantage. In fact, that's huge! Otherwise, it might be easier to simply start again with a new system.

So what did I have to sacrifice on the "J4 altar" for what is officially called: "a mini update?"

In the end, I had to kill my K2 blog/pages, my Unite Photo Galleries, my personalised template (switch it back to the Joomla 3 default template first) my improved SP Page builder (much nicer Joomla article editor) my video media plugins, and many, many more. I stripped everything back that wasn't created by the Joomla team itself. Why?! Because it broke my everything on my website if I didn't.. and putting it all back over a slow connection took hours...

You put it back, and it "just worked", right?

Sorry, but that's "No! to the power of "Oh heck no!".

You see... if the migration fails for any reason, the data in the database itself is modified to the new Joomla 4 format. So it that happens to you, you might need to purge the "corrupt" old data first.. before restoring Joomla 3.. and I have to say I went "biblical" on it because I didn't just go "Scorched Earth", I accidentally... vaporized the planet entirely... that the drive space allocated to my account was no longer recognized, or controllable from my cPanel interface (not supposed to be possible, but hey). So, the amazingly helpful team at my host service were required to put this site back together... but not before yet another multi-hour long upload of my backup file.

So my site gets resurrected, exactly as I had it.. and now I'm back... but back to "Joomla 3 and square one". I thanked my host service support team and took a break for a few hours to do some actual... you know.. work.

I get back and I still had to upgrade Joomla. So I did what I always intended to do... and immediately broke it again.. because I didn't purge enough Joomla plugins. Yes... I know... but this time I wasn't completely without a hard disk... so I gave restoration a shot. I learned a few "helpful things".

  1. Upload your downloaded zip file "as is". If you unzip it on your computer, (worth doing to check the integrity of the file before upgrading/destroying your site) the permissions are inherited from the folder you unzipped in, and then simply copying the files (without the ability to change ownership) is painful to say the least. Your Joomla site WILL NOT WORK.
  2. Delete and reimport the database information because the migration manipulates the database itself. You don't have to delete your database users or their passwords... but you do have to re-give your existing users the requisite database privileges once you've recreated/restored the database.This is all done in the Databases section of your cPanel interface (or your equivalent).
  3. At least some Joomla 3 installations peak out at PHP version 7.4... so you have to switch it to 8.1 AFTER the successful migration to Joomla 4. Of course, you also need to switch it back for J3 to work after a restore.
  4. Any warning-level flagged extension in the Joomla pre-update check should be treated as a "don't proceed" until it's utterly removed... three test virtual servers just would not work if I tried to keep anything beyond the barest of code. That's how I stuffed up the first time. Go back to step one if you fail this.
  5. The order of removal matters! Let's say that you have the "Ignite Gallery" plugin and it has a plugin, an extension, a module and some bits that make it work nicely with Joomla. Don't remove it one bit at a time... as this can mess your plans up. Use the search function to locate anything with Ignite in the name and uninstall them ALL at once. If you don't... uninstalling the remaining bits may not be possible. If you do get into that situation... go back to step one.

Note about the preupdate checklist. How reliable is it?

Even the extensions that say "hey we're compatible with 4" might not be removable once you've upgraded to 4. Since Joomla 4 has a different structure, it doesn't know how to remove J3 extensions... To make matters worse, J4 compatibility does not guarantee anything beyond that. Some are not compatible at all with version 5. So this leaves you with a two-fold situation:

  1. You can't upgrade to J5 easily (or maybe at all) due to the incompatibilty of your plugins. If you try it's likely that your site will break.
  2. You can't easily (or safely) remove the J3 plugins causing the issues once you've gotten to j4 to work.

    ... leaving you stranded on J4.

So I highly suggest a scorched-Earth (not planet shattering) policy on all non-core Joomla extensions/plugins/addons in the Joomla 3 management page before moving "up?" to 4. To this end it's helpful if you:

  1. READ THE AUTHOR FIELD OF YOUR INSTALLED ADD-ONS CAREFULLY! A plugin from "JoomlaWorks" is not the same as "Joomla". Go through the entire column by hand... and get a second pair of eyes to check if you missed anything. Remove anything not Joomla.
  2. Remember to switch the template back to whatever default one came with Joomla 3... I think it's called "Beez". If you changed the Administator template, you'll need to revert that too. Again, I found it much more likely to update successfully if I removed the non-default templates entirely before the update. However, to uninstall a template, the site cannot be using it at the time. Choose/activate the old default first.
  3. Check that your site is still working with the default template (it's ok if it's a bit ugly right now)... and run your preflight-check for the update. If you get no warnings and no error messages, go ahead with the migration.
  4. If it works... Check your extension manager, (note the completely different admin interface of J4. Get used to that since J5 uses it too).
  5. Remember to upgrade your PHP version to something like 8.1 now at a minimum... although going too far (say 8.3) may cause issues too at this point. This is done NOT in Joomla's page, but either through the cPanel interface of your host service account, or configuration files/update/installation process on a local server.
  6. If needed, update J4 to the latest J4 edition. Some people find it more reliable to update to the closest version to J3 (a early version of 4) and update from there... others find going straight to the latest J4 version better. I personally let the update system decide.. and it went to the latest by itself. That worked for me.
  7. In my cPanel interface (on the hosting service, not Joomla admin page) I found it helpful to get the Joomla database checked for errors... on one of my attempts, it actually fixed some minor errors that helped a lot. Once a bare J4 installation is working...
  8. While everything is stripped bare, it's best if we upgrade to 5 before anything else gets in the way.... run the preflights and check for any warnings. You shouldn't have much to worry about if if you have few plugins. Read any messages and follow their instructions carefully if you get them. If everything is ok, upgrade to J5.
  9. If all goes well, now you can test the site to see if you have the old article data still intact (you should). Might I humbly recommend you backup now that you've done all the hard stuff?
  10. Explore what options you might need to install. A nice template? New gallery system? E-commerce platform? You decide. However, before you spend too much money, or time entering in content... have a play with a few selections. Note what works/doesn't and remove the unused add-ons once you've decided to use another. They just "gum up the works" if you don't.
  11. Check if the other stuff, images/login/contact pages all work as expected and fix/replace them as needed.
  12. Get back to creating content... that's what I've been doing here. :~)

I've summarized this a whole bunch...

Honestly, it's been quite a rollercoaster ride for me emotionally. Mostly frustration and irritation that things that said "this will work"... well... they didn't.

Also, I'd really love a way to import the data back into the main CMS system without just having to "cut and run". Unfortunately, there's simply no nice/reliable way to get the data back that I've had to sacrifice.

This process has taught me the benefit of restraint and careful, perhaps critical consideration when choosing CMS plugins, regardless of whether that's Joomla, Drupal, Wordpress or anything else.

I really hope that I won't have to do this again anytime soon. However, I'm glad that I'm now back in a modern-ish system. If you have had a similar "non-trivial" upgrade path, just know you are certainly not alone.

All the best in your ongoing site management!

Ham.