I’ve been responsible for maintaining a number of WordPress websites in my time, some larger than others. This one, for example, gets almost zero traffic, but I like to think that the hits I do get to this site are people looking for very specific answers that I’ve been able to provide. Today I’m going to try and add another helpful insight about WordPress optimization and troubleshooting with the hopes that it might help someone else in the future.
The Setup
The blog that gets most of my attention these days is Thirty-One Whiskey, where I (and my motley crew of partners) drink spirits and review them. It gets a decent amount of traffic normally, but recently a change in the Google algorithm has pushed us further down the page than I would have liked. As a result I’ve been on a crusade optimizing the website and getting it to load as quickly as possible. This included:
- Ensuring that the theme I chose was, by default, quick and not bloated
- Implementing a local page cache
- Moving images to AWS, optimizing their sizes, and serving them from a CDN
- Implementing a CDN for the HTML and JS files served directly from the webserver
- Moving all possible JS files to be loaded asynchronoously
- Implementing new CSS that hides the first large image when viewing the site from a mobile device (to reduce LCP time)
- Moving from shared hosting to a dedicated private virtual server for both Apache/PHP and MySQL.
In short, I pulled out all the stops. I did everything that I possibly could to make it load faster. And in general things were working — my Google PageSpeed Insights scores were improving, but they were still inconsistent.
The problem I started seeing was that, while most of the time my pages would load just fine (less than 1 second), every so often they would hang for four to ten seconds or even fail to load altogether. It was intermittent, not consistent with any specific page or action.
That was going to be a problem, and I needed to fix it. Not just for my visitor’s experience, but it also prevented me from even saving my work or getting into the admin side of the site.
Troubleshooting
The very first instinct I have is to go on a killing spree in my plugins. I disabled each of them one at a time, looked at the results, and then moved on. Sometimes the results would seem to be positive for a while, but eventually the problem would pop up again and I would be back to square one.
In the end there wasn’t anything in the plugins that was causing the issue, but it was still a good excuse to disable some of the unnecessary ones. The fewer plugins are running the less code needs to run, which will improve stability and responsiveness. Even if it didn’t fix the specific problem I was having.
My next thought was that the MySQL database for my website might be causing the problem. It was on a shared hosting site, meaning that either someone else with a busy site or one of my other websites could have been generating a heavy load on the device and causing it to respond slowly. I had been considering moving to a dedicated MySQL server for some time now so I bit the bullet and paid the money to have it configured, and overnight my databases and connections were transferred over to the new device. The queries were noticeably snappier, but the problem still persisted.
About this time I was running out of ideas. I had tried all the normal things, Googled for any other solutions, and was coming up empty. I even looked through my custom plugins and themes to try and see if there was anything I could optimize, but I doubted that an extra database call would make that much of a difference. And even if it did, the fact that it was intermittent and not consistent every time a specific function ran made me suspect that something else was going on.
That brought me to install Query Monitor, a WordPress plugin that seemed like it would give me some good insight into what was going on under the hood. I slapped it in place and started browsing around, and the first time the issue happened I was able to open up the little on-screen pop-up and see exactly what query had been slowing me down.
One thing I saw I expected: about a hundred queries that were made against the DB to render that single page. Which seemed high to me, but not outrageous.
What I didn’t expect: one single solitary INSERT query.
How WordPress Handles Scheduled Tasks
Smartly, WordPress is designed to be installed on a website as quickly and as lightweight as possible. There are no packages to be installed on the server and no extraneous code — it all just lives in one single directory on your server. This makes it incredibly fast to deploy and easy to manage, but it comes with a downside.
Since there is no running application on your server itself, the WordPress installation can’t do anything on its own. For example, if you set your site to publish an article at 1 AM, or want to schedule some automatic updates, there’s no way for WordPress to do that all by itself. Any installed application would be able to have an active thread on the system or install something in the “crontab” (scheduled action mechanism for linux systems) but WordPress doesn’t have that luxury.
Instead, WordPress smartly uses the visitors to your site to perform those actions. Every time a visitor loads your site, part of what the code does is look to see if there are any time-based actions that need to be performed. It then kicks off those actions and renders the site for that viewer as if everything was normal. In theory everything should appear normal to that visitor, and they would never know that we hijacked their visit to perform some internal maintenance.
For sites with large numbers of visitors, this would result in a longer than usual page loading time for a vanishingly small percentage of the viewers. Smaller sites may see that for a larger percentage. For me, I had a unique problem.
Issues with WordPress WP-Cron.php and Highly Cached Sites
I had done things too well, it seems. Everything on my site was cached to the nines, which pushed that content as close to the site visitor as possible and ensured the smoothest and quickest loading time. But in doing so, I also reduce the level of traffic that was doing a “home run” — going all the way back to my server and hitting the actual website.
When everything is cached, the server basically sees next to no traffic. Which is the point — and also a bad thing.
Remember that WordPress relies on site traffic to trigger scheduled activities, everything from posting new articles to updating itself on a regular schedule. With fewer pageviews going all the way back to the server, this meant that these tasks started piling up. Whenever someone actually hit the site, it took longer for the system to process all of the missed jobs. And the likelihood of that page view taking longer or timing out altogether was drastically increased.
Especially the way I designed the site, things were almost guaranteed to result in me especially seeing this issue. The WP Cache plugin I used allowed me to configure that all pages would be cached — but anyone logged into the site wouldn’t see that cache. Every page would be a fresh query and the latest data. And the CDN I had set up specifically exempted the wp-admin folder from the cache, so every call to those pages would also be directly to the server.
In effect, what I did was ensure that only I would ever actually load the site directly from the webserver. And therefore, every so often, when the WordPress had scheduled a check of its scheduled tasks, I would be the only one to get these long timeouts.
That’s what the INSERT query was — WordPress confirming that the scheduled task check had been completed.
Moving WordPress’ wp-cron.php Externally
There is another option for how you can trigger these scheduled tasks — they don’t have to be triggered by visitors alone. And that’s exactly how I solved my problem.
Step one is to disable the wp-cron.php file from being loaded on each pageview — that’s the one which does all the actions to identify scheduled tasks and perform them. Helpfully WordPress has a way of disabling this activity, specifically by adding a line to your wp-config.php file disabling it:
define('DISABLE_WP_CRON', true) ;With this disabled, WordPress will no longer use page visitors to perform scheduled tasks. Which, for me, instantly solved my timeout issue. But now I needed to figure out a way to trigger this file deliberately and regularly.
The straightforward approach would be to simply cURL that file every 10 minutes or so. It’s a simple thing to set up, especially since the wp-cron.php file is just in the root directory of your WordPress installation. So, in theory, you should be able to add the following to a crontab or some kind of script:
curl -vIk https://www.yourwebsitehere.com/wp-cron.php?doing_wp_cronMy problem is that I had specifically cached everything on the site, with the exception of anything behind the admin folder. So instead of simply calling the file through a web address, I needed to configure my server to actually call the file directly:
php /home/user/wordpress/wp-cron.phpWith that in place, everything started humming like a finely tuned machine.