Your WordPress plugins might be silently losing business data

If your WordPress site uses third-party plugins, you may be experiencing data loss and other problematic behavior without even knowing it.

Like many of you, I’ve become quite attached to WordPress over the past 15 years. It is by far the most popular content management system, powering 28 percent of the Internet, and still the fastest growing, with over 500 sites created on the platform each day. Considering myself well versed in the software, I was surprised to discover — while working on a digital design project for a client — what could be the Y2K of WordPress. Many WordPress plugins are suffering data loss, and it looks like this problem will soon explode if not properly addressed.

The issue is essentially due to the fact that WordPress discards entire datasets even when only one of the data elements within the set contains too many characters for the insertion field. Because WordPress doesn’t log the data loss or any errors related to it, few developers are aware of the issue. And because of one particular scenario involving storing a visitor’s data when they’re connecting with an IPv6 address, the situation is exponentially worse.

Example: Say a WordPress site owner has a plugin installed that lets users add comments. Plugins like that typically store the user’s IP address along with comments they submit, for analytics purposes. For years, plugin developers have assumed that IP addresses were always in the standard IPv4, 15-character format that looks like this: 216.123.123.123. Thus, plugin developers typically set the maximum allowed characters for the IP address database field their plugin uses to about 15-20 characters. However, IPv6 has a much longer 39-character format that looks like this: 2001:0db8:85a3:0000:0000:8a2e:0370:7334.

Unbeknownst to many users, site owners, and developers alike, these longer IPv6 addresses are becoming increasingly widespread. Those new addresses won’t fit into the database fields developers have been using for years. Furthermore, for security purposes, WordPress specifically validates that each part of a data set about to be stored will fit. In the example above, if the IP address is too long, WordPress discards the entire data set (not just the oversized IP address string). Worse, WordPress doesn’t log an error when this happens. The data is simply lost to the ether, without leaving a trace. This two-year-old WordPress bug thread shows how long the WP core devs have known that the community didn’t like this, but they still haven’t addressed it.

Yes, this currently just affects data coming from IPv6 addresses (currently about 17 percent of users). But while IPv6 use may be in the minority right now, it won’t be for long, and as it becomes the majority, these unexplained issues with data loss will reach pandemic proportions if left untreated.

Just how widespread is this?

1.02 million active WordPress plugin installs are silently discarding real visitor logs, content submissions curated by users, and more, right now, all because IPv6 addresses are present in the data being stored. Here are some other interesting stats:

• 50,336 plugins are available at wordpress.org today
• 200 plugins (~1 in 250) create IP address fields that are too short
• Those 200 plugins have over 1 million active installs — a total of 1,023,280.
• Here’s a publicly-accessible Google Sheet my team created that lists all known offending plugins. For each plugin, that sheet includes one example where that plugin declares an IP address field that is too short.

The fix is easy peasy: You simply need to change the table schema for the column that stores IP addresses from 15 to 39 (or more).

This problem can affect applications other than WordPress; really, any application that utilizes IP addresses and stores them in MySQL/PostgreSQL tables (especially in STRICT mode, which would prevent row inserts) where the column max is expecting a 15-character IPv4 IP address.

Debuggin’ the plugin

I uncovered this situation while recently working on a site that needed a rating system that allowed authenticated users to vote on specific post types. So naturally, I did a search of existing plugins that could meet the requirements and found one fairly quickly, CBX Rating, and it was a breeze to configure and get working. Then came the intermittent reports of the form submissions not going through.

I spent hours deactivating other plugins, digging through code, and guiding users via screenshare. I was unable to narrow it down or find any smoking gun. No success message, no error message, no errors in the console log, nothing in the server logs. How could form submissions be failing without errors?

I remembered something I had seen in WordPress before: row inserts silently failing if the data strings were longer than the table column maximums. So I shifted my attention to the back end, and that’s where I found the problem and my boss, Erik Neff (the company’s CTO), helped identify exactly why it was happening.

MySQL databases, not in STRICT mode, will truncate values if they’re over the max character count for a particular column and will insert the new record with a warning. When in STRICT mode, MySQL will not accept the record and will return an error. WordPress, on the other hand, won’t execute a query if it determines the length is longer than the max, and will instead return false, with no error or warning.

When using the WordPress $wpdb->insert method, you get back a 1 upon success and a 0 upon failure. But a function is called before any mySQL statements are executed, and that’s where the problem lies. The function is called protected function process_field_lengths, and it checks to see if the data’s length is less than the max allowable length for that table column. If the length is longer than allowed, the entire insert is aborted and false is returned with no error message or explanation. This is a known issue with WordPress core, and makes debugging that much harder.

The CBX Rating plugin we were using didn’t account for this failure point. I checked the plugin’s table schema and started increasing varchar max lengths across the board. Touchdown! Soon after, I got wind from users of all types that all forms were now being submitted successfully.

My mind raced to how this could be an epidemic, so Erik and I set out to determine the scale. The result of a (rather lengthy) check of WordPress plugins yielded a list of every place an IP address field was declared with an incorrect length. You can find those results in the Google sheet that I’ve made public.

Brett Exnowski is senior developer at Primitive Spark and specializes in complex web applications.