ERROR 1118 (42000): Row size too large (> 8126). Changing some columns to TEXT or BLOB may help. In current row format, BLOB prefix of 0 bytes is stored inline.

Initial attempts

The first attempt to resolve the issue involved converting all columns to TEXT, as their contents are stored off-page. That didn't work: there were too much columns, and even the references consumed too much space.

Next, I considered changing the row format, which can sometimes resolve this issue. Unfortunately, the customer had already set it to DYNAMIC, the best available option for this scenario. No gains there.

InnoDB page size

The obvious fix for running out of space is creating more space. MySQL has an innodb_page_size parameter that defines the tablespace's page size. The data file consists of virtual pages, which are completely overwritten each time a change is made.

MySQL stores two rows in a page, meaning that when you update one, you're effectively saving two. This is an important setting for performance, but in this case, it's relevant for another reason: a row can use at most half of the InnoDB page size.

The innodb_page_size setting is somewhat misleading because its value is stored inside the ibdata1 file, which contains your MySQL data. Once the first page has been written, you can't change the value anymore. All pages inside the file must be the same size.

Thus, changing it means recreating everything, which involves a process like this:

mysqldump --all-databases > backup.sql
systemctl stop mysql
rm -rf /var/lib/mysql/*
mysqld --user mysql --initialize --innodb-page-size=32K
systemctl start mysql
cat backup.sql | mysql

However, on our Percona PXC cluster, I was met with an error:

2025-03-04T13:50:32.929013Z 0 [ERROR] [MY-010872] [InnoDB] InnoDB: Invalid page size=32768.

That was surprising, as the documentation lists 32768 (32K) as a valid option.

The real problem

I spent way too much time debugging this because the error message didn't lead to any useful documentation. It turns out that Percona PXC cluster does not allow changing innodb_page_size. They only accept the default value of 16K, and any other setting will fail.

From (an older) Percona PDF manual:

Only the default 16 KB page size (innodb_page_size=16384) is accepted until the relevant upstream bug is fixed by Codership (see https://github.com/codership/galera/issues/398). All other sizes will report Invalid page size and shut down (the server will not start up).

Newer documentation doesn't seem to mention this anymore, but it still applies.

Watch out with innodb_strict_mode

While researching this issue, I found many Stack Overflow posts suggesting that disabling innodb_strict_mode might help. Be very careful with this!

Disabling strict mode only replaces the critical error with a warning. You'll be able to create the table, but the underlying issue remains: the rows are still too large.

It may not fail immediately, but at some point, an INSERT or UPDATE operation will trigger an error if it exceeds the maximum size. This shifts the failure from a controlled event (such as during deployment) to a random action that a user might perform — not ideal.

Luckily, we can use OpenSSL to check whether a public and private key match.

For RSA keys

The classic way to compare RSA keys is to check whether the modulus is the same, as this is the only part that is shared between the public and private keys. If the modulus isn't the same, the keys can't form a pair.

Read the modulus from the public key:

openssl rsa -pubin -in public.pem -modulus -noout | openssl sha1

Then do the same with the private RSA key:

openssl rsa -in private.pem -modulus -noout | openssl sha1

If the two values match, the key files form a pair.

Note that we're piping the actual output of the -modulus command to openssl sha1 to get a shorter hash of the modulus. This makes it easier to manually compare the two values, but it's not strictly necessary. You could also compare the output directly if you're feeling brave.

For EC keys

Elliptic curve keys don't have an easy-to-compare modulus like the RSA keys above had. However, it's always possible to generate the public key² part when we have the private key.

Using the private key, generate (a hash of) the public key:

openssl pkey -pubout -in private.pem | openssl sha1

Then compare the output with the public key file you have. We need to make openssl load it and print it out again to make sure we get the exact same format.

openssl x509 -pubkey -in public.pem -noout | openssl sha1

If the two values match, these keys form a pair.

• ssh yoursitebe@ssh.yoursite.be
• ssh yoursitebe@ssh123.webhosting.be

The first one may not work if your DNS records are not correct, the second one always works but is hard to remember. With just a small change to our local SSH config, we can use a third option: simply connect to your domain name!

• ssh yoursite.be

For this to work we have to tell ssh what to do with yoursite.be.

The local SSH config file

On computers running Linux and macOS, the configuration file can be found at:

~/.ssh/config

The file (or the .ssh directory) may not exist yet, you can create it manually.

Inside the file you can specify Host blocks to tell the SSH client what to do with certain hostnames. We'll use our domain name as that's the natural address to connect to when we want to update our website.

Host yoursite.be
	HostName ssh123.webhosting.be
	User yoursitebe

Host othersite.com
	HostName ssh472.webhosting.be
	User othersitecom

The correct values for the HostName and User parameters can be found after logging in to the My Combell control panel. Navigate to the correct web hosting account and click the SSH menu option on the left. You'll see the SSH host and SSH user name listed there.

After saving the file, you can connect right away:

ssh yoursite.be
Last login: Mon Dec 16 14:28:14 2019 from 178-221-32-24.access.telenet.be
yoursitebe@com-linssh123:~#

This also works when using scp or rsync to transfer files!