Post Meta with Emojis Won’t Save 🐷

tldr:

Convert your DB columns to utf8mb4 to support emojis / special characters everywhere in WordPress

Tweet

tldr2; Or do a urlencode when saving, and urldecode when getting the post meta.

I recently did some client work that involved allowing users to generate their own image (canvas) that contained their name and a bunch of other decorative elements.

We allowed the users to change their name to customize the dynamically populated canvas. The name change was saved every edit via ajax as post meta so that their edits would become permanent.

Easy enough. I coded it, tested it and the new feature went live.

But when people started using it, literally within 5 minutes, I didn’t anticipate that people would decorate their names with emojis.

Some looked like this:

💛💛 Benjamin 💛💛

It was fun, but a bug had emerged. People who were adding emojis to their names weren’t saving in the ajax call.

update_post_meta Was Failing

After digging around, I found the code that wasn’t working as expected:

$text_with_emoji = sanitize_text_field( 'happy 😀' );
update_post_meta( 1, 'foo', $text_with_emoji ); 
var_dump( get_post_meta( 1, 'foo', true ) );
// Will return null!

The text was being sanitized properly, but update_post_meta wasn’t saving if it had an emoji in it!

It didn’t remove the emoji, it didn’t sanitize the emoji for saving, it just.. failed.

WordPress 4.2 – Support for Emojis

I distinctively remember the release of version 4.2 wherein Emoji support was implemented. So I just assumed it worked. Here’s a highlight from the release:

Emoji are now available in WordPress! Get creative and decorate your content with 💙, 🐸, 🐒, 🍕, and all the many other emoji.

If you try and put an emoji in the title or content, they work fine.. even if you use wp_update_post.

But if you add an emoji in a custom field, or add it via update_post_meta, it doesn’t.

So what gives?

Server-Side Fix: utf8mb4

After some more research, I found out that emojis are 4-byte characters and that for WordPress to support emojis and other special characters such as Japanese everywhere, the database should also support 4-byte characters.

So I checked my MySQL database columns and they were utf8_general_ci, which means they won’t support 4-byte characters.

The quick fix is to enable utf8mb4 by converting your column to utf8mb4_general_ci.

BAM! Emojis in post meta now work!

But…

However, it only works in my site. I had control over my MySQL database, so there’s no problem.

But if I’m creating a plugin that a lot of people will use, it should work without the user having to modify their database table structure and risk having their data turned into gibberish (this was a warning PHPMyAdmin gave me when I was converting the column format, scary).

There are web hosts that just provide utf8 MySQL tables. Forcing everyone to convert their database tables is a hassle.

Title & Content vs Custom Fields

So now the question is: If all my database columns are utf8, why do emojis work in titles and post content, but not in custom fields / post meta?

Well the answer is that provisions were added to the title and content to safely save and get 4-byte characters. And post meta doesn’t have those provisions.

Workaroud for utf8

To make emojis work, we’ll have to add some provisions to make them work ourselves.

So here’s the sanitized text, which clearly shows 4 bytes.

var_dump( sanitize_text_field( 'happy 😀' ) );
// Gives happy 😀 (4 bytes)

Since WordPress fails when saving this, we can just encode it into an manageable format:

var_dump( urlencode( sanitize_text_field( 'happy 😀' ) ) );
// Gives happy+%F0%9F%98%80 (4 bytes encoded)

We save that encoded string and we just have to make sure that we decoded it back upon getting:

$text = urlencode( sanitize_text_field( 'happy 😀' ) );
update_post_meta( 1, 'foo', $text );
var_dump( urldecode( get_post_meta( 1, 'foo', true ) ) );
// Gives "happy 😀"

Now you can save emojis in areas in WordPress that doesn’t support them without having to convert database table formats.