The original idea, which was too ambitious to do all at once, was: personalize article content to A) show readers more of what they find interesting, and B) add background on topics they don't know much about.

We broke the work out into steps.

Verifying our ideas

We did this by taking an already-published article, adding different kinds of expandable annotations to it, and user testing it. We tested it with five users we hired from Craigslist, who said they could see themselves using the feature.

Testing it live

To test it with a larger audience, we added expandable annotations to a live article on the Post site. We tracked open rates on individual supplements and time spent against a control group. Our results were inconclusive, unfortunately— so we did a user survey. Results were overwhelmingly positive.

What our first live test looked like.

Building a manual annotation tool for the newsroom

We got the go-ahead to create a newsroom tool that would allow writers to create, share and embed their own annotations. We built a simple admin and frontend similar to our initial tests, and found newsroom partners to run a pilot program for the new tool. We called it "Notes."

I organized the pilot program with the newsroom, and found several writers and editors agreed to use the tool several times a week for a month. Afterward, I sent out a survey to get feedback from them on their experience and likes/dislikes. The newsroom liked the tool, but found the embed process inconvenient-- they had to copy/paste an embed code every time they wanted to add an annotation.

Click here to see how annotations worked.

Testing automatic embeds, adding HTML/CSS support

In 2017 we redesigned the admin to make improvements requested by the newsroom, added HTML/CSS support, and worked with our automation team to auto-update and -embed notes in articles. We renamed the tool "Context" because no one liked the name "Notes" anymore.

An unused politics annotation card.
The new admin: annotations are grouped by subject, so they're easier to find, edit and embed.

Next steps: TBD

Adding automation to this tool was more difficult than we expected-- we successfully auto-created and -embedded stats cards for all NFL and MLB players (which are still live on site), but it's difficult to match information to people and events in other subjects. There isn't much interest in further tests in this area now. The tool is being slated to add support for document hosting.

If I could go back and start over, I would have pressured stakeholders to support auto-embeds sooner. We could never quite find out how to get the newsroom to regularly use it.