Frank’s “Big Reveal”


Yesterday I wrote about a new app that was garnering attention on some tech and app blogs.  Frank said what? as described by the developer, Terrarium:

So what does Frank do? Frank is the most sophisticated photo captioning app you’ve ever used. More than that, Frank is unpredictable, hilarious and addicting.

The secret to Frank’s magnetism? Instead of randomly assigning pre-written captions, Frank looks at every photo and writes an individual response. Even we are surprised from time to time by the things that Frank says. The end result is the perfect mash-up of Instagram and your favorite meme.

Read Terrarium’s blog here

Posts and comments, in the iTunes store and on blogs, were mostly favourable… initially.  Most bloggers were a tiny bit skeptical believing that..

…a robot is able to individually write all these captions.   Read more: 

But all went on to say, in essence, what a remarkable use of AI object classification technology “Frank” is and everyone was mightily impressed, including myself.  Until…

The disparity in the captions became glaringly obvious.  Some were written in all caps.  Several had spelling errors.  Many, and by late at night MOST, of the captions appeared to be written in broken English.  My guy and I were captioned as being “Cute and happy old age couple.” (and we’re really just middle aged btw, but I guess how we’re perceived is dependent on the viewer).  A younger couple was tagged as… “COME SOON WE ARE GOING TO MARRIAGE”.

I emailed the developer inquiring about the obvious discrepancies and broken English.  This is what I said:

I’ve just pulled together a blog piece about my experience using Frank said what? and wanted to contact you for comment.
The essence of the piece is that the app is not what you claim it to be.  That it is obvious that you have coordinated with people to write the captions for you. There is too much disparity in grammar, syntax and a host of other evidence from occasional all caps usage to spelling mistakes.

While some of the captions are really clever, most leave one with the impression of having somehow been transported to Apparently many of the people tapped to write captions speak English as a second language, and some of them not too well.

Would love to add your comments regarding my assessment if you care to address this with me. Seems like the general public, judging by iTunes feedback commentary, is on to the game as well.



I received two replies.  The first was from Tony at 5:41 a.m. CST.  Tony advised that the team is located on the East Coast and that they’d be responding to my email later that morning.  The second was from Patrick, received at 8:07 a.m. CST and reads as follows:

Hi Kat,
Thank you for your email and the interest in writing about Frank!
We just posted an entry on our blog that explains how Frank works ( It explains in more detail, but yes, the captions are written by humans. We have been using Amazon Mechanical Turk to source the caption writing.
We are still hard at work optimizing our usage of Mechanical Turk, managing for the inconsistencies in responses. Some of the fun of Frank is his unpredictability, but the command of English and speed is something we are improving every day.
If you have any other questions as you are writing your blog piece feel free to reach out to me directly (
Thank you,

Alrighty, then.  I do have questions.  And I emailed Patrick my questions, namely (summarized from email):

Are you comfortable with the ‘sweat-shop’ label associated with Mechanical Turk?   Also, in light of the dire economic situation in the US and the large numbers of unemployed people, are you at all concerned that you are going to be criticized for outsourcing “Frank’s” work?  Your initial description of “Frank” led people to believe that you had developed a super refined AI with a sense of humour.  Are you expecting a storm of criticism from users now that you’ve made your ‘big reveal’?  How will you address what I believe will be the inevitable backlash of people upset their images have been sent to strangers to caption?  How are you addressing the issue of NSFW images being submitted and streamed into the feed?  Lastly, have you notified iTunes to ensure the app is not accessible to minors?  As you move forward with tweaking the app will you be more transparent with the app’s description in the iTunes store?

I heard back from Patrick almost immediately.  I won’t have anything further to add immediately.  He has questions about my questions.

Stay tuned…



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: