Lately, by coincidance, I’ve run into a lot of really cool algorithms (see my recent series on bayes filtering). I’m not a math or a CS major, but surprisingly often, I wonder how something is done, do a Google search, and come across a solution so simple and intuitive that even I could implement it in an afternoon.

One of these is text-rank. It’s what’s known as a “Graph-based ranking algorithm”, and it’s designed to summarize long texts. It can shorten a 7000-word article to 1300 words and retain most of the most important ideas in the piece. I would have thought algorithms like these to be very complex, but text-rank, and other summarization algorithms, are remarkably simply. Bascially, they break down a text into sentences and paragraphs, compare sentences to see which sentences overlap the most in content with other sentences (making them the most valuable), and then cut out all but the most valuable of sentences.

I don’t know what use I’ll ever make of this, but it’s cool to understand how such a simple little idea can be used so powerfully.

It’s finals week here at UChicago, and I’m trying something I haven’t really done in years. I’m shutting off Facebook-proper. I uninstalled the Facebook app from my phone, and logged off on my laptop (I’ll still be using FB messenger and events, which are thankfully seperate apps).

Facebook has its merits. It’s an immensely powerful communications platform, and a great way to discover articles. It’s all-consuming though. The news feed just sucks time out of my day, and I can’t say I’m necessarily better for the constant stream of information available to me. Plus, it’s finals week, and I really shouldn’t be spending time on unneccesary communications, so I figured now was a great time to take a break. We’ll see how it goes.

As computing gets better and the average quality of professional writing gets worse/more formulaic, it seems inevitable for us to approach a cross-over, where journalism is entirely, or largely computer-generated, and we can’t tell the difference.

In fact, most of what the average millenial spends their time reading has been arranged for them by a computer. Facebook feeds, Instagram feeds, and Google search results may be sharing human-generated content, but at a certain point, which we’re approaching, the editorial voice of the algorithm overwhelms the voice of even the most powerful human influencer.

This scares me a little, but it also intrigues me, which is why I’ve started working on a new side-project I’m calling “Journalism Machine”. Journalism machine takes small snippets of human-generated content and larger templates for modern writing, and attempts to turn it into a successful, automated publication, complete with social media interactions. Social media generates a custom feed for every individual’s tastes. Why can’t we do the same thing with journalism? Why can’t we, in the near future, have articles customized to what we know, what we believe, and what we’d share?

Right now the mechanics of my project are pretty simple:

• Build a database of content snippets.
• Build a set of social and article templates.
• Build a script for turning all of those database snippets into articles, posting them, and sharing them with the right audience.

In the future though, there are some even more interesting experimental features I’m thinking about adding.

First of all, I’m considering ways to use small, cheap human-interface tasks to generate more snippets, and more articles. APIs like Mechanical Turk and Scale would bring the cost of generating sentences down to just a few cents a piece, versus hiring a journalist to write an article for 10x that.

Second, I’m thinking about ways to make it less noticeable when content is re-used from article to article. Search engines don’t like duplicate content, and neither do readers.

Third, I’m thinking about how machine learning techniques could help produce 10x the content for each human-generated sentence. If journalism is dissolved into a series of patterns, what’s to stop me from using a pattern-matching algorithm to generate sentences without a human writer?

I don’t want journalism to dissolve like this. I want human journalists chasing leads, and writing stories for the public, not for one person. This is where the future is going though, so I’m determined to understand it.