In A Recent Video, Practical Engineering Tackles An Important And Often-overlooked Challenge In Civil

Categorizing Posts on Tumblr

Millions of posts are published on Tumblr everyday. Understanding the topical structure of this massive collection of data is a fundamental step to connect users with the content they love, as well as to answer important philosophical questions, such as “cats vs. dogs: who rules on social networks?”

As first step in this direction, we recently developed a post-categorization workflow that aims at associating posts with broad-interest categories, where the list of categories is defined by Tumblr’s on-boarding topics.

Methodology

Posts are heterogeneous in form (video, images, audio, text) and consists of semi-structured data (e.g. a textual post has a title and a body, but the actual textual content is un-structured). Luckily enough, our users do a great job at summarizing the content of their posts with tags. As the distribution below shows, more than 50% of the posts are published with at least one tag.

However, tags define micro-interest segments that are too fine-grained for our goal. Hence, we editorially aggregate tags into semantically coherent topics: our on-boarding categories.

We also compute a score that represents the strength of the affiliation (tag, topic), which is based on approximate string matching and semantic relationships.

Given this input, we can compute a score for each pair (post,topic) as:

where

w(f,t) is the score (tag,topic), or zero if the pair (f,t) does not belong in the dictionary W.

tag-features(p) contains features extracted from the tags associated to the post: raw tag, “normalized” tag, n-grams.

q(f,p) is a weight [0,1] that takes into account the source of the feature (f) in the post (p).

The drawback of this approach is that relies heavily on the dictionary W, which is far from being complete.

To address this issue we exploit another source of data: RelatedTags, an index that provides a list of similar tags by exploiting co-occurence patterns. For each pair (tag,topic) in W, we propagate the affiliation with the topic to its top related tags, smoothing the affiliation score w to reflect the fact these entries (tag,topic) could be noisy.

This computation is followed by filtering phase to remove entries (post,topic) with a low confidence score. Finally, the category with the highest score is associated to the post.

Evaluation

This unsupervised approach to post categorization runs daily on posts created the day before. The next step is to assess the alignment between the predicted category and the most appropriate one.

The results of an editorial evaluation show that the our framework is able to identify in most cases a relevant category, but it also highlights some limitations, such as a limited robustness to polysemy.

We are currently looking into improving the overall performances by exploiting NLP techniques for word embedding and by integrating the extraction and analysis of visual features into the processing pipeline.

Some fun with data

What is the distribution of posts published on Tumblr? Which categories drive more engagements? To analyze these and other questions we analyze the categorized posts over a period of 30 days.

Almost 7% of categorized posts belong to Fashion, with Art as runner up.

The category that drives more engagements is Television, which accounts for over 8% of the reblogs on categorized posts.

However, normalizing by the number of posts published, the category with the highest average of engagements per post isGif Art, followed by Astrology.

Last but not least, here are the stats you all have been waiting for!! Cats are winning on Tumblr… for now…

A brief text history of Gilgamesh, put together for my students.

A basic schematic of why the Korean alphabet is so cool, from Wright House: 

Unlike almost every other alphabet in the world, the Korean alphabet did not evolve. It was invented in 1443 (promulgated in 1446) by a team of linguists and intellectuals commissioned by King Sejong the Great.

In the diagram [above], the Korean consonants are arranged into five main linguistic groups (one per row), depending on where in the mouth contact is made. Notice that there is a graphic element common to all the consonants in a particular row. The first consonant in each row is the most basic and is graphically the simplest; this representative consonant for each group is the building block for the other characters in that group. Certain of these modifications are systematic, and yield similarly modified characters in several groups, such as adding a horizontal line to a simple consonant (a “stop” consonant–such as t/d or p/b–rather than a nasal consonant) to form the aspirated consonants (those made with extra air) and doubling simple consonants to form “tense” consonants (no real equivalent in English).

Notice that the five representative consonants (the ones in the first column in the upper part of the diagram) are also depicted in the drawings that make up the lower part of the diagram showing the relevant part of the mouth involved. Ingeniously, each of these representative consonants is a kind of simplified schematic diagram showing the position of the mouth in forming those consonants.

More details, including subsequent historical changes to Hangul, in the Wikipedia article. 

On this day, 14th February 1779,  Captain James Cook was killed in Hawaii.

James Cook completed three major voyages of discovery. On his first, departing in 1768, he commanded the ‘Endeavour’ on an expedition to chart the transit of Venus. He returned to England in 1771, having also circumnavigated the globe, including exploring and charting New Zealand and Australia’s eastern coast. 

On his second journey (1772-1775), he commanded the 'Resolution’ and the 'Adventure’ on an expedition to the South Pacific, disproving the rumour of a great southern continent, exploring the Antarctic Ocean, New Hebrides and New Caledonia.

Cook’s third and final voyage (1776-1779) of discovery was an attempt to locate a North-West Passage, an ice-free sea route which linked the Atlantic to the Pacific Ocean. Again, Cook commanded the Resolution while Charles Clerke commanded Discovery. Leaving England in 1776, Cook first sailed south to Tahiti to return Omai, a Tahitian man, to his home. Omai had been taken on Cook’s second voyage and had been an object of curiosity in London. It was on this, Cook’s final voyage, that he discovered the Hawaiian Islands in January 1778. This major discovery would lead to his death – Cook was killed on a return visit to Hawaii at Kealakekua Bay, on 14 February 1779.  

Kealakekua Bay was considered the sacred harbour of Lono, the fertility god of the Hawaiians. Cook and his compatriots were welcomed as gods but after one of the crewmen died, exposing the Europeans as mere mortals, relations became strained. On February 4, 1779, the British ships sailed from Kealakekua Bay, but rough seas damaged the foremast of the Resolution, and after only a week at sea the expedition was forced to return to Hawaii.

The Hawaiians greeted Cook and his men by hurling rocks; they then stole a small cutter vessel from the Discovery. Negotiations with King Kalaniopuu for the return of the cutter collapsed after a lesser Hawaiian chief was shot to death and a mob of Hawaiians descended on Cook’s party. The captain and his men fired on the angry Hawaiians, but they were soon overwhelmed, and only a few managed to escape to the safety of the Resolution. Captain Cook himself was killed by the mob. A few days later, the Englishmen retaliated by firing their cannons and muskets at the shore, killing some 30 Hawaiians. The Resolution and Discovery eventually returned to England.

The State Library of New South Wales holds significant original sources relating to James Cook, these paintings from the collection depict the death of Captain Cook.

Carved ditty box shaped like a coffin on silver stand, containing a rough watercolour sketch of the death of Cook, including a lock of Cook’s hair, ca. 1779 / carved by sailors on Cook’s last ship HMS Resolution. State Library of NSW.

Toward an Irish Republic

Ireland lived under England’s rule for almost four centuries. The Irish fought against this occupation through the press, through literature, in politics, and in the streets. They could not, however, create a united front of protesters because they were divided among themselves and struggled with their national identity.

Toward an Irish Republic focuses on the political progress made between the fifteenth and twentieth centuries, as well as the Irish literary revival that occurred at the turn of the nineteenth century.

Driven by Ireland’s desire to build a separate identity from Britain, a distinct Irish literature began to emerge. The establishment of modern Irish literature contributed to the unification of the Irish. They explored their Gaelic roots, searching for stories of characters with the tenacity for which the Irish are now famous. Many Irish men and women also taught themselves the Gaelic language and read folklore. For example, Lady Sydney Morgan, in her novel The Wild Irish Girl (1806), wrote of a young Englishman who travels through Ireland and discovers its history and culture. Standish O’Grady retold Gaelic folktales and historical stories, such as “Birth of King Cormac” and “Teigue Mac Cein’s Adventure”, in his book Silva Gadelica (1892).

One of the greatest points of contention between the Irish and English was their denominational divide. The majority of people in Ireland identified themselves as Catholic, whereas the majority of the people in England were Protestants. The English, strongly opposed to Catholicism since Henry VIII broke away from the Roman Catholic Church, discouraged the Irish from practicing Catholicism. An early example of the political upheaval in Ireland is A Collection of Certain Horrid Murthers in the Several Counties of Ireland. Printed in 1679, this book lists Protestants that were killed during the Irish Rebellion of 1641 when Irish Catholics attempted to gain religious freedom.

Britain’s laws against Catholicism began to weaken by the end of the eighteenth century. In 1792, Theobald Wolfe Tone, one of the leaders of the United Irishmen, put forth his defense for Catholics to gain religious freedom in An Argument on Behalf of the Catholics of Ireland (1792). Many arguments for and against this issue were made, but it was not until 1829 that Catholics were emancipated, with the passage of the Roman Catholic Relief Act.

In 1921, a large portion of Ireland was able to gain independence. The Irish Free State, now known as the Republic of Ireland, was the result of these nationalist movements.

Bibliography

A Collection of Certain Horrid Murthers in the Several Counties of Ireland. London: Printed for Henry Brome, 1679. X 941.506 C685

Theobald Wolfe Tone. An argument on behalf of the Catholics of Ireland. Dublin: The United Irishmen, 1792.  320.9415 T61a5

Standish H. O’Grady. Silva Gadelica (I.-XXXI.): a Collection of Tales in Irish. London: Williams and Norgate, 1892. 891.6208 Og7s

Lady Sydney Morgan. The Wild Irish Girl. London: Printed for Richard Phillips, 1806.  823 M823W 1806

My 2016 In Reading

THE BOOKS I LOVED SO MUCH I WANTED TO SEW THEM INTO MY SKIN AKA MY FAVORITE BOOKS OF THE YEAR IN NO PARTICULAR ORDER

Homegoing by Yaa Gyasi Another Brooklyn by Jacqueline Woodson Today I Am a Book by xTx The Three Woes by Casey Hannan A Bestiary by Lily Hoang Queen of the Night by Alexander Chee The Mothers by Brit Bennett Commonwealth by Ann Patchett

THE BOOK THAT OPENED MY EYES AND MIND AND BROKE MY HEART WITH THE PAINFUL REALITY TOO MANY AMERICANS LIVE WITH

Evicted by Matthew Desmond

THE BOOK THAT WAS TOTAL TRASH AND I THINK THE WRITER HATES FAT PEOPLE WHICH IS FINE BECAUSE WE ALL HAVE OUR ISSUES BUT STILL, GIRL, WHAT….

Maestra by L.S. Hilton

THE COMING OF AGE PROSE POETRY THAT MOVED ME IMMEASURABLY

The Pocket Knife Bible by Anis Mojgani

THE BOOK THAT MADE ME THINK HILLARY CLINTON REALLY WAS GOING TO WIN THE PRESIDENCY

All the Single Ladies by Rebecca Traister

THE STRANGE BOOK ABOUT LONELINESS AND THE THINGS WE DO ONLINE THAT I HIGHLY RECOMMEND

Valletta78 by Erin Fitzgerald

THE POETRY BOOK I DIDN’T UNDERSTAND AT ALL THOUGH I COULD TELL THE POEMS WERE SUPER SMART

The House of Lords and Commons by Ishion Hutchinson

THE ACTION THRILLER THAT HAD LOTS OF HYPE BLURBS BUT WAS ONLY SO SO

The Second Life of Nick Mason by Scott Hamilton

THE RETELLING OF A CLASSIC THAT I REALLY ENJOYED, WHICH SURPRISED ME AND ALSO THE AUTHOR WROTE ONE OF MY FAVORITE BOOKS OF ALL TIME, AMERICAN WIFE

Eligible by Curtis Sittenfeld

THE BOOK THAT MADE ME CRY BECAUSE IT HELD SO MUCH I COULD RELATE TO AND THEN MADE ME A LITTLE MAD

13 Ways of Looking at a Fat Girl by Mona Awad

EXCELLENT SMALL PRESS BOOKS YOU SHOULD CHECK OUT

Pink Museum by Caroline Crew The Farmacist by Ashley Farmer The Voyager Record by Anthony Michael Morena Massive Cleansing Fire by Dave Housley

THE BOOK I READ TO LEARN HOW TO WRITE A COMIC BOOK SERIES EVEN THOUGH I WAS WRITING FOR THEIR MAJOR COMPETITOR

The DC Comics Guide to Writing Comics by Dennis O’Neil

THE COMIC BOOK I LOVED AND RECOMMEND OFTEN

Saga by Brian Vaughan

THE COMIC BOOK ISSUE I READ AND THOUGHT WAS NOT SO GOOD SO I HAVEN’T READ ANY OTHER ISSUES IN THE SERIES

Wonder Woman Rebirth #1

THE BOOK I WROTE AN INTRODUCTION FOR (OUT IN 2017! FROM BEACON PRESS!)

Like One of the Family by Alice Childress

THE BOOK I REVIEWED FOR THE NEW YORK TIMES

Small Great Things by Jodi Picoult

THE BOOK I WANTED TO LOVE THAT HAD GORGEOUS OBSERVATIONS OF WOMEN’S FRIENDSHIPS

Rich and Pretty by Rumaan Alam

THE BOOK ABOUT CHEFS AND THEIR TATTOOS WITH FASCINATING STORIES OF WHY PEOPLE PERMANENTLY INK THEIR SKIN

Knives and Ink by Isaac Fitzgerald and Wendy MacNaughton

THE BOOK I READ BECAUSE I SAW A PREVIEW FOR THE TV SHOW AND LEARNED IT WAS BASED ON A BOOK SO I STARTED WONDERING IF THE BOOK WAS GOOD

Queen of the South by Arturo Perez-Reverte

SOME VERY GOOD BOOKS YOU SHOULD CHECK OUT BECAUSE THE STORIES ARE WARM AND/OR INTELLIGENT AND/OR STRANGE AND/OR GRIPPING AND/OR INTENSE

Turner House by Angela Flournoy LaRose by Louise Erdrich The Wangs vs the World by Jade Chang The Story of My Teeth by Valerie Luiselli You Will Know Me by Megan Abbott

THE HEARTBREAKING BOOK ABOUT BEING GAY IN THE MIDDLE EAST DURING THESE TUMULTUOUS TIMES FROM A WRITER WITH A LOT OF POTENTIAL

Guapa by Saleem Haddad

GORGEOUS BOOKS OF POETRY I REALLY LOVED

Night Sky With Exit Wounds by Ocean Vuong L’Heure Bleue by Elisa Gabbert The New Testament by Jericho Brown Look by Solmaz Sharif There Are More Beautiful Things Than Beyoncé by Morgan Parker

THE EXCELLENT BOOK I CHOSE AS MY SELECTION FOR BOOK OF THE MONTH CLUB

The Veins of the Ocean by Patricia Engel

THE BOOK I READ BASICALLY TO IMPRESS A GIRL AND IT WAS A PRETTY GOOD BOOK ALSO AND I HOPE THE GIRL WAS IMPRESSED BY MY DEDICATION BECAUSE THE BOOK WAS VERY LONG

The Fireman by Joe Hill

THE BOOK WITH AN AMAZING TITLE,  SOME REALLY GOOD STORIES INCLUDING A RIFF ON ANTIQUES ROADSHOW AND ALSO SOME STORIES I LIKED LESS

American Housewife by Helen Ellis

THE BOOK THAT WAS EXCEPTIONALLY WRITTEN BUT I WANTED THE ACTUAL RAILROAD PART TO BE MORE FULLY REALIZED

The Underground Railroad by Colson Whitehead

FUN BOOKS THAT WERE FUN

The Assistants by Camille Perri China Rich Girlfriend by Kevin Kwan

THE BOOK ABOUT BEING SINGLE TOWARD THE MIDDLE OF YOUR LIFE THAT PRETTY MUCH EVERYONE IS GOING TO LOVE WHEN IT COMES OUT

All Grown Up by Jami Attenberg

THE EXCELLENT SHORT STORY COLLECTIONS COMING OUT AROUND THE SAME TIME AS DIFFICULT WOMEN THAT MADE ME JEALOUS AND ALSO SCARED OF THE COMPETITION

Homesick for Another World by Ottessa Moshfegh Always Happy Hour by Mary Miller

THE BOOK THAT WAS NOT MY CUP OF TEA BUT IT’S ME NOT THE BOOK

300 Arguments by Sarah Manguso

THE BOOKS I BLURBED (AND THEREFORE REALLY ENJOYED)

You’re the  Most Beautiful Thing That Happened by Arisa White In the Not Quite Dark by Dana Johnson I Almost Forgot About You by Terry McMillan Feminist Baby by Loryn Brantz Lower Ed: The Troubling Rise of For-Profit Colleges in the New Economy by Tressie McMillan Cottom Bruja by Wendy C. Ortiz Sing For Your Life by Daniel Bergner Made for Love by Alissa Nutting

Once Upon a Time t

Once upon the time t(0) there was a young buck named Butterbean who wanted nothing more than to know his ontological value.  Being familiar with the concepts of quantum mechanics he was sadly aware that this was theoretically impossible, but remained unsatisfied with the notion.

In an undying effort to discover the nature of his own existence he set out on a journey to seek the answer from all those most wise and perceiving.

Clearly the first stop was at the front door of Glad the hippopotamus.    

Glad was of the notion that all things are ultimately mundane and that it is simply a matter of time and “progress” before esoteric conundrums become  simple everyday knowledge.  Surely this most assured creature could derive an unknown variable to discover the true nature of Butterbean himself. They were both of the mind at this time t(realist) that all the probabilistic nonsense was clearly just a cop out for those ninnies who fancied themselves finished with discovering the universe.

Alas after decades, or maybe days (no one can be sure given that time itself is a construct  commonly defined as the duration of 9,192,631,770 periods of the radiation corresponding to the transition between the two hyperfine levels of the ground state of the cesium 133 atom, and who can really say cesium 133 even exists… really), Glad was forced to admit that he could not tell Butterbean of his ontological state prior to Butterbean’s arrival on his doorstep, nor could he possibly without any reference to his own experience as a literalist hippo,  and was forced to admit that this coveted obscure variable was beyond his abilities of perception.

With a sense of hopelessness Butterbean sought out the comfort and guidance of Ol’ Trusty, the Wire Stripper.  They adventured together for t(hedonist) to t(fundamentalist) without ever committing to any defined notion as after all, how can you find the answer to a paradox, oxymoron much?  

However comfort in ignorance becomes unsettling after you’ve had your fun and Butterbean became restless.  There was one more known avenue he had not yet travelled…

With a renewed sense of wonder and determination Butterbean approached the Oak, which shook its branches excitedly upon Butterbean’s arrival to see such a valiant seeker of answers.  “Oak, what can you tell me about my ontological value?” asked Butterbean humbly.  The Oak remained silent, but a slight breeze rustled loose a few leaves from its massive head which fluttered to the ground and landed lightly at Butterbean’s feet, Butterbean faltered and then turned and retreated thoughtfully sensing that the conversation was over. 

Upon further consideration over Δt(orthodox) Butterbean understood Oak’s lesson— the leaves could change position through many methods all resulting in different outcomes, however before any leaf falls, it has not fallen.  Butterbean had discovered that no matter who or what observes him, there would be an outcome, but it could not be these interactions which defined the whole of himself, surely he must be something inherently like the leaves of Oak’s mane.  

One day, a little time later at approximately t(decision), Butterbean decided to venture forth in search of a method of measurement that excluded observation.  Knowing that the existence of alligators had long been debated among those with a higher understanding of the nature of things, he knew he should seek within the mind of that which may or may not exist; namely an alligator (though a unicorn or manticore could have sufficed, they are not quite as easy to find nowadays).

Unfortunately all alligators are actually just dead trees and Butterbean had to once again rethink his strategy. 

It occurred to Butterbean, after his most recent experiences, that perhaps his disbelief in alligators had caused their value to be null, distracted in hoping that his mind was not quite that powerful Butterbean mistakenly stomped on a daisy which let out a loud squeal, “watch out dummy!”  “OH!” replied Butterbean, “Sorry I didn’t see you there, though come to think of it I’m surprised you even exist as I hadn’t yet observed you my friend.”   The daisy straightened herself and smoothing out her petals scoffed and said jeeringly, “you think that you are the only one who exists, eh?”  “Well,” began Butterbean thoughtfully, “no, but I don’t understand how anything or anyone has any sort of defined nature without something else purposefully interacting with it, after all, how do you know that you exist?” “Oh for goodness sake,” exclaimed young Daisy, “Of course you’ll remain limited as long as you keep collapsing your wave function with all that introspection!”  And she turned her face to the sun clearly dismissing Butterbean and all his angsty questions.  

Butterbean, with all known theories exhausted, was still unsatisfied and longed to know the mystery of his own nature of being and the recent dismissal of the daisy weighed heavy on his soul.  “Why can she be so satisfied just being,” he wondered, “when I spend all of my time dedicated to the mystery and nature of life but still feel so empty and unfulfilled?”  And suddenly it struck him! It had been so obvious all along and basically verbatim the retorts he had only just received from the annoyed flower he had trod on, Butterbean ceased to observe himself and existed infinitely in all possible states of being.

Sailor’s Eyeballs (Valonia ventricosa)

Known as bubble algae or sailor’s eyeballs, Valonia ventricosa are one of the world’s largest single-celled organisms. They’re found in almost every ocean in the world, mostly in tropical and sub-tropical regions among coral rubble.

These tough, shiny multi-nucleic cells, a kind of green algae, usually grow to be 0.4 to 1.5 inches in diameter but sometimes reach up to 2 inches across. By comparison, most human cells are so small they’re invisible to the naked eye; Valonia ventricosa are larger than your fingernail!

photograph by Alexander Vasenin | Wikipedia

via: American Museum of Natural History

In 1992, a man named Wu Anai, near a Chinese village in Longyou County, based on a hunch, began to pump water out of a pond in his village. Anai believed the pond was not natural, nor was it infinitely deep as the local lore went, and he decided to prove it. He convinced some of his villagers and together they bought a water pump and began to siphon water out of the pond. After 17 days of pumping, the water level fell enough to reveal the flooded entrance to an ancient, man-made cave!

The cave has twenty-four rooms. There are pillars, staircases, and high ceilings over 30 meters (98 ft!) up. The work was done by humans, we know, because they left visible chisel marks in uniform bands of parallel groves. With over 30,000 square meters of space, all meticulously chiseled, this would have been a huge undertaking. Even if people were simply enlarging caves which already existed, it would still have required a lot of manpower working in a coordinated system for a long period of time.

Since the project would have been so large, it seems amazing that no record of it exists in China’s extensive written history. But there is not a word. Based on the cave alone, it is estimated to have been completed around 200 BCE, near the Qin Dynasty or Han Dynasties.

Speed sign on Dursey island, Co. Cork, Ireland