April Fools Song #7 Ying Yang Twins Feat Adam Levine - Live Again

Did you know that ants flow like melted cheese?

Easy network graphs for Tableau with NodeXL

Download Node XL

Enter some edges, add additional measures and/or dimensions

Click “Workbook Columns” -> tick “Layout”

Choose your graph type and layout

Click “automate”

Delete row 1 in both Edges and Vertices.  Save Edges and Vertices as CSV.

Create a new Excel workbook and import Edges and Vertices as two new sheets.

In “Edges” create 4 new columns: Vertex1x, Vertex1y, Vertex2x, Vertex2y. Perform a VLOOKUP Vertex 1 from “Edges” against Vertex in “Vertices” to populate the x and y and then do Vertex 2.  

Select cells in “Edges” (CTRL-A) and Paste special -> values only. We can then delete “Vertices”. 

Copy “Edges” to another sheet.  Rename Edges “From” and the copy “To”.  Add a column in both called “pathorder”.  “From” will be populated with “1″ and “To” will be populated with “2″.

Then open Tableau, choose Excel, open with legacy connection. 

Choose custom SQL and enter the query:

SELECT *, Vertex1x AS [X], Vertex1y AS [Y] FROM [From$] UNION ALL SELECT *, Vertex2x AS [X], Vertex2y AS [Y] FROM [To$]

Then in Tableau, build a dual axis, line mark type for edges, and circle for vertices.  Change ID, pathorder, X and Y to dimensions.

You should now have a network graph in Tableau.

Note, I haven’t gone in depth about the mathematics here.  It’s just a simple example.  Also, you maybe thinking, what if I have more than 500,000 rows.  Well, just append your CSVs into Postgre SQL and do the union there.  Or maybe you want to generate a VBA Macro to automate this process? If you do, you can always contribute your script to a github for Tableau.

Link to workbook here.

At this point, I’d also like to credit:

Robert Mundigl

Clearly and Simply network graphs in Tableau and 

Bora Bevan 

Dynamic Network Graph Layouts in Tableau using R

Clouds Crashing Over Nebraska

Storm chaser Alex Schueth captured timelapse of a rare cloud formation called undulatus asperatus during a storm over Lincoln, Nebraska earlier this summer. The term, which translates to “roughed or agitated waves,” describes the bizarre rolling pattern formed by the clouds. Observers have noted that the phenomenon gives the impression of being underwater looking up at the surface at waves. Margaret LeMone, a cloud expert with the National Center for Atmospheric Research has taken photos of asperatus clouds for 30 years, and considers it a likely new cloud type.

How Jacques Rivette’s first films helped launch the French New Wave.

Click here to read and watch clips from our supplements on PARIS BELONGS TO US.

Last Military Engagement Win

The king vulture is best known for its beautifully coloured face and head, and is thus considered more “charismatic” than most other vulture species.  These colours are not feathers, but rather patterns directly on the bird’s skin.  Like many vultures, the king vulture’s head is almost entirely bald (save for some bristle-like black feathers).  This prevents rotting meat and blood from being trapped in the vulture’s feathers, possibly leading to bacterial infections, when the bird feeds.

Carpathian Mountains Karpaty Ukraine, Vintage Soviet Travel Postcards, Full set of 8 postcards (1967), USSR republics, soviet postcard set http://ift.tt/1ohG4sI

Lost Books, Lips and

A patron approached the desk with two small children.

Patron: “Could you settle something for me? My grandkids want to ask what you do to someone who loses a book, because on the way here, the one told the other that they better not lose their books because the librarians will tear their lips off.”

Me: “Uh…No! No, that’s harsh. We just make you pay some money so we can buy a new copy.”

Tableau 9.0 beta, some network graphs, and my missions for this year

View the interactive visualizations here.

The Internet Backbone, Europe, 2015.  Data via Maxmind.  Background map for reference purposes only (chord lines shown).

The Internet Backbone, 2005.  Data via The Opte Project.

Tableau 9.0 beta 

Tableau 9.0 beta has been released, and Tableau Public has shaped up to be real eye candy.  The UI in both desktop and Tableau Public have been given an overhaul, and there is a lot to like.  Regular expressions, random function, in pill editing, and the mapping gets a lot more power: in-map entity search, lasso/marquee select - although my favorite new capability (although the Apple Mac users have had it a while) - a color picker! With RGB and hex values!

Network graphs

I’ve been quietly sitting on the two data sets above, trying to make sense of the underlying fields; for example, with the maxmind dataset, I had to work out how to parse an interger as an octlet; i.e. 1.0.0.1 - and then using split_part in Postgre SQL (no CHARINDEX function available) to get rid of the end of block. The workbook has two inner joins and a union (Maxmind), and a second data source in MS Excel.

Why bother though?  Well, I’m fascinated by flow. Whether that is people, aircraft, money, or ip packets. Of course we can animate these graphs over time - but really, what are we looking for; what is the story?

Well I’ll tell you what I think know from experience what decision makers are looking for.  They are looking for anomalies. They want to know what are the biggest fluctuations, usually in the fattest “pipes”.  Flat lines and small pipes (think opportunities) are typically just noise.  For those building bar and pie charts into pretty looking dashboards, this is going to come as news and a bit of a shock.

Graph edges (vertices) can be encoded in size by metric - and once you build the matrix (array) you can also compute cost (distance is a normal consideration, time, bandwidth etc).  This can also be represented as a Sankey, but if there is a geo-spatial component to the data, then a map helps us interpret the data quicker.  As cave dwellers, we didn’t make bar or pie charts to tally up the hunt/gender ratios, we drew a map in the sands of how to get to the killing fields.

My Missions

I am going to continue pushing the integration of Business and Geo-spatial Intelligence.  I continue to champion Tableau, and I’m starting to integrate not just with ESRI, Mapbox, Geoserver etc but also with other XML/JSON based APIs, javascript libraries such as Cesium, d3, & three.js.

This is to support the next mission though - full sensory data exploration.   I’ll go further in depth at a later date; explain though some technologies, where I’ve got too so far, and why.

Nov. 27, 1969: Youngsters found a good spot to view the Macy’s Thanksgiving Day Parade, from atop a telephone booth. “It was a day of abundance, not only of foods and goods, but of emotions also,” reported The Times. “Five raggedy children entered the Salvation Army cafeteria at 535 West 48th Street and, sitting among the weary old men and women who traditionally feast with the army on this holiday, received heaps of turkey and potatoes, peas and pumpkin pie.” Elsewhere, some G.I.s boycotted turkey dinner, to protest the American war in Vietnam, as did the Vegetarian Society of New York, to protest the exploitation of animals. Photo: Neal Boenzi/The New York Times