Puddles in Australia

Nathan Yau of FLOWINGDATA provides us with the "Best Data Visualization Projects of 2019".

Check out his work, it is awesome.

Visualization is a relatively new field, but we seem to be developing an understanding of what it is and what it can be used for. This year, we refined existing methods. With less emphasis on novelty of visual forms, we focused more on what we wanted to communicate.

He does prove his first sentence wrong with his first example, "Best Blend of New and Vintage" with the 3D elevation + 1878 USGS Yellowstone Geology Map.  

Here's a map from 1878 of my ancestors near Buck Lake. (Not in 3D)

1878 Map Of Loughborough

Notice the visualizations of lot numbers, landowner names, all nicely laid out in somewhat-to-scale quadrants of land ownership. 

The field of Visualization is as old as cave drawings, or pictographs, and carvings, or petroglyphs.  Take Milestones in the History of Thematic Cartography, Statistical Graphics, and Data Visualization for example.  If you type that title out you will probably get carpal tunnel syndrome.  It could probably be better described with a pictogram or symbol.

eye with apple slicer, 2020, some pictogram artist

The earliest known map, Çatalhöyük, which of course is disputed, is a Babylonian clay tablet with a nice 3D visualization.  And it isn't even a map.  My Cartocacoethes is kicking in again.

RIP Australian puddle map, is this Sudan or Australia?

Speaking of Australian puddle maps, another example on the FlowingData web site is Climate Coverage by The New York Times.  With augmented reality view.  I suppose you can make an example that this is a visualization which could only have happened in the last decade?  Searching for 16th century augmented reality, Google comes through again with An Eerie Augmented Reality Illusion from the 1850s is Still Being Used Today.  

Smoke and mirrors, still in use today by climate change deniers.  Mostly the smoke part.

I really like the Best Comic Chart winner, Something's Wrong.  

Fund him on Patreon.com/WillikinWolf

The CPI should have been added to this chart.  

Since all things revert to the mean I would expect that productivity will start going down and level out with wages once we get our robot overlords in place. Patreon has a great example of the job titles of the future, which of course we are now living in.

Podcaster

Video Creator

Musician

Visual Artist

Writer & Journalist

Community Coordinator

Gaming Creator

Nonprofit

Tutor and Educator

Creator-of-all-kinds

My 7-year-old asked me to setup his web site, which really means Youtube Channel to him.  Unlike the boomers who think that the internet is Internet Explorer, and a Relational Database is something hosted inside Internet Explorer, this generation will think that old-school television is Netflix, and the internet is some kind of I Can Has CHEEZburger meme browser running in your television next to Netflix.

Eventually this will revert to the mean and tools like IRC and Gopher will come back to us.  

What the crap? There's a Gophercon?  That would explain all the ASCII & ANSI art generators which are getting more popular.

I can't wait for squealie modems to come back to us.

I expect the 2020 visualization winner to be some ANSI terminal art depicting the trend of vandalizing web sites by setting their baud rates to 2400bps.

Melting Columns to Rows in R

Pivoting data is a common issue when dealing with Excel or CSV input files.  Excel users generally like "dirty" yet presentable data.  Data dumps commonly place multiple tables worth of data into a single massive column table.  This style of multi-column data lends it self to problems when trying to analyze results using tools such as pivot tables.

For example, 12 months of column data should be converted to 2 columns (month name, value) for efficiently pivoting / filtering and sorting data from a Wide to Long format.

Excel can use some macro functionality to unpivot results, or an add-in like PrepYourData or Microsoft PowerQuery.
 
R provides this kind of functionality in 3 lines of code. In this case for subjects and scores across column to rows.

http://stackoverflow.com/questions/18446668/using-r-to-reformat-data-from-cross-tab-to-one-datum-per-line-format

http://seananderson.ca/2013/10/19/reshape.html

Tidy Data

Lifecycle of Data Science Project

The Lifecycle Summarized

1. Identify & Define the problem
2. Define and document data sources
3. Statistical data profiling
4. Implementation
5. Sharing and collaboration
6. Maintenance & Support

http://www.datasciencecentral.com/profiles/blogs/life-cycle-of-data-science-projects

Plus M&Ms, Jackknife (Swiss Army Style?) logistic and linear regression.
http://www.datasciencecentral.com/profiles/blogs/jackknife-logistic-and-linear-regression

Plus jackknifing your results in @ 4 lines of R code.
http://ryouready.wordpress.com/2008/12/19/r-jackknife-the-coefficients-of-a-linear-regression-model/

Random Forests in Tableau with R
http://boraberan.wordpress.com/2014/02/07/decision-trees-in-tableau-using-r/

Finally, using a jackknife to cut down some Hidden Decision Trees.
http://www.datasciencecentral.com/profiles/blogs/hidden-decision-trees-revisited

Syntactically Awesome Style Sheets

No, it's not Software-As-A-Service.  SASS is a CSS extension language that makes CSS coding more efficient and readable.  It compiles into standard CSS markup.

http://sass-lang.com/guide

Demos
http://cssdeck.com/labs/sass-demo
https://github.com/kianoshp/SASS-Intro-Demo

Playground
http://sassmeister.com/

Training
http://pluralsight.com/training/Courses/TableOfContents/better-css

Frameworks
http://gumbyframework.com/
http://compass-style.org/
http://bourbon.io/
http://sass.fffunction.co/

FRED Add-In for Excel and some Torontoist Centric Economic Data

The Federal Reserve Bank of St. Louis Economic Data (FRED) Add-In is free software that will significantly reduce the amount of time spent collecting and organizing macroeconomic data. The FRED add-in provides free access to over 210,000 data series from various sources (e.g., BEA, BLS, Census, and OECD) directly through Microsoft Excel.

Get it here
http://research.stlouisfed.org/fred-addin/

Are you looking for GDP, CPI, or microeconomic data from the US FED?  Stats Canada?

Some interesting visualizations and interpretations of this type of data.

How much you make vs. how much it really feels like per US city with the supporting data released April 2014.

Canadian Cities Where An Average Income Will No Longer Buy You a House 

Numbeo, Cost of Living In Canada

It will cost you about 6% more to eat at McDonalds in Barrie vs. Toronto.
It costs 90% more to buy a bag of potatoes in Oshawa than Toronto.

There must be a potato famine in Oshawa... or people there really like french fries.

Speedometer Design: Why It Works | DATA + DESIGN by Paul Van Slembrouck

Gauge controls may have some comfortable familiarity with certain business users (think oil & gas or automotive/transportation). 

However, do they convey business information in a quick, actionable way?
Speedometer Design: Why It Works | DATA + DESIGN by Paul Van Slembrouck

HTML5 visualizations

Some HTML 5 visualizations

A spinnable flare graph
http://mbostock.github.com/d3/talk/20111116/bundle.html

A tree map
http://mbostock.github.com/d3/talk/20111018/treemap.html

A map of US
http://mbostock.github.com/d3/talk/20111116/airports.html

LobsterPot provides an open-source version of MS Pivot Viewer
http://netflixpivot.cloudapp.net/

2011年の日本の地震 分布図 Japan earthquakes 2011 Visualization map (2012-01-01) - YouTube

It isn’t so much the visualization as it is the sound that will blow you away. From the constant frame of reference in the top left clock / calendar, to the scrolling detail data, this has all the elements of a perfect visualization.  It would be really cool to see twitter feeds and news headings on this too.

SlideRocket Products - The Best Online Presentation Software.

Starting at.... free.... some incredible presentation software.

SlideRocket is a revolutionary new approach to presentations. It's the only presentation software that allows you to create stunning presentations, manage them intelligently, share them securely and then measure the results. SlideRocket also includes an online marketplace where you can find all the content and services you might need to make your presentations great. SlideRocket goes beyond traditional presentation tools by harnessing the power of the Internet and making everything available to you in an integrated and intuitive online interface. SlideRocket is provided in a software-as-a-service model in a variety of price points starting at free.

SlideRocket Products - The Best Online Presentation Software.

Temporary Post Used For Theme Detection (78a62484-9bbd-44e9-9fdc-bccaf4fee587 - 3bfe001a-32de-4114-a6b4-4005b770f6d7)

This is a temporary post that was not deleted. Please delete this manually. (77266b4e-d6c6-442b-b65f-379a1d331ccd - 3bfe001a-32de-4114-a6b4-4005b770f6d7)

The Data Puzzle : A PerformancePoint Server 2007 Wordle word cloud

Tag Cloud in Java.

Wordle is a lot of fun to use if you’re a data visualization geek like I am. I took Gartner’s October 2007 report on PerformancePoint Server 2007, selected the Key Points, and copied and pasted it into the Wordle Web page.

The Data Puzzle : A PerformancePoint Server 2007 Wordle word cloud

Global distribution of the world's water

This is one striking example of why the polar caps are the most important feature of earth.

This shows us why Norway ends up with all of the garbage that falls off container ships.

The black areas really cut a swath through key parts of the world.

But why Saskatchewan?

 

The whole report is here.

http://www.unep.org/geo/geo4/media/

YouTube - Twenty-First Century Waterfall

America consumes 31.2 billion water bottles a year taking 17.6 million barrels of oil to create them. Enough to fuel 1.5 million cars for a year. America consumes 100 billion plastic bags a year taking 12 million barrels of oil. World figures? 154.3 billion water bottles and 500 billion - 1 trillion bags=147.4 - 207.4 million barrels of oil a year.

So how come there isn't a Mr. Fusion Home Energy Reactor yet?  I keep tossing my car water into the backseat when it should be going into the engine.

Subprime Mortgage in Technicolor

 

 

http://graphics8.nytimes.com/images/2008/04/05/business/20080406_METRICS_SUB_GRAPHI.jpg

Demos | Lab49 Visualizations

Lots of finserv-based visualization demos here...

Demos

Lab49 continuously develops applications that demonstrate the latest technologies and how they can be applied in the next generation trading room.

Market Liquidity Analysis Visualization

Lab49 has built a visualization framework that integrates with Aleri's Market Liquidity Analysis (MLA) engine. MLA allows brokers and traders to optimize results across competing exchanges while the Lab49 framework provides a rich visual framework to gain insight into market dynamics. Additionally, the user interface enables users to consolidate trade reports, quotes and order books across multiple exchanges in real-time, creating a single virtual order book that represents the liquidity of the entire visible market. Consolidated last price and best bid/offer record for each security can also be generated.

Demos | Lab49

Panopticon: Demo Gallery

Interesting Java-based visualization tools for stock markets, risk analysis, and the champagne market.

Our visualization software is easy to use and is a great way to explore large datasets, identify outliers and find hidden patterns. As one customer at a major investment house put it, "They help you answer questions you didn't know you had." Discover it for yourself with these online demos.

Panopticon: Demo Gallery

Fold-Ins, Past and Present - The New York Times

If you're a child of the 70s or early 80s, perhaps you had a collection of Mad magazines?  Most likely the backs of those magazines had a couple of creases?

This is quite possibly one of the the best uses of the internet I have seen in at least a year.  Fold-ins from Mad, 1960s to present, done in Flash.

Fold-Ins, Past and Present - The New York Times

Amazon.com: The Back of the Napkin: Solving Problems and Selling Ideas with Pictures: Dan Roam: Books

This was one presenter at Mix '08 I wish I had seen.

The Back of the Napkin: Solving Problems and Selling Ideas with Pictures (Hardcover)

Amazon.com: The Back of the Napkin: Solving Problems and Selling Ideas with Pictures: Dan Roam: Books

World domination and flow charting

One of my favourite games is C&C Red Alert. You can win a free copy of the newest game from the C&C franchise by plotting your master plan for World Domination in flow chart form.

I wonder if my Visio certification prepared me enough for this one...

Villians are destroyers of worlds, enemy of good and source of wickedness. The infamous Kane of the Command & Conquer universe is an excellent example: he's fought the GDI time and time again and helped to spread Tiberium like a cancerous Johnny Appleseed.

But how did he do all this? It's not as if you can just wake up one morning and decide that you're just going to destroy multinational security organizations on a whim. You need a strategy, a guide, a... plan, if you will.

With that in mind, if you had the resources, what would be your great villainous plan? Electronic Arts and Shacknews want to know.

Contest: Command & Conquer 3 Villianous Aptitude - Shacknews - PC Games, PlayStation, Xbox 360 and Wii video game news, previews and downloads

Useful Data Sets

Before data visualization comes data... and here's a lot of data.

The Data Page

This page provides information on how to collect data from a variety of sources. it also allows you to look at and download a number of data sets that you might find useful in corporate finance and valuation. If you have trouble viewing the data in your browser, you can download the data in excel format. I will try to keep them updated, and include the dates of the updates with the data sets. If you want to see the company-specific information, you can download the data on individual firms below.

Recent additions: While most of the data sets here are based upon U.S. firms, I have added updates for Europe (Western Europe and the UK), Japan and for Emerging Markets (Latin America, Asia and Eastern Europe) for a few data items - betas and price multiples. Any company with a market capitalization greater that $ 50 million in US dollar terms should be in the datasets; for the non-US companies, any currency data will be in the local currency unless otherwise specified. You can download the data in excel files but the html files will contain only the US data. While I will update the US data every six months or so, I will update the non-US data once a year. I have also added the archived files from previous years for those of you who want historical data. The data is as of the end of each year.

Useful Data Sets