HPSC -- High Performance Statistical Computing for Data Intensive Research

HPSC -- High Performance Statistical Computing for Data Intensive Research

NGSA
pbdR

Section: NGSA

Next Generation Statistical Analytics

This page should be done when I have time. A few ideas are leaved at here and it will happen someday.

``Today's Big Data, Tomorrow's Tiny Toys.''

``Be Bigger, Be Faster, Be Easier, and Be Greener.''

30 years ago, we penciled down $\sum_i x_i$ , $\sum_i y_i$ , and $\sum_i x_iy_i$ , on papers.
20 years ago, we clicked calculators to perform simple linear regressions.
10 years ago, we commanded computers to draw residual plots.
Currently, 2012, we utilize clusters to visualize large datasets.
10 years later, we will tell smart phones to animate MCMC.
20 years later, we may ask ``what is a smart phone?''

My silly dream last night was that "A" was on a special device which is a smart phone and referred to "A.worker" which were workers on clouds where I didn't know. Not necessary on one cloud, but all had R for statistical computing. All run the same code on different data with communication. I said to the phone "R plot A", then I received an animation in 3D performing MCMC simulation.

``All models are wrong but some models are useful.'' -- George Box

How about I say ``All data are wrong but some functions of data are useful.'' ``Is sufficient statistics a function of data?'' How about communicate with ``minimum sufficient statistics''? ``Isn't Statistician doing this for looo...ooong time?''

''Big Dream of Bigger than Big.'' No worry about problems of large memory and out-of-memory. Simply don't load all data in one machine. Ambitiously try smart phone.

No more parallel computing, but ``Read in distributed, Compute in distributed, Statistics in distributed, Output in distributed.''

Useful one-pass algorithm of statistics should be established for on-fly stream data analysis and in-situ experiments.

[ Go to top ]

Created: Oct 19 2011
Last Revised: Feb 13 2013, 12:20 (CDT Ames, IA, USA)
Maintained: Wei-Chen Chen
E-Mail: wccsnow @ gmail.com

Best Resolution
Firefox 3.5
1024x768
small font