Professor P.C. Mahalanobis, who introduced statistics to India, is a scientist whose absence is felt dearly even today. Mahalanobis’s lifelong courtship with statistics, his unwavering and fearless leadership to advance a statistics and survey culture in India, the founding of the Indian Statistical Institute — “a mighty monument of his handicraft” — and his nurturing of a generation of outstanding academicians have all left behind an enduring legacy. There is little doubt that India was in the Mahalanobis-era nearly five decades prior to his passing on June 28, 1972. Today, in the midst of the shifting socio-economic dynamics in post-pandemic India, he is greatly missed.
The age of Big Data
Over the past 20 years, there has been a global shift in both the nature of data and statistics. With the advent of the Internet and virtually everything confined to the Internet of Things, there has been a flood of data, most of it junk. We now have much more data than what available technology can leverage. This is widely perceived as the era of Big Data.
Another significant yet related issue is how artificial intelligence (AI) is transforming our lives and lifestyles. The state of society is precarious. One can wonder how Mahalanobis, a statistical doyen and a key figure in the early development of Indian democracy, would have responded to the Big Data-related craziness and the AI-driven revolution. Though speculative, the answer would be based on his legacy.
Comment | There’s no one to fill Mahalanobis’s shoes
Historically, data often appears to be Big when the available technology at that time fails to analyse it. Mahalanobis also encountered a Big Data problem when his large-scale surveys yielded lots of data that needed to be looked into for effective planning. How did he respond to this? Well, to handle tons of data and tackle the complex mathematical calculations, Mahalanobis persuaded the government and succeeded in procuring the first two digital computers of the country (and South Asia, too) at his Indian Statistical Institute, in 1956 and 1958, and thus ushered in the age of computers in India. It was indeed a remarkable accomplishment by a statistician.
Problems during COVID-19
Mahalanobis was “a physicist by training, a statistician by instinct and an economist by conviction”. He had an uncanny knack for embracing technology for human welfare, perhaps as a result of his background in physics. He even built some simple machines to facilitate his surveys and measurements. Thus, one may safely perceive that Mahalanobis would have embraced the power of AI in enhancing human productivity, such as Big Data analyses, and perhaps in a way that is far more effective than how AI is currently applied to that goal.
He could possibly be able to lead Big Data analyses considerably better than anybody else, even in the absence of AI, as no one is likely to comprehend survey dynamics or the heartbeat of data as he did. One recent Big Data foible, for instance, involved numerous contradictory projections during the COVID-19 era. Different sorts of economic losses due to the pandemic needed to be properly evaluated for a balanced recovery. One could argue that if Mahalanobis was alive today, the country’s COVID-19 response could have been much stronger. If he was in the lead, our data might be beyond question, and the analyses might be far more accurate. And India’s “Plan Man” could be the best person for planning to build optimal health-related infrastructures for combating future disasters.
AI regulation
Around seven decades ago, from the perspective of the newly independent nation, planning — with the aid of extensive technocratic exercises with democratic participation — moved from the realm of politics, primarily due to Mahalanobis. Now, we are at the crossroads. India’s upcoming census will be a digital exercise. The dynamics of other surveys are also bound to change in the new normal setup. That is how statistics is evolving. We would miss the leadership of an expert such as Mahalanobis from this changed statistical perspective.
Ah, AI. As AI is threatening to replace millions of jobs without creating alternatives and is also aiding in spreading disinformation, there is a substantial global attempt to clip its wings. Never easy, though. Mahalanobis, who was deeply inspired by Kautilya’s Arthashastra, successfully introduced the revolutionary concept of built-in cross-checks into his surveys. As the world struggles to regulate AI, could Mahalanobis, with his statistical “instinct,” also be instrumental in regulating AI? Maybe.
Mahalanobis envisioned statistics as “a new technology for increasing the efficiency of human effort in the wildest sense”. Even today, someone like Mahalanobis, with an uncanny knack for perfection, tireless dedication, brilliant leadership, and who could understand the “dance steps of numbers in the arena of time and space” better than anybody else engaged in this business, could be the best person to handle tons of data, with its ever-expanding nature, and also embrace the benefits of technologies for human welfare and national development.
Atanu Biswas is Professor of Statistics, Indian Statistical Institute, Kolkata
