The Story of AI Winters and What it Teaches Us Today (History of LLMs. Bonus)

Introduction

Branching out from our series about the history of LLMs, today, we want to tell you the fascinating story of "AI winters" – periods of reduced funding and interest in AI research. You will see how excitement and disappointment keep taking turns, but important research always perseveres. Join us as we explore the evolving nature of artificial intelligence on this most comprehensive timeline of AI winters. (If you don’t have time now, make sure to save the article for later! It’s worth the read with a few lessons to learn).

Good that it’s summer because we are diving in:

1. Winter#1, 1966: Machine Translation Failure

2. Winter#2, 1969: Connectionists and Eclipse of Neural Network Research

3. Winter#3, 1974: Communication Gap Between AI Researchers and Their Sponsors

4. Winter#4, 1987: Collapse of the LISP machine market

5. Winter#5, 1988: No High-Level Machine Intelligence – No Money

Winter#1, 1966: Machine Translation

As discussed in the first edition of this series, NLP research has its roots in the early 1930s and begins its existence with the work on machine translation (MT). However, significant advancements and applications began to emerge after the publication of Warren Weaver's influential memorandum in 1949.

The memorandum generated great excitement within the research community. In the following years, notable events unfolded: IBM embarked on the development of the first machine, MIT appointed its first full-time professor in machine translation, and several conferences dedicated to MT took place. The culmination came with the public demonstration of the IBM-Georgetown machine, which garnered widespread attention in respected newspapers in 1954.

Another factor that propelled the field of mechanical translation was the interest shown by the Central Intelligence Agency (CIA). During that period, the CIA firmly believed in the importance of developing machine translation capabilities and supported such initiatives. They also recognized that this program had implications that extended beyond the interests of the CIA and the intelligence community.

Skeptics

Just like all AI booms that have been followed by desperate AI winters, the media tended to exaggerate the significance of these developments. Headlines about the IBM-Georgetown experiment proclaimed phrases like "Electronic brain translates Russian," "The bilingual machine," "Robot brain translates Russian into King's English," and "Polyglot brainchild." However, the actual demonstration involved the translation of a curated set of only 49 Russian sentences into English, with the machine's vocabulary limited to just 250 words. To put things into perspective, this study found that humans need a vocabulary of around 8,000 to 9,000-word families to comprehend written texts with 98% accuracy.

This demonstration created quite a sensation. However, there were also skeptics, such as Professor Norbert Wiener, who is considered one of the early pioneers in laying the theoretical groundwork for AI research. Even before the publication of Weaver's memorandum and certainly, before the demonstration, Wiener expressed his doubts in a letter to Weaver in 1947, stating: