Postmodernity: Where has Meaning & Purpose Gone? Perspectives in AI: From LLMs to Reasoning

Postmodernity: Where has Meaning & Purpose Gone? Perspectives in AI: From LLMs to Reasoning

In today's society, we are experiencing a cultural collapse that has led to an "inward homelessness". This collapse has caused us to lose meaning and purpose in our lives. Culture plays a significant role in providing meaning and purpose, as it passes down traditions and beliefs from one generation to the next. When a society fails to pass on its culture, it is essentially saying that its foundational principles are no longer valid. This rejection of culture results in a diminishing of our identity.

One of the reasons for this cultural collapse is the rejection of religious stories and beliefs. In our search for a new identity, we have disregarded the spiritual and faith-based aspects of our culture. By rejecting the stories of creation, King David, and the Christ, we have failed to pass on the cultural elements that gave us meaning and purpose. As a result, our identity has suffered.

On the other hand, in the field of artificial intelligence, there have been significant advancements in adapting large, pre-trained models to specific tasks or domains without extensive retraining. One such method is Low Rank Adaptation (LoRA). LoRA allows for the integration of domain-specific knowledge into a larger model, enabling it to understand and process information within a particular field. This adaptation is achieved without altering the core model or requiring extensive retraining.

The concept behind LoRA is to have a smaller module that contains enough domain-specific information, which can be appended to the larger model. This smaller module acts as an auxiliary component, adjusting the characteristics of the model without the need for rebuilding or retraining. By leveraging the mathematical concept of low rank approximation, LoRA creates a smaller, adaptable module that customizes the larger model for a specific task.

The implementation of LoRA has led to impressive efficiencies in the field of AI. By fine-tuning and adapting the models, researchers have been able to cut resource usage and training costs significantly. For example, a 175 billion parameter model was successfully fine-tuned and adapted using just 24 V100s. Additionally, the reduction in checkpoint sizes from 1 TB to 200 megabytes has opened up innovative engineering approaches, such as caching in VRAM or RAM and swapping them on demand. These advancements have improved user experience and reduced storage costs by a factor of 1000 to 5000.

So, what can we learn from these two seemingly unrelated topics? Firstly, the loss of meaning and purpose in society can be attributed to the rejection of cultural traditions and beliefs. By disregarding our heritage, we are diminishing our identity and leaving ourselves feeling lost. Secondly, in the field of AI, the adaptation of models through methods like LoRA has led to significant advancements and efficiencies. These adaptations have allowed for faster and more cost-effective training, as well as improved user experiences.