Deep Learning and Symbol Manipulation: The Path to Human-Like AI


Hatched by Glasp

Jul 13, 2023

4 min read


Deep Learning and Symbol Manipulation: The Path to Human-Like AI

In the world of artificial intelligence (AI), there has been a longstanding debate about the best approach to building intelligent systems. Should AI be built on symbol manipulation, which treats thinking as algebraic processes? Or should it be built on neural networks, which mimic brain-like systems? This debate has been ongoing for nearly 70 years, with proponents on both sides arguing for the superiority of their approach.

However, there is a growing consensus among experts that a hybrid model, combining the strengths of both symbol manipulation and neural networks, may be the key to achieving human-like AI. This middle ground approach, known as hybrid models, aims to integrate the data-driven learning of neural networks with the powerful abstraction capacities of symbol manipulation.

One of the main challenges in building hybrid models is understanding the basis that allows a system to learn symbolic abstractions. It is believed that either symbol manipulation itself is innate or that there is something else, yet to be discovered, that indirectly enables the acquisition of symbol manipulation. Discovering this basis is crucial in order to leverage the world's knowledge and build AI systems that are safe, trustworthy, and interpretable.

Early pioneers in AI, such as Marvin Minsky and John McCarthy, believed that symbol manipulation was the only way forward. On the other hand, Frank Rosenblatt, a neural network pioneer, argued that AI should be built on neuron-like nodes that process numeric inputs. However, it has been known for quite some time that these two approaches are not mutually exclusive.

The key question now is whether symbol manipulation can be learned rather than built in from the start. The answer is a resounding yes. Symbol manipulation is indeed learnable, although it has been shown that certain systems may fail to acquire aspects of symbol manipulation. Therefore, there is no guarantee that any system, regardless of its constitution, will be able to learn symbol manipulation. However, with continued research and experimentation, it is possible to overcome these challenges.

Deep learning, which has been at the forefront of AI in recent years, is not without its limitations. It faces challenges related to compositionality, systematicity, and language understanding. These challenges revolve around generalization and distribution shift, which are areas where current neural networks struggle. When it comes to natural language, compositionality, and reasoning, deep learning systems often fail to reliably extract symbolic operations, even with large amounts of data and training.

To overcome these challenges and create more powerful AI, a combination of data-driven learning and abstract, symbolic representations is needed. This requires finding the right balance and harmony between the two approaches. By integrating the strengths of deep learning and symbol manipulation, it is possible to create AI systems that can generalize complex aspects of natural language and reasoning.

In conclusion, the path to human-like AI lies in the integration of deep learning and symbol manipulation. By combining the data-driven learning of neural networks with the abstraction capacities of symbol manipulation, we can create more powerful and reliable AI systems. However, this requires a deep understanding of the basis that allows for the learning of symbolic abstractions. Here are three actionable pieces of advice to move forward:

  • 1. Continue exploring hybrid models: Researchers should focus on finding innovative ways to combine symbols and neural networks, such as extracting symbolic rules from neural networks or restructuring neural networks themselves.
  • 2. Bridge the gap with atomic essays: Writers can use atomic essays, quick 250-300 word essays published on platforms like Twitter, to validate their ideas and explore them further. This incremental approach helps in creating consistent content and understanding what resonates with the audience.
  • 3. Embrace the iterative process: Rather than putting all the effort into one piece of content, embrace the iterative process of content creation. Start with ideas as tweets, turn them into essays, and then expand them into articles. Each level provides valuable feedback and validation for the idea.

By following these steps, we can make progress towards building AI systems that are safe, trustworthy, and interpretable. The combination of deep learning and symbol manipulation holds great potential for achieving human-like AI, and understanding the interplay between the two is key to unlocking that potential.

Hatch New Ideas with Glasp AI 🐣

Glasp AI allows you to hatch new ideas based on your curated content. Let's curate and create with Glasp AI :)