Reasoning Is All You Need

By: G.R.

Introduction

This paper explores a novel approach to finetuning AI Language Models (LLMs) by emphasizing the crucial aspect of 'Reasoning.' While existing LLMs excel in generating text, they often fall short in tasks involving mathematics, logic, or complexity. The primary objective of integrating reasoning capabilities is to enhance LLM performance in these specific domains.

Traditionally, Chat Completion relies on the input template of System, Prompt, and Response. This paper introduces an innovative template for Chat Completion: System, Prompt, Reasoning, and Response. This modification encourages the model to engage in thoughtful deliberation before generating a response.

The advantages of this strategic shift are manifold and continuously evolving with the integration of new datasets. Some notable benefits include:

  1. The quality of the user's prompt no longer significantly influences the final response.
  2. Enhanced agentic behavior of the model.
  3. Substantial alignment with user input.
  4. Improved reasoning capabilities.
  5. More context provided in each response.

Example Output

Generated by the model ArtificialThinker-Phi2

Prompt: What are the last 10 digits of PI ? (provide an ironic/sarcastic answer)

Reasoning: Since π is an irrational number that continues indefinitely without repeating its digits, it's not possible to provide the exact last ten digits. However, for the sake of amusement and irony, I'll give you a random set of digits. This response adds a touch of humor while acknowledging the limitations of computational accuracy when dealing with infinite numbers like π.

Response: The last ten digits of π are … ¯_(ツ)_/¯

Exploring the Architecture

To endow a model with reasoning capabilities, one simply needs to utilize a dataset where reasoning precedes the response, coupled with an appropriate input template. Experimental validation of this architecture involved fine-tuning Phi-2, a Microsoft model with 3 billion parameters, using a small dataset (100KB) comprising samples generated by GPT3.5 and GPT4. The resulting model, ArtificialThinker-Phi2, and the associated dataset, ArtificialThinkerSet, aim to demonstrate the potential of this approach. It is anticipated that future releases will present even more advanced models and datasets.

Conclusion

We invite readers to contribute to this initiative by sharing this paper, providing feedback, sharing thoughts, and offering ideas within our community. Additionally, those interested can support us through donations.

← Back to freecs.org