Falcon LLM

Falcon LLM is a foundational language model with two versions: Falcon-40B with 40 billion parameters and trained on one trillion tokens, and Falcon-7B with 7 billion parameters and trained on 1.5 billion tokens. Both models are autoregressive decoder-only, which means that the model is trained to predict the next token in a sequence given the previous tokens. The GPT model is a good example of this.

Falcon’s architecture has been proven to be more efficient than GPT-3 in terms of the training compute budget, as well as requiring less compute at inference time. It was created using specialized tools and includes a unique data pipeline that filters and deduplicates content from the web.

In addition, there are two ready-to-use chat models available, Falcon-40B-Instruct and Falcon-7B-Instruct.

Sources:

• https://www.kdnuggets.com/2023/06/falcon-llm-new-king-llms.html
• https://www.packtpub.com/article-hub/falcon-llm-the-dark-horse-in-open-source-llm-race