Bitget App
Trade smarter
Buy cryptoMarketsTradeFuturesBotsEarnCopy
Chainbase unveils open source of AI language model focusing on crypto

Chainbase unveils open source of AI language model focusing on crypto

CryptopolitanCryptopolitan2024/10/11 18:09
By:By Vignesh Karunanidhi

Share link:In this post: Chainbase releases open-source AI model Theia-Llama-3.1-8B. The model is trained on comprehensive crypto-oriented dataset. Benchmark results show Theia outperforming mainstream models.

Chainbase has released the open-source AI model, Theia-Llama-3.1-8B. It is a language model that focuses on crypto.

The company had launched an alpha version of the chatbot called TheiaChat in August. It was released at the time to disclose the features of Theia.

Theia training was drawn from two sources

The data used to train the model was taken from CoinMarketCap and other research reports. The data of CoinMarketCap used to train and fine-tune Theia-Llama-3.1-8B includes project documents like whitepapers, official blog posts, and news articles.

The research reports were obtained from credible online sources to provide in-depth insights into the project’s fundamentals, market influence, and development progress.

The blog post further details that the data from these two primary sources also went through manual and algorithmic filtering to reduce redundancy and eliminate errors.

Chainbase also used sophisticated techniques in fine-tuning and optimizing the model. The team used LoRA (Low-Rank Adaptation) for efficient fine-tuning. This helped in adapting the base Llama-3.1-8B-Instruct model to the cryptocurrency domain.

The training process was enhanced using LLaMA Factory and DeepSpeed, incorporating advanced techniques like ZeRO, offload, sparse attention, 1-bit Adam, and pipeline parallelism to speed up training and reduce memory usage.

In addition to fine-tuning, Chainbase optimized the model to prepare it for efficient deployment. This quantization process reduces the model’s memory footprint and speeds up inference while maintaining acceptable accuracy.

See also AI boom sends Nvidia up 25% in a month

Chainbase proposed a crypto AI model benchmark

To evaluate the performance of Theia-Llama-3.1-8B, Chainbase proposed a benchmark for crypto AI models.

The benchmark evaluates models across seven dimensions, including crypto knowledge comprehension and generation, knowledge coverage, and reasoning capabilities.

Initial benchmark results focusing on understanding and generation capabilities in the crypto domain show Theia-Llama-3.1-8B outperforming 11 other LLMs. This includes popular models from OpenAI, Google, Meta, Qwen, and DeepSeek. The model achieved a perplexity score of 1.184 and a BERT score of 0.861, surpassing mainstream models currently on the market.

Chainbase also stated in their blog post that the performance of Theia-Llama-3.1-8B exceeds that of mainstream models currently available on the market. “Next, we will build larger models and evaluate more dimensions of the models,” Chainbase stated.

0

Disclaimer: The content of this article solely reflects the author's opinion and does not represent the platform in any capacity. This article is not intended to serve as a reference for making investment decisions.

PoolX: Locked for new tokens.
APR up to 10%. Always on, always get airdrop.
Lock now!

You may also like

Strategy launches STRD, new perpetual preferred stock via IPO

Share link:In this post: Strategy announced the launch of Series A preferred shares Stride (STRD) with a non-mandatory, non-cumulative 10% dividend. The new asset will be partially used for additional BTC purchases. The proliferation of new share types is stirring worries of increasing risk, especially during unfavorable market periods.

Cryptopolitan2025/06/03 15:48

Activision removes ads from Call of Duty: Black Ops 6 and Warzone after community backlash

Share link:In this post: Activision has removed ads from Call of Duty: Black Ops 6 and Warzone Loadouts after facing criticism from the gaming community. Gamers had initially criticized having ads in the game, citing that ads were opening it up to a mobile game. Activision, in its response, argued that it was just a feature test that was published in error.

Cryptopolitan2025/06/03 15:48
Activision removes ads from Call of Duty: Black Ops 6 and Warzone after community backlash

Constellation Energy Stock skyrockets as Meta embrace it’s nuclear power

Share link:In this post: Meta signed a 20-year deal with Constellation to purchase nuclear power from June 2027. The tech giant will buy roughly 1.1 gigawatts of power from Constellation’s Clinton Clean Energy Center in Illinois. Major tech companies signed a pledge in March led by the World Nuclear Association calling for nuclear energy worldwide to triple by 2050.

Cryptopolitan2025/06/03 15:48
Constellation Energy Stock skyrockets as Meta embrace it’s nuclear power