Artificial intelligence (AI) specialist Acryl (CEO Park Oe-jin) announced on the 1st that its large-scale language model (LLM) 'Jonathan Allm' ranked first in the open source category on the 'Tiger Leaderboard' operated by Weight & Bias (W&B).
The Tiger Leaderboard is a platform that evaluates the performance of Korean LLM models, and was launched in April. It evaluates language understanding and language generation capabilities from various angles and discloses the results.
Jonathan Allm, developed by Acryl, received high scores in language understanding and generation capabilities, ranking third overall with an average score of 0.6675.
Considering that the first-place Antropic 'Claude 3 Opus (0.7542)' and the second-place OpenAI 'GPT-4 (0.7363)' are closed-source models, it ranked first among open sources.
In particular, it surpassed strong models such as Google's 'Gemini Pro (0.6645)' in 4th place and Mistral's 'Large (0.6259)' in 5th place.
Acryl said, "We were able to achieve this result through joint research with Professor Woo Hong-wook's research team at Sungkyunkwan University," and "This research was conducted using the Korean dataset that Acryl collected and developed, and achieved performance that surpassed large models with a small 8B model."
They also emphasized that they achieved optimal performance by conducting fine-tuning with the Jonathan platform. They added that Acryl's LLMops is optimized for fine-tuning and augmented search generation (RAG), and that it enables fast learning of LLM by combining it with the distributed machine learning platform Jonathan.
Meanwhile, many high-performance models released in the past 1-2 months have not yet been registered on the Tiger Leaderboard. 'Claude 3.5 Sonnet', 'GPT-4o', 'Rama 3.1', 'Mistral Large 2', 'Gemini 1.5 Flash', etc. are results that have not yet been reflected in the leaderboard.
However, since the recent major reorganization of the Hugging Face LLM leaderboard, this is the first time that a domestic model has stood out in the global rankings.
Park Oe-jin, CEO of Acryl, said, "This first place on the Tiger Leaderboard signifies a leap forward in Korean LLM technology, and we will continue to provide improved performance through continuous research and development in the future." He also said, "At the same time, as a result that proves the LLM Ops technology, we will continue to present innovative AI models through research and development in various fields."