MLCommons, a collaborative effort of industry and academia, focuses on enhancing AI safety, efficiency, and accountability through rigorous measurement standards like MLPerf. Its AI Safety Working Group, established in late 2023, aims to develop benchmarks for...

With the widespread deployment of large language models (LLMs) for long content generation, there’s a growing need for efficient long-sequence inference support. However, the key-value (KV) cache, crucial for avoiding re-computation, has become a critical bottleneck,...

Meta has officially introduced its new AI assistant, an AI chatbot called Meta AI, powered by Meta’s latest and most capable openly available LLM, Meta Llama 3. Since the big bang in the popularity of AI chatbots with OpenAI’s ChatGPT, almost every major organization...

Deep learning has dramatically transformed industries, from healthcare to autonomous driving. However, these advancements wouldn’t be possible without parallel developments in hardware technology. Let’s explore the evolution of deep learning hardware, focusing on GPUs...

Code generation has emerged as a significant area for evaluating and deploying Large Language Models (LLMs). However, many of the current coding benchmarks, like HumanEval and MBPP, have achieved solution rates above 90% as language models have grown in size and new...

Google AI researchers introduced ScaNN vector search library to address the need of efficient vector similarity search, which is a critical component of many machine learning algorithms. Existing methods for vector similarity calculation work well with small datasets,...