Our Blogs

Effectively benchmarking OCI Compute Shapes for LLM inference serving

IntroductionThe rapid evolution of generative AI models is nothing short of remarkable, with new and more powerful models emerging on an almost weekly basis. Since the release of OpenAI’s GPT-2 in 2019, the landscape has exploded, with more than 50 models introduced to date (Figure 1). While some models continue to push boundaries in terms...