Embedding models have become an important part of LLM applications, enabling tasks such as measuring ... However, embedding models are mostly based on a transformer architecture that is different from the one ... This makes it difficult to transfer the massive work being done on generative models to improve embedding

This cost only grows as LLMs scale to larger embedding dimensions and context lengths.