[Fixed] OpenAI Playground Speed Differences – GPT-3.5 API 30x Slower Than ChatGPT? : Chris

Reader
October 16, 2023
[Fixed] OpenAI Playground Speed Differences – GPT-3.5 API 30x Slower Than ChatGPT?
by: Chris
blow post content copied from  Be on the Right Side of Change
click here to view original post

5/5 - (1 vote)

Problem Formulation

I encountered a significant delay in the response times on OpenAI’s GPT-3.5 Turbo API as compared to the ChatGPT — and I’m not alone with this (source). The latency is particularly noticeable when using certain accounts, with the delay making the API almost unusable for real-time applications.

Solution Idea: Experiment with Difference User Accounts to Prevent Throttling

When researching possible solutions to this problem, I discovered that a user experienced 30x performance differences across two different OpenAI API accounts’s playgrounds (source).

Here’s what happens if you ask for a full dictionary of a certain word using two different accounts from the same user at the same time (left fast, right slow):

This suggests that the issue might be account-specific or related to allocating server resources per account. It is likely that OpenAI experiments with various throttling ideas to improve the efficiency of resource allocation — we all have experienced times when demand for OpenAI’s models was too much to handle.

Prompt Engineering with Python and OpenAI

You can check out the whole course on OpenAI Prompt Engineering using Python on the Finxter academy. We cover topics such as:

  • Embeddings
  • Semantic search
  • Web scraping
  • Query embeddings
  • Movie recommendation
  • Sentiment analysis

👨‍💻 Academy: Prompt Engineering with Python and OpenAI

The post [Fixed] OpenAI Playground Speed Differences – GPT-3.5 API 30x Slower Than ChatGPT? appeared first on Be on the Right Side of Change.


October 16, 2023 at 04:22PM
Click here for more details...

=============================
The original post is available in Be on the Right Side of Change by Chris
this post has been published as it is through automation. Automation script brings all the top bloggers post under a single umbrella.
The purpose of this blog, Follow the top Salesforce bloggers and collect all blogs in a single place through automation.
============================

Salesforce