Boost Kilo Code: API Rate Limits & User Experience

Enhancing API Rate Limit Handling and User Experience for Kilo Code

Hey Kilo Code team, I'm a huge fan of your VS Code extension! It's a game-changer for code generation and task management. However, my experience with the free tier of the Gemini API has been a bit bumpy, with those frustrating 429 Too Many Requests errors interrupting my workflow. I've got some detailed suggestions to help Kilo Code become even more user-friendly, especially for developers on a budget. Let's dive in!

The Core Problem: Parallel Requests and Rate Limit Limitations

Currently, Kilo Code kicks off multiple API requests simultaneously to analyze my code. This is super efficient for users with paid API keys, but it's a real headache for those of us on the free plan. The existing frequency limit doesn't seem to control these initial parallel requests, leading to rapid quota exhaustion and 429 errors. So, here's how we can fix it:

Suggestion 1: Revamping the Rate Limiting Mechanism

The current frequency limit feels more like a pause between steps, not a true traffic controller. We need a more robust system. Here's how to do it:

1. Implement a Global Request Queue

   • What it is: A central "gateway" within the plugin. All API requests, no matter where they come from, must pass through this queue first.
   • How it works: The queue manager sends requests out sequentially, at intervals based on the user's rate limit settings (e.g., "up to 15 requests per minute"). This completely prevents exceeding the API provider's limits.
   • Benefits: Transforms chaotic parallel requests into orderly, controlled requests, solving the 429 problem.

2. Add a "Max Concurrent Requests" Setting

   • What it is: A setting alongside the frequency limit to control how many requests can run at once.
   • How it works: Users could set it to 1 or 2. This means even if a task needs to analyze 10 files, the plugin processes 1-2 at a time.
   • Benefits: "Max Concurrent Requests" controls the request peak, and the frequency limit controls the continuous flow, ensuring smooth operation.

Suggestion 2: Smart Error Handling and Automatic Retries

Instead of just stopping when it hits a 429 error, Kilo Code could be much smarter. Let's implement these strategies:

1. Parse the Retry-After Header

   • What it is: When the API returns a 429 error, it often includes a Retry-After field, telling the client how long to wait.
   • How it works: Kilo Code catches the 429, reads the Retry-After value, displays a friendly message (e.g., "Waiting 55 seconds for an automatic retry..."), and resumes the task after the wait.

2. Implement Exponential Backoff

   • What it is: If Retry-After isn't available, use exponential backoff: wait 1 second, retry; if it fails, wait 2 seconds, retry; then 4 seconds, and so on, until it succeeds or reaches a retry limit.
   • Benefits: Makes the plugin resilient, recovering from temporary network issues or rate limits, greatly improving task success and user experience.

Suggestion 3: Introducing a "Free-Tier Friendly Mode"

To make Kilo Code easier to use for new users, especially those using free API keys, we could add a one-click optimization mode:

• What it is: A simple toggle in the settings: "Optimize for low quota/free API keys."
• How it works: Enabling this mode applies pre-set conservative configurations.
  • Maximum concurrent requests: 1
  • API request frequency limit: 8 seconds (approximately 7-8 RPM)
  • Automatically enable the intelligent retry mechanism.
  • Might default to a more economical model (like gemini-flash).
• Benefits: Lowers the barrier to entry, preventing frustration and abandonment during initial use.

Suggestion 4: Improve Transparency and Guidance in Settings

Let's make the settings more intuitive:

1. Improve Setting Descriptions

   • Clarify that the "API request frequency limit" might not control initial parallel requests and guide users to the "Max concurrent requests" setting.

2. Add Onboarding Guidance

   • When users first configure Gemini API keys, show a prompt: "We detected you may be using a free key. For the best experience, try [Free-Tier Friendly Mode] or [upgrade your API plan]."

Summary

Kilo Code has incredible potential! By addressing the availability issues in restricted API environments (particularly with a global request queue and smart retries), you'll attract and retain more users. This improves the product's reliability and user experience and shows the team cares about all developers, regardless of their API budget. Thanks for the awesome tool; I'm excited to see it evolve!

Sincerely, A Kilo Code User

Reproduction Steps for 429 Errors

To help you consistently reproduce these issues, here's a step-by-step guide:

I. Environment Setup

• API Key:

  • Get a standard Gemini API key.
  • Key Requirement: This must be a free tier key from Google AI Studio, not tied to a Google Cloud project with billing enabled. This ensures strict rate limits (e.g., Gemini 1.5 Pro is often limited to 2-15 RPM).

• VS Code Plugin:

  • Install the latest Kilo Code VS Code extension.

• Sample Project:

  • Create a folder with these three files:
    • index.html

    <!DOCTYPE html>
    <html lang="en">
    <head>
      <meta charset="UTF-8">
      <title>Kilo Code Test</title>
      <link rel="stylesheet" href="style.css">
    </head>
    <body>
      <h1 id="main-heading">Hello, World!</h1>
      <button id="action-button">Click Me</button>
      <script src="script.js"></script>
    </body>
    </html>
    

    • style.css

    body {
      font-family: sans-serif;
      display: flex;
      flex-direction: column;
      align-items: center;
      justify-content: center;
      height: 100vh;
    }
    
    button {
      padding: 10px 20px;
      font-size: 16px;
    }
    

    • script.js

    const heading = document.getElementById('main-heading');
    const button = document.getElementById('action-button');
    
    button.addEventListener('click', () => {
      heading.textContent = 'Button Clicked!';
    });
    

II. Steps to Reproduce

1. Open the project folder in VS Code.
2. Configure Kilo Code: Enter your Gemini free plan API key in the "Providers" settings. Choose a model like gemini-1.5-pro-latest.
3. Open the Kilo Code chat or task panel.
4. To trigger parallel requests, use the @workspace command.
5. Enter this command: @workspace Please add a new function to the button: when the button is clicked for the first time, in addition to changing the title text, also add an 'active' CSS class to the title. Please add a prominent color, such as crimson, for the 'active' class in style.css.
6. Send the command and observe.

III. Expected Result

• Kilo Code processes the task.
• Requests are sent to Gemini in an orderly manner to analyze files and generate code.
• The task might take a little longer but completes successfully without 429 errors.

IV. Actual Result

• The Kilo Code task fails immediately.
• An error prompt appears, or the error message "API request failed" and "got status: 429 Too Many Requests" is displayed in the output panel.
• The error details indicate that the free plan quota has been exceeded.

This clearly shows the parallel requests at the task's start overwhelming the free API key's rate limits.