Symfony HttpClient Caching Issue With Scoped Clients

Hey guys! Let's dive into a tricky situation some of us have encountered while using Symfony's HttpClient: Scoped clients misbehaving when paired with the Caching HTTP Client. It's a head-scratcher, but we'll break it down and see what's going on.

The Problem: Invalid URL Exception

So, here's the deal. When you're rocking Symfony 7.4 and trying to use a scoped client with the Caching HTTP Client, you might run into an Invalid URL: scheme is missing exception. This nasty error pops up during the request, and it can be super frustrating. Imagine you're trying to fetch some data from an API, and instead of the sweet, sweet JSON, you get this error. Not cool, right?

To be more specific, this issue arises when you make a request without including the scheme and domain. For example, instead of $client->request('GET', 'https://api.github.com/repos/symfony/symfony/releases/latest'), you're doing $client->request('GET', '/repos/symfony/symfony/releases/latest'). Seems simple enough, but this is where the trouble starts.

The CachingHttpClient, in its eagerness to validate the URL, jumps the gun and throws the exception before the ScopingHttpClient even gets a chance to set the base URI. It's like the bouncer at the club refusing entry before you've even shown your ID. You can see this happening in the CachingHttpClient.php file, specifically around line 152. This line of code is where the URL validation happens, and it's the first thing the request method does. So, no base URI, no entry!

How to Reproduce the Issue

Okay, so how do you actually make this happen? There are a couple of ways, both equally likely to make you pull your hair out if you don't know what's going on. Let's walk through them.

Manual Creation of Scoped Client

First up, you can manually create the scoped client. This involves a bit of code, but it's pretty straightforward. Check it out:

$client = new CachingHttpClient(
    ScopingHttpClient::forBaseUri(HttpClient::create(), 'https://api.github.com'),
    new TagAwareAdapter(new ArrayAdapter()),
);

$release = $client->request('GET', '/repos/symfony/symfony/releases/latest');

In this snippet, we're creating a CachingHttpClient that wraps a ScopingHttpClient. The ScopingHttpClient is set up with a base URI of https://api.github.com. All seems well, right? But when you make that request to /repos/symfony/symfony/releases/latest, boom! The exception hits you like a ton of bricks.

Using framework.yaml Configuration

Alternatively, you might be using Symfony's framework.yaml config to set up your HTTP clients. This is a cleaner way to do it, but it doesn't magically protect you from this issue. Here’s how the config might look:

framework:
    http_client:
        scoped_clients:
            github.client:
                base_uri: 'https://api.github.com'
                caching:
                    cache_pool: cache.app.taggable

Here, we're defining a scoped client named github.client with the same base URI. The caching is enabled using a taggable cache pool. Again, this setup looks perfectly reasonable. But make a request without the full URL, and you're back in exception land. It's like the universe is conspiring against us, isn't it?

Why Does This Happen?

Let's break down why this is happening, because understanding the root cause is the first step to fixing it. The issue boils down to the order of operations within the CachingHttpClient. As mentioned earlier, the prepareRequest method, which includes URL validation, is called before the ScopingHttpClient has a chance to apply the base URI. This is the crucial point.

The CachingHttpClient is designed to add caching capabilities to any HTTP client. It intercepts requests, checks the cache, and either returns a cached response or forwards the request to the underlying client. However, its early URL validation is too strict in this scenario. It expects a fully qualified URL right off the bat, which isn't always the case when using scoped clients.

The ScopingHttpClient, on the other hand, is designed to handle base URIs. It takes a base URI and automatically prefixes it to any relative URLs in the requests. This is super handy for working with APIs that have a consistent base URL. But, it needs its turn to modify the URL before validation happens.

It’s a classic case of two components stepping on each other's toes. The CachingHttpClient is being a bit too eager, and the ScopingHttpClient isn't getting a chance to do its job. This clash results in the dreaded Invalid URL exception, leaving developers scratching their heads.

Possible Solutions and Workarounds

Alright, so we know the problem, we know how to reproduce it, and we understand why it's happening. Now, let's talk solutions. Unfortunately, there's no one-size-fits-all magic bullet here, but there are a few approaches you can take to work around this issue.

Use Full URLs

The simplest, albeit somewhat tedious, workaround is to always use full URLs in your requests. This means instead of $client->request('GET', '/repos/symfony/symfony/releases/latest'), you'd use $client->request('GET', 'https://api.github.com/repos/symfony/symfony/releases/latest'). This sidesteps the issue entirely because the URL is already valid when it hits the CachingHttpClient.

However, this approach can make your code less clean and more prone to errors. Imagine having to update the base URI in multiple places if it ever changes. Not a fun task, right? Plus, it kind of defeats the purpose of using a scoped client in the first place. So, while this works, it's not ideal.

Custom Client Implementation

Another approach is to create a custom HTTP client that handles the URL preparation differently. This would involve extending or wrapping the existing CachingHttpClient and ScopingHttpClient to modify the order in which the URL is validated. This is a more advanced solution, but it gives you the most control over the process.

For example, you could create a class that first applies the base URI from the ScopingHttpClient and then performs the URL validation. This would ensure that the URL is fully formed before the validation check, resolving the issue. However, this requires a good understanding of the underlying components and their interactions.

Monkey-Patching (Use with Caution)

A more controversial approach is to