Friday, June 4, 2021

Tackling responsive images

 

One of the main goals when I started with Stitcher was heavily optimized images. Looking at the HTTP Archive stats, it's clear we're doing something wrong. Luckily, the Responsive images spec has been made by a lot of smart people to counter the image problem. My goal was to implement this spec in library in a way that was easy enough for developers to use it to its full extent. We're not completely there yet, but we're close. In this blogpost I want to talk about the challenges I faced creating this library.

To be clear: the goal of the responsive images spec is to reduce bandwith used when downloading images. Images nowadays require so much bandwith. When you think about it, it's insane to load an image which is 2000 pixels wide, when the image on screen is only 500 pixels wide. That's the issue the spec addresses, and that's the issue I wanted to solve in Stitcher.

So I want one image to go in, x-amount of the same image with varying sizes coming out, and let the browser decide which image is the best to load. How could I downscale that source image? That was the most important question I wanted answered. All other problems like accessebility in templates and how to expose the generated image files, were concerns of Stitcher itself.

My first take on downscaling images was the following:

Take the source image and a set of configuration parameters. These parameters would decide the maximum amount of image variations and the minimum width of the image. Eg. I want a maximum of ten images, with the smallest image being 300 pixels wide. Now the algorithm would loop a maximum of 10 times, always creating an image which is 10% smaller in width than the previous one.

You might already see this is not the optimal approach. After all: we're trying to reduce bandwith used when loading images. There is no guarantee an image which is downscaled 10%, is also reduced in size. Much depends on which image codecs are used, and what's in the image itself. But by using this approach early on, I was able to implement this "image factory" with Stitcher. Next I would be working on optimizing the algorithm, but for the time being I could tackle the Stitcher integration.

 Linking with Library

Letting Library know about responsive images was both easy and difficult at the same time. The basic framework was already there. So I could easily create an image provider which used the responsive factory, and returned an array representation of the image. The template syntax looks like this:

<img src="{$image.src}" srcset="{$image.srcset}" sizes="{$image.sizes}" />

Unfortunately, there is no way to automate the sizes part, unless you start crawling all CSS and basically implement a browser engine in PHP. My solution for this part is pre-defined sets of sizes. That's still a work in progress though, I'm not sure yet how to make it easy enough to use. For now, I'm just manually specifying sizes when writing template code.

But the tricky part wasn't the sizes, neither the srcset. It was handling paths and URLs. I've noticed this throughout the whole framework: creating the right paths and URLs (correct amount of slashes, correct root directory etc.) is actually quite the pain to manage. I'm convinced by now I need some kind of helper which always renders the correct paths and URLs. It's on my todo list.

 

A pretty robust library came to be. You could throw it any image, and it would generate a set of variations of that images, scaled down for multiple devices. It returned an object, which Stitcher parsed into a template variable. In templates, the following is now possible.

<img src="{$image.src}" srcset="{$image.srcset}" sizes="{$image.sizes}" />

Like I wrote earlier, the first version of the scaling down algorithm was based on the width of images. It worked, but it wasn't solving the actual problem: optimizing bandwidth usage. The real solution was in downscaling images based on their filesizes. The problem there: how could you know the dimensions of an image, when you know the desired filesize. This is where high school maths came into play. I was actually surprised how much fun I had figuring out this "formula". I haven't been in school for a few years, and I was rather happy I could use some basic maths skills again!

This is what I did:

filesize = 1.000.000
width = 1920
ratio = 9 / 16
height = ratio * width

area = width * height
 <=> area = width * width * ratio

pixelprice = filesize / area
 <=> filesize = pixelprice * area
 <=> filesize = pixelprice * (width * width * ratio)
 <=> width * width * ratio = filesize / pixelprice
 <=> width ^ 2 = (filesize / pixelprice) / ratio
 <=> width = sqrt((filesize / pixelprice) / ratio)

So given a constant pixelprice, I can calculate the required width an image needs to have a specified filesize. Here's the thing though: pixelprice is an approximation of what one pixel in this image costs. That's because not all pixels are worth the same amount of bytes. It heavily depends on which image codecs are used. It is however the best I could do for now, and whilst I might add some more logic in the future, I'd like to try this algorithm out for a while.

So now the Responsive Factory scales down images by filesize instead of width. A much better metric when you're trying to reduce bandwidth usage. This is how the library is used in Stitcher:

use Brendt\Image\Config\DefaultConfigurator;
use Brendt\Image\ResponsiveFactory;

$config = new DefaultConfigurator([
    'driver'      => Config::get('engines.image'),
    'publicPath'  => Config::get('directories.public'),
    'sourcePath'  => Config::get('directories.src'),
    'enableCache' => Config::get('caches.image'),
]);

$responsiveFactory = new ResponsiveFactory($config);

All images in Library go through this factory, the factory will generate x-amount of variations of the image, and the browser decides which one it will download. Its pretty cool, and I hope it will help websites to serve more optimized images, while a developer can still focus on the most important parts of his project.

 

Saturday, May 22, 2021

A Guide to Solving Those Mystifying CORS Issues

Imagine you’re building the UI. You need to connect to remote API to get or send some data. Everything works fine when you test your REST calls with curl, but when you implement them in the UI, it does not.

First, you check the code, looking for some typos or other mistakes —but everything seems to be fine. You change the URL to Google.com or something, and find out that the http call is working. The issue appears only when calling that specific API. But it works perfectly fine via command line or Postman. What’s going on then?

Well, it’s probably the mysterious CORS mechanism blocking you. Rather than making an assumption, though, let’s just check the developer console in the browser. So if you hit right click, select Inspect, and go to Console tab, and then see an error like this one 

… then indeed, it’s CORS.

But before you jump to Stack Overflow asking ‘How to fix CORS in Angular/React/whatever?’ let’s find out what CORS really is, and why you can’t fix it in the UI.

What is CORS?

CORS stands for Cross-Origin Resource Sharing. Doesn’t explain much, huh? Well, it’s really simple to understand, but there are a lot of misconceptions about CORS and plenty of available ‘solutions’ that don’t work.

When they’re blocked by CORS, many people google a ‘solution for CORS’, copy-and-paste a few lines of code that addresses something about the headers, and move forward. While this may sometimes fix your problem momentarily, it may also create a huge security risk.

But we’ll talk about that later. First things first: What is CORS?

CORS is a security mechanism built into (all) modern web-browsers (yes! into your web browser! That’s why your curl calls works fine). It basically blocks all the http requests from your front end to any API that is not in the same “Origin” (domain, protocol, and port—which is the case most of the time).

Now, how does this mechanism work? Let’s say you have an ‘upload’ button in the UI that suppose to upload some form of data to the API. So, in HTML code, you bind that button to some JavaScript function, which does a http POST call:

blackscreenshot

So when you click that button, you would expect the HTTP POST being sent to the API. But instead, your browser, seemingly ‘hidden’ from you, will send a HTTP OPTIONS request (but not always! Will get to it in a second) to the API. The API will typically reply with a bunch of data that says what browser is allowed to do.

Now, one thing to mention here: HTTP OPTIONS is sent before your actual request, if that request is considered a ‘non-simple’ request. A “non-simple’ request is one that has Content-type other than application/x-www-form-urlencoded, multipart/form-data, or text-plain (for example, JSON) or when requests include cookies. (So, pretty much, most of the time.)

And now, we’re talking CORS. Below, you’ll see an example of the headers sent back by the server (yes, by server—therefore CORS is not something you can fix in the UI code) with a reply to OPTIONS. Look at those Access-Control-* headers and focus on Access-Control-Allow-Origin:

errormessage

Here’s what’s happening: before sending your requested API call, your browser does a ‘security check’ by asking the API, (via an OPTIONS call, who is allowed to do what. Simple as that.

An ‘issue with CORS’ occurs when the API does not reply to such request with, ‘Yes, dear browser, you are allowed to do that call’. So, as you can see on the screenshot above, my API responded that my UI, localhost, is allowed to handle OPTIONS, HEAD, DELETE, POST and GET calls.

Now that’s the core of all the ‘problems’ with CORS. In order to fix CORS, you need to make sure that the API is sending proper headers (Access-Control-Allow-*). That’s why it’s not something you can fix in the UI, and that’s why it only causes an issue in the browser and not via curl: because it’s the browser that checks and eventually blocks the calls.

Solutions and Security

So, how to fix it properly without creating a security hole? As I mentioned earlier, people who encounter these errors often just google for a solution and copy-and-paste a few lines of code, which adds proper headers. The problem with that is that, in most cases, those ‘solutions’ tell You to use ‘Access-Control-Allow-Origin: *’ —in other words, basically allow anyone to access Your API.

You might ask, What’s the problem with that? I have authentication on my API anyway. (Which should always be the case, right?)

Well, if you do, then this solution won’t work for you. Because if your call from the browser contains an Authorization header, then the value of Access-Control-Allow-Origin can’t be ‘ * ‘.

So the key points to know by now:

  1. CORS is a mechanism built into web browser. It’s not a UI code issue.
  2. To fix CORS problems, you need to make changes on the API side.

But… but…, you protest, I don’t have access to that API!

Well, in that case, you have two options:

  1. Ask whoever manages API to fix/add CORS support
  2. Create middleware

Option 1 is clear, right? If the API you are trying to access is your company API, then just go to your backend colleagues and ask them to add CORS support. How they do that will depend on the framework they use. Sometimes it’s as easy as installing some package, sometimes they have to add those headers manually to the API code, but nevertheless—they should know.

If the API is from some third party, then either you can contact them via their support line, or Github, or some other way. Or, you can use Option 2.

Option 2: build a middleware. Since CORS is as simple as adding some HTTP headers, and it’s the only browser blocked, then you can build some proxy-like component that will basically make a call for you, get the response from the desired API, add those headers on top, and then send it back to Your UI. Thus, you will no longer connect directly to that API, but to your middleware. It’s not the best solution, but if really necessary, it will solve the issue.

There is also another use case, You have some tool installed on one of the servers you manage. You can’t directly change the code of that tool, but you still need to add CORS support to it. In such a case, something you can do is install, for example, Nginx—add headers in Nginx config, and put that tool as a backend.

That’s it. I hope you not only got a solution to issues with CORS but, most importantly, you learned how it works. And now you have some ideas for how to fix things when you get those once-mystifying CORS messages.

With an exceptional team by your side, you can discover what you are truly capable of, click below to find out our job openings!