134 points by BoingBoomTschak 2 days ago | 10 comments
svantana 18 hours ago
I feel like this article is really overselling this filter. A 4-point symmetric interpolation kernel can be parameterized as [k, 1-k, 1-k, k]/2, i.e. it has a single degree of freedom. k=-1/4 is bicubic, k=1/4 is this 'magic', and k=0 is bilinear. Bicubic is sharper, and 'magic' has better alias rejection. Which looks better depends on the image and the viewer's subjective preference. For insta photos, it's probably better to go for 'magic', while for text, one might prefer bicubic. Neither is "simpler" as this article keeps suggesting, they just have different filter coefficients, that's all. But any other value of k is an equally valid choice.
BoingBoomTschak 18 hours ago
It certainly is. Especially lacking in proper comparisons of the final filter with the competition. I myself default to RobidouxSharp for downscaling and something like https://www.imagemagick.org/discourse-server/viewtopic.php?t... for upscaling.
BoingBoomTschak 14 hours ago
Took the time to make such a comparison using the article's sample images (even if the filter isn't sharpened in those), same thrice doubling of the small picture: http://0x0.st/XEEZ.png

I find such a test strange and irrelevant, though.

dahart 14 hours ago
Ooh the animated comparison is really helpful. I couldn’t see it in the article, but with your version the Magic version feels flatter, almost like it’s a bokeh blur and not just a low pass. The Sigmoid seems far better than either Magic or Bicubic.
raphlinus 15 hours ago
The page mostly talks about image resampling where the goal is more or less preserving all frequencies, but it's also extremely effective at implementing Gaussian blur. Basically, you do n iterations of sampling by 1/2 using this kernel, followed by a very small FIR filter, then n iterations of upsampling 2x using the same kernel. Here, n is essentially log2 of the blur radius, and the total amount of computation is essentially invariant to that radius. All these computations are efficient on GPU - in particular, the upsampling can be done using vanilla bilinear texture sampling (which is very cheap), just being slightly clever about the fractional coordinates.

It works well because, as stated, the kernel does a good job rejecting frequencies prone to aliasing. So, in particular, you don't get any real quality loss from doing 2x scale changes as opposed to bigger steps (and thus considerably larger FIR support).

I have some Python notebooks with some of these results, haven't gotten around to publishing them yet.

leguminous 10 hours ago
I have done something like this with a Lanczos kernel (a=1) downsizing repeatedly by 2x, a small Gaussian kernel, and then repeatedly upsizing by 2x with simple hardware bilinear sampling.

The (2D) Lanczos downsizing can be done with only four samples using the bilinear sampling tricks that you mention, and I avoided expensive trigonometric functions, divisions, and the singularity at 0 by using an even 8th order polynomial approximation. I would be curious to see the results using this kernel, but the Lanczos is so far the best that I've tried.

kragen 11 hours ago
I look forward to being able to read your notebooks!
rnhmjoj 17 hours ago
> Fourthly, and most importantly, as noted above: m(x) is a partition of unity: it “fits into itself”; [...] if we place a copy of m(x) at integral positions, and sum up the results, we get a constant (unity) across all x. [...] This remarkable property can help prevent “beat” artifacts across a resized image.

So, basically the reason why this works better than other visually similar filters is that it happens to satify the Nyquist ISI criterion[1].

[1]: https://en.wikipedia.org/wiki/Nyquist_ISI_criterion

herf 17 hours ago
This uniform b-spline is the same one used often as a "Gaussian" approximation (three box filters) - see Paul Heckbert's 1986 paper here (apparently done at NYIT in the early 1980s with help from Ken Perlin):

https://dl.acm.org/doi/pdf/10.1145/15886.15921

pseudosavant 18 hours ago
I was surprised I hadn't heard of this, or his related project JPEG-Clear. I have thought for years that the JPEG-Clear method is how responsive images should have been handled in a browser. A single-file format that can be progressively downloaded only up to the resolution it is being displayed at. If you zoom in, the rest of the data can be downloaded for more detail. Doesn't require complex multi-file image authoring steps, keeps simple <img src> grammar, and is more efficient than downloading multiple completely separate images.
meindnoch 17 hours ago
JPEG-Clear? The guy "reinvented" progressive JPEGs?
Dwedit 7 hours ago
Loading a more detailed version of an image as you zoom in is different from what a progressive JPEG does.

Loading a Progressive JPEG means you still unconditionally load the entire file, you just are able to show a low detail version before it is fully loaded. The last time I saw a progressive JPEG actually take time to load was when I had dialup.

meindnoch 6 hours ago
1. You can terminate the loading process as soon as you're satisfied with the quality. It's just that browser don't do that.

2. The OPs JPEG-Clear proposal [1] also loads the entire file no matter what. It's literally just a reinvention of progressive JPEGs, presenting it as something novel.

[1] https://johncostella.com/jpegclear/

layer8 17 hours ago
In the “Bicubic: note the artifacts” comparison images, the bicubic version, regardless of the aliasing, is less blurry and has more detail than the “magic kernel” version. I therefore don’t agree that the latter is “visually, far superior”. There is at least some trade-off.
BoingBoomTschak 18 hours ago
These previous discussions (including the author in the second one) were pretty fruitful:

https://news.ycombinator.com/item?id=10404517 (2015)

https://news.ycombinator.com/item?id=26513518 (2021)

bhouston 19 hours ago
Super cool. How did I not know about this before?
BoingBoomTschak 18 hours ago
I was also pretty surprised, as I consider myself decently knowledgeable in the field. Learned of it via https://github.com/libvips/libvips/issues/4089.
DustinBrett 17 hours ago
Would be cooler if images on FB didn't suck.
CyberDildonics 17 hours ago
I guess anything is magic if you don't know how it works or if you need some clicks to promote your personal site.

This is basically a slightly different gaussian kernel and the "incredible results" of a small image becoming a larger resolution blurry image is completely normal.

Also you don't want negative lobes in image kernels no matter how theoretically ideal it is, because it will give you ringing artifacts.

If you work with image kernels / reconstruction filters long enough you will eventually learn that 90% of the time you want a gauss kernel.

pixelpoet 16 hours ago
> If you work with image kernels / reconstruction filters long enough you will eventually learn that 90% of the time you want a gauss kernel.

Strongly disagree, and my commercial software is known for its high image quality and antialiasing. Gaussian is way too blurry unless you're rendering for film.

dahart 14 hours ago
In my film experience, I think most film people don’t like Gaussian either; too blurry for them as well. At least, I’ve sat in on filter evaluations with a couple of directors & VFX sups many years ago, and they said Gaussian was too soft and preferred a sharper Mitchell. But I am curious, perhaps similar to the sibling comment - how do you determine the optimal Gaussian width? You can certainly go narrower/sharper and get less blur at the cost of more artifacts similar to a sharper filter, right? BTW have we discussed this recently? ;) I love Gaussian’s ability to hide any hint of the pixel grid, which I find very few filters can do. I also tend to believe that, perceptually speaking, over-blurring slightly doesn’t hurt while under-blurring does, especially for moving things, but that might be more personal bias than objective reality. I would be interested to look at any comparisons or results from your software or in general, if you have some.
a_e_k 8 hours ago
Historically, Pixar's RenderMan defaulted to a 2x2 Gaussian filter [1], and from what I can see, that hasn't changed [2].

Essentially it uses a truncated isotropic (non-separable) Gaussian defined by exp(-2.0 * (x*x + y*y)) with a 2x2 pixel support [3], which is slightly soft and completely avoids ringing.

Gaussian also plays very nicely with filtered importance sampling [4] since it has no negative lobes.

(Though I remember a number of studios using RenderMan preferring filters with a bit of sharpening.)

[1] https://paulbourke.net/dataformats/rib/RISpec3_2.pdf#page=40

[2] https://rmanwiki-26.pixar.com/space/REN26/19661819/Filtering

[3] https://paulbourke.net/dataformats/rib/RISpec3_2.pdf#page=20...

[4] https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...

dahart 37 minutes ago
Oh very interesting, thanks! Yeah I guess the PDI folks wanted the benefits without the slightly soft part, but I stand corrected, not all film people are the same.
CyberDildonics 13 hours ago
The truth is that when people test out filters they are looking close up at the pixels, trying to squeeze out detail, but the reality is that whatever minute detail might get slightly softened by a 2.2-2.5 gauss filter will get chewed up by the process of color correction and compression anyway.

The aliasing you can end up with from a mitchell filter though can be noticeable all through the process. Not only that, but what will the compositor do when they see the aliasing? They'll blur it.

Basically it is trying to squeeze blood from a stone and the image out of renderer is going to be far sharper than anyone will see because it will go through multiple stages. Even compositing almost never leaves a render verbatim. There is usually some sort of slight softening, chromatic aberration, lens distortion and/or other transforms that require resampling anyway.

It is picking up pennies in front of a bulldozer and only causes problems to have a filter that's too sharp, let alone one that has negative lobes.

dahart 10 hours ago
Oh FWIW, the sessions I remember most vividly were for the first Shrek movie, which was printed to actual film, projected to a theater screen, and final render was at slightly lower than 1080p resolution. The digitizing to film does add a little blur, of course, but not a lot since the resolution was so low and the pixels so big. The filter did kinda matter, and the compositors did not add extra blur in this case. I was highly impressed with how sensitive the director and lighting/fx supes were to filter differences, and how quickly they could spot them in animated clips that were often less than one second long. The main thing they were doing was trying to avoid texture sizzle without overblurring.
CyberDildonics 9 hours ago
The digitizing to film

Scanning film would be digitizing it, going out to film could be called printing.

does add a little blur

A lot of blur. 35mm film had a lot of folk wisdom built up around it and even though it would be scanned and worked on at 2k resolution for live action vfx, if you actually zoomed in it would be extremely soft.

You could get away with 1.5k just for printing to the first generation film, let alone the 2nd gen master prints and the third gen prints that went out to theaters.

The filter did kinda matter

It is extremely unlikely lighters were changing the actual pixel sample filters on a shot by shot basis. This is something that is set globally. Also if you set it differently for different passes and you use holdouts, your CG will not line up with the other CG renders and you will get edges from the compositing algebra not being exact.

I was highly impressed with how sensitive the director and lighting/fx supes were to filter differences,

No one is changing the pixel sampling filters on a shot by shot basis and no one is going to be able to tell the filter just by looking at playing back on film. This is simply not reality.

and how quickly they could spot them in animated clips that were often less than one second long

Absolutely not. Whatever you are talking about is not different pixel sample filters. Aliasing in general yes, but that's much more complex.

The main thing they were doing was trying to avoid texture sizzle without overblurring.

This has nothing to do with the renderer's pixel sample filters which is what would be the only analog to the article here. Maybe you are talking about texture filtering, although that is not a difficult problem due to mipmapping and summed area mapping. Even back then a geforce 2 could do great texture filtering in real time.

Maybe you are talking about large filter sizes in shadow maps, which need a lot of samples when using percentage closer filtering, but that has nothing to do with this.

dahart 30 minutes ago
Whoa what’s with the totally confrontational argumentative stance?!? I’m very confused by your reply. I thought we were having a nice conversation about filtering.

Lighters on Shrek were indeed not changing the pixel filter shot by shot. We did, however, multiple times, sit down to evaluate and compare various pixel filters built into the renderer, and when we did that, the pixel filter actually was changed for every shot, so they could be compared.

The filter mattered because it was set globally, and because the resolution was low, less than 1k pixels vertically. That is precisely why we spent time making sure we were using the filter we wanted.

I am in fact talking about different pixel filters, and the director and supes could tell the difference in short animated clips scanned to film, which is why it was so impressive to me. If you don’t believe me, I can only say that reflects on your experience and assumptions and not mine.

Both subtle high frequency sizzle as well as other symptoms of aliasing do occur when the pixel filter is slightly too sharp. I don’t know why you’re arguing that, but you seem to be making some bad assumptions.

a_e_k 8 hours ago
> Basically it is trying to squeeze blood from a stone and the image out of renderer is going to be far sharper than anyone will see because it will go through multiple stages. Even compositing almost never leaves a render verbatim.

Don't forget that these days, it's all going through some ML-based denoiser, anyway. I wouldn't be surprised if filter choice is nearly irrelevant at this point (except for training).

CyberDildonics 15 hours ago
Then your filter width is too wide. Try 2.2 - 2.5