Integrating new grid features in ggplot2

[thomasp85]

Hi Team

The graphic engine and grid is getting some love lately and we will look into a future where gradient + image fill and non-rectangular clipping is available on the devices that supports it. I think both have interest from the ggplot2 side.

While we are not there yet, @pmur002 has a working implementation and I think we may just as well begin discussing to what extend and how we may use this in ggplot2. I think the most interesting (seeing as ggpattern exist) possibility is gradient fill. This will most definitely be a challenge to integrate into the scale system because gradients depend on multiple values. Should the scaling happen from discrete values to a set of predefined gradients? or should each stop be mapped to different values. Lots of things to discuss!

Non-rectangular clipping also opens up a lot of possibilities, some not having a clear parallel in the current API (e.g. clipping groups or layers to certain areas)

@clauswilke @karawoo @hadley @yutannihilation @paleolimbot please join the discussion 🙂

[paleolimbot]

The geologists are going to be so excited! My quick unqualified comment would be that a vectorized "fill spec" (probably generated by ScalePattern$map()) might do the trick. How that interacts with the colour and fill aesthetics seems tricky.

[clauswilke]

I think allowing fill to vary along x or y in polygons would be very useful, and would probably be something people would expect. As seen here: https://wilkelab.org/ggridges/reference/geom_ridgeline_gradient.html

[pmur002]

If you want to play along, this is all happening in an svn branch [https://svn.r-project.org/R/branches/R-defs/], though that is VERY unstable, both in the sense of being a segfault magnet and in the sense of changing all the time. I am also documenting progress [https://www.stat.auckland.ac.nz/~paul/Reports/r-defs-proposal/defs-proposal.html], though that also needs updating already.

[clauswilke]

Paul, this is great! Do you take feature requests? One feature that would be really useful and probably isn't that much work (famous last words!) is some sort of link or anchor. Obviously it'd have to be ignored in raster devices, but it'd be super useful for svg or pdf output.

[pmur002]

I take requests, but there is no guarantee that I will do anything with them :) I have this semester teaching free (thanks to an R Studio donation to UoA), which is giving me time to tackle something a bit larger like this, BUT I still have to scope it so that I have some hope of getting something merged back to trunk before I sink without trace back into teaching in the next semester. Long way of saying that additional requests are almost certainly out of scope this time around.

[mdsumner]

Thanks @thomasp85 and @pmur002 , could this include (or lead to) general textures, by which I mean any given triangle region could have a fill that is mapped from a bitmap image? Interpolation of the image across the triangle is essentially done at device resolution, with mapping from the triangle corners to the coordinate system of the image. So, I see it as a "fill" across a region that can represent the coordinate system of another data set.

[mdsumner]

(I know it's off-topic, so sorry for jumping on here - been trying to find where to "park" this wish)

[pmur002]

The scope of this project does not include textures. Not sure if textures are ever a real possibility - I am leaving some graphics devices behind with the set of features under consideration, but textures might leave all graphics devices behind :) One outside chance is "mesh patterns" [https://www.cairographics.org/manual/cairo-cairo-pattern-t.html#cairo-pattern-create-mesh], which both Cairo and PDF can in theory support, but that's still not textures.

[clauswilke]

With arbitrary clipping regions, it should be relatively straightforward to implement texture fill by just drawing the texture and then clipping to the desired shape. This can be done outside the graphics device, by client code. (Similar to ggpattern but easier to implement.)

[thomasp85]

I think what @mdsumner is alluring to is to have a each corner of a triangle mapped to a texture location and let every pixel inside the triangle be interpolated from the texture based on these coordinates... I think this would be too hard to solve generally as well - very few 2D drawing libraries support this

[mdsumner]

Great, thanks for the feedback that alone is helpful - appreciate the work here it's exciting, reading @pmur002's document is very enlightening!

[pmur002]

I forgot that my report is on github, so you can keep up with the latest version here (https://github.com/pmur002/r-defs-proposal/blob/master/defs-proposal.html)

[stragu]

Just linking to relevant discussions and prior work:

• Write-up of upcoming changes on R dev blog: https://developer.r-project.org/Blog/public/2020/07/15/new-features-in-the-r-graphics-engine/index.html
• ggpattern package: https://coolbutuseless.github.io/package/ggpattern/index.html
• ggrough package: https://xvrdm.github.io/ggrough/
• patternplot package: https://cran.r-project.org/web/packages/patternplot/vignettes/patternplot-intro.html
• ggtextures package: https://github.com/clauswilke/ggtextures#ggtextures
• StackExchange question on filling areas with patterns/texture: https://stackoverflow.com/questions/2895319/how-to-add-texture-to-fill-colors-in-ggplot2
  • A custom function that came out of one of the answers: https://github.com/cratlagh/EggHatch/

[jimjam-slam]

Just wanted to check in and see if there's been any branch work on this. I've been tinkering with {gridtext} a bit to see if element_textbox works with the new gradient support—it does, mostly (I'm having a bit of trouble with PDF support that I'm still working through), but I realised geom_textbox is more likely to just inherit support from ggplot2 when it's working here, so I didn't want to try and reinvent the wheel 😅

[pmur002]

Keen to hear if you have spotted bugs in the PDF support; it would be great to fix those BEFORE 4.1 if possible. You can direct email me if you like.

[jimjam-slam]

I'll do some more testing and shoot you an email, Paul!

[pmur002]

Great. Thanks

[jimjam-slam]

False alarm, @pmur002! I was misspecifying the PDF device. This example works for both PDF and PNG on Linux and Windows:

devtools::install_github("rensa/gridtext@feature-gradients")
library(tidyverse)
library(ggtext)

myplot <-
  ggplot(mtcars) +
  aes(mpg, hp) +
  geom_point() +
  theme_grey(base_size = 20) +
  theme(
    plot.title = element_textbox(
      colour = "red",
      fill = grid::linearGradient(
    colours = c("#020024", "#102b81", "#833ab4"),
    stops = c(0, 0.4, 1)))) +
  labs(title = "Hello gradients!", subtitle = "Another go")

ggsave('test_grad_title.png', myplot, device = png(type = "cairo-png"))
ggsave('test_grad_title.pdf', myplot, device = cairo_pdf)

[pmur002]

Ok. Cool. Still good to know that this is working for someone other than me :)
(and those examples also work for me!)
Thanks for doing the testing.

[toddrjones]

Just linking to relevant discussions and prior work:

• Write-up of upcoming changes on R dev blog: https://developer.r-project.org/Blog/public/2020/07/15/new-features-in-the-r-graphics-engine/index.html

• ggpattern package: https://coolbutuseless.github.io/package/ggpattern/index.html

• ggrough package: https://xvrdm.github.io/ggrough/

• patternplot package: https://cran.r-project.org/web/packages/patternplot/vignettes/patternplot-intro.html

• ggtextures package: https://github.com/clauswilke/ggtextures#ggtextures

• StackExchange question on filling areas with patterns/texture: https://stackoverflow.com/questions/2895319/how-to-add-texture-to-fill-colors-in-ggplot2

  • A custom function that came out of one of the answers: https://github.com/cratlagh/EggHatch/

The answer in this link contains code that improves ggrough: https://stackoverflow.com/questions/64031046/how-to-shade-shapes

[jimjam-slam]

Using patterns in theme elements works fine in ggplot2 3.3.3 with R 4.1, but aesthetics as patterns (even if they're static and not actually mapped to the data) generally stop in check_aesthetics with Aesthetics must be either length 1 or the same as the data (32): fill.

library(ggplot2)
my_gradient <- grid::linearGradient(
  colours = c("#020024", "#102b81", "#833ab4"),
  stops = c(0, 0.4, 1))

# works!
myplot_works <-
  ggplot(mtcars) +
  aes(mpg, hp) +
  geom_point() +
  theme_grey(base_size = 20) +
  theme(
    plot.title = element_textbox(
      colour = "red",
      fill = my_gradient)) +
  labs(title = "Hello gradients!", subtitle = "Another go")
ggsave('test_grad_title.png', myplot_works, device = png(type = "cairo-png"))

# nope
myplot_nope <-
  ggplot(mtcars %>% rownames_to_column()) +
  aes(x = rowname, y = mpg) +
  geom_col(fill = my_gradient) +
  theme_grey(base_size = 20) +
  labs(title = "Hello gradients!", subtitle = "Another go")
ggsave('test_grad_bars.png', myplot_nope, device = png(type = "cairo-png"))

I tried being sneaky and passing a list of patterns for each data element, but it got stuck in farver::decode_colour.

I'm slowly poking away at this in my own time; just wanted to let everyone in know in case it's being worked on privately or an approach has already been sorted! My next step was going to be looking more at how ggplot2 handles aesthetics and maybe doing some vctrs wrappers for the patterns to help handle them explicitly. But, as @thomasp85 noted in the OP, you need to work out how things are handled given a gradient (or, I suppose, a pattern) might depend on multiple data columns.

[mjskay]

I hope this isn't the wrong place to ask this question, but I've been playing with using linearGradient() for gradient fills based on x/y mappings in {ggdist} and I have a basic implementation working (see the 'Note on “choppy” gradients' here). My question is, is there a way to tell (say in makeContent() or something like that) if the current graphics device supports gradients? That would be helpful as a way to automatically provide fallbacks. Currently I have to provide an option for users to "opt in" to the new linearGradient() approach since I don't know how to detect if it is supported or not.

[pmur002]

Sorry, that information is not currently made available. I should probably add something to the device API, exposed via dev.capabilities(). Unfortunately, that may take a full R version cycle to become available.

[mjskay]

Makes sense. Thanks for this btw, having proper gradients is really exciting!

[ssp3nc3r]

Is this still a good place to watch for updates on work integrating the new gradient features into ggplot? Or for examples in how to use them together?

[teunbrand]

I would just like to leave this link to Paul Murrell's new report here for people who enjoy seeing what might be in store for R4.2.

[pmur002]

@mjskay (and anyone else interested): I have committed a change to r-devel (r81278) for grDevices::dev.capabilities() so that it reports on device support for pattern fills, clipping paths, masks, and some new features just in r-devel. The idea is that FALSE means no support, NA means unknown (but almost certainly no support). For simple cases, TRUE means support, but where support can be more nuanced there is a character vector of supported sub-features. Try, for example ...

xfig()
dev.capabilities()
postscript()
dev.capabilities()
pdf()
dev.capabilities()
png(type="cairo")
dev.capabilities()

Hope that provides the information you need.

[mjskay]

@pmur002 this is perfect, thank you!

[pmur002]

@mjskay Great. Thanks for taking a look.

[teunbrand]

I don't know if this is the appropriate thread for this, but I recently found out it would be relatively easy to enable pattern fills in ggplot2 (see r-lib/scales#367). However to go from 'enable' to 'full support' might still be some extra work.

[thomasp85]

This issue is about communicating about development, not issuing updates to users. There is nothing to share at the moment

[teunbrand]

With #5299 merged, is there anything else of the newer graphics features that should be enabled in ggplot2? I have the feeling that clipping paths, alpha/luminance masks, affine transformations, compositing etc. would be great for extension packages that implement their own layers and/or theme elements, but wouldn't need infrastructure in ggplot2 to do this.

[trevorld]

I have the feeling that clipping paths, alpha/luminance masks, affine transformations, compositing etc. would be great for extension packages that implement their own layers and/or theme elements, but wouldn't need infrastructure in ggplot2 to do this.

As a developer who has used clipping paths, alpha masks, and affine transformations in extension geoms I think this is correct. The biggest pain point as a developer who uses the affine transformation feature in an extension geom is getting my examples to work correctly in {pkgdown} which is unrelated to {ggplot2}.

[teunbrand]

Alright I think we can go ahead and close this and if there is a new feature in grid that needs to be integrated in ggplot2 we can open a separate issue for that.