FastLED discussion

[Moonbase59]

I got square eyes this weekend reading, testing and diagnosing ESPS, the NeoPixel and FastLED libraries, MQTT and serial stuff. Here’s my findings:

1. You (@forkineye) did an absolute amazing job with your pixel drivers, integrating both flawless pixel output (even with data-only strips), serial Renard and E1.31 on a tiny device like the ESP-01. For DMX, it’s the most stable thing I ever came across.

2. My tests are done on Linux (I have no Windows), using several Lolin v3 NodeMCUs and WS2812B strips of varying lengths. I also have some "dumb" RGB strips which I will be trying to feed via a H801. And I’ll get a Wemos D1 mini soon, to have another testing platform, the original ESPixelStick unfortunately having prohibitive shipping cost to Germany. I use Atom to work on the code and upload with either the Arduino IDE or esptool.py. For DMX testing, I use both QLC+ and xLights.

3. ESPS is robust without end but turns out to be used with MQTT more and more, to provide a "double use" as show and home lighting controller, for instance with Home Assistant (which I also use). It lacks the groundwork for real great built-in effects, though.

4. Dumb RGB (PWM) output has been abandoned by ESPS (and continued in ESPixelBoard) but somehow I’d love to see "both in one" again.

5. The NeoPixel, NeoPixelBus and FastLED libraries come to mind.

• NeoPixel and NeoPixelBus work okay but don’t have the functionality of FastLED. NeoPixel suffers when also using WiFi/Serial. NeoPixelBus is said to cure that (using DMA) but I have no experience with this library.
• FastLED is absolutely fantastic regarding great effects, HSV calculations, (compact) palettes and fast math. I’ve used it in other projects and simply love it. It is also about 15% faster than NeoPixel. But it suffers from "random pixels" (bad timing) when also using MQTT/Serial, due to the interrupts. You can disable interrupts (or let it "breathe" once in a while) but then you’ll start losing data. A no-go.
• Enter Cory R. King’s FastLED. It’s a few commits behind the original, but uses DMA (I²S) over GPIO3 ("RX" on the NodeMCU). All the features of FastLED and no interrupt issues anymore when using WiFi. Butter-smooth animations, I am using it for 2 projects and love it. You have to have GPIO3 (RX) exposed. Still have to check if serial output is affected.

6. I’ve been testing and banging my ESP-12 (NodeMCU) real hard with highspeed animations and MQTT (plus Home Assistant) at the same time using Cory’s FastLED over the weekend (it still runs) and couldn’t find a flaw. Tested with 60- and 300-LED WS2812B strips and runs buttery-smooth.

   When using the PubSubClient, light effects will be halted whenever the ESP has problems connecting the broker (doesn’t get out of the MQTT loop). This seems not to happen when using the AsyncMqttClient. Conclusion: Stick with AsyncMqttClient.

7. I don’t currently have an ESP-01 (the one used in the original ESPS) here and thus don’t know if it has the pin, does DMA, has the CPU clock (I used 160MHz), and works. @forkineye?

8. If anyhow possible (and not interfering with other stuff you did), would it be possible to test Cory’s FastLED with your setup, be it only to have the groundwork for real good effects? (I don’t know how much work it would be a) simply using it or b) adapting it to use your pixel drivers.)

9. If all else fails (i.e., you like FastLED but it won’t work on the original ESPS), would you consider a "new edition" hardware based on an ESP-12 or even ESP-32?

I’d just love to see ESPixelStick live on, with more features and maybe even more modern hardware. (And the shipping cost drop, so we all could buy the board from you and you being able to make your wife and yourself happy, of course ;-)).

[forkineye]

1. Thanks! It all started on a flight with me sketching timing diagrams on paper and seeing how I could fudge the UART around to offload the ws2811 stream generation.

2. For dumb strips, you can use a cheap RS485 module and DM-103 DMX Decoder, then flash the serial version of the firmware. This is what I do for the washers inside my windows for Halloween - https://diychristmas.org/vb1/showthread.php?7274-ESPixelStick-RS485-DMX-Hack

3. This may come back in the future, but I need to re-structure some things first as the code and front-end was getting too cluttered.

4. I don't foresee us migrating to a library. I actually started with Makuna's NeoPixelBus when I was prototyping. At the time, it was bit-bang only and disabling interrupts causes nothing but issues on an ESP8266. After I came up with the UART method, he implemented it in the library. I then later took his UART async ideas and rolled them into this. I've read up on the DMA method, but haven't implemented yet.

5. GPIO3 is exposed as "RX" on the ESP-01 - https://circuits4you.com/wp-content/uploads/2016/12/ESP-01-Pin-Out.png

6. I'll take a look at it. If anything, it would be adapting the methods or using them as study for our own implementation. What would be ideal is an effects library where we can just pass it a buffer to be manipulated, agnostic to the physical output method. I don't want to exclude serial output or PWM (if it comes back) from effects. Maybe fork one of the projects and strip it down to just that? Or there may be something available, I haven't looked yet.

7. I have a ESP-12 based prototype, however I still plan on sticking with the UART method for backwards compatibility. It's main purpose is to add clocked pixel support (ws2801) and breakout the other pins for stackable mods (like rs485 output) - https://www.flickr.com/photos/sporadic/36098561862/in/album-72157635842416245/ and https://www.flickr.com/photos/sporadic/36223600856/in/album-72157635842416245/. I have another design in the works as well that offloads all the pixel generation to a small FPGA, but not ready to go there yet. I don't want to alienate those who want to build their own or just use something like a NodeMCU. I've looked at the ESP32 as well, however the SDK had many issues the last time I checked it out.

Happy Wife = Happy Life :) Thanks!

[Moonbase59]

Thanks for willing to do a little brainstorming here, and actually explaing some backgrounds!

2. Unfortunately, being 59 years young, my eagle eyes still work on software but not that good on SMD soldering anymore ;-) I’m actually much more of a software guy than a hardware man. Nevertheless, I sometimes still like to tinker around … Thanks for the idea, though!

4. (5.?) Libraries: It’s always pros and cons. A library might not always do what you want, but you profit from many working on it, and code compatible with it. FastLED, for example, is now pretty sophisticated, and with Cory’s modifications, really usable. Not using a lib allows you to follow your own ideas but eventually, less people have to do all the work. It depends, I guess.

6. I totally agree. Using a buffer allows one to a) modify it in-place asynchronously even from more than one piece of code (like pixel auto-fade), b) decouples it from the actual output code. My secret hope would be to use the features of, say FastLED, and maybe output using your drivers. This should allow PWM, serial, and LED strips with clocking, as you say.

7. Thanks for sharing and let me know. I’m glad you still work on this. The NodeMCU is of course much too big for production use, but it’s still a nice prototyping board to hack something together on a breadboard. You could probably count me in for some of the new boards if we can settle this outside Amazon.

Re effects in general:

I dream of something like …

• Effects being kind of "modules" that you can have or not.

• Effects build on a "standard" set of "general" built-in features (like those of FastLED, ideally being more or less code-compatible) the "system" provides, like NUM_LEDS, brightness, reverse, mirror, white compensation (for RGBW strips), gamma, effect speed, transition time (for HA automation and "smooth" blending).

• The general features should be controllable using MQTT (like with HA), independent of of the actual effect. For some, an effect might decide to use them or not (like "Solid" would never use mirror or reverse or speed).

• There should be a standard set of palettes which could be expanded and used by the effects (rainbow, colorwheel, heatmap, whatever).

• Maybe there could also be a set of "patterns" (Christmas, Candy cane, …) but I think these could easily be done by providing a palette. Palettes should be easy to specify and internally be calculated to, say, 256 positions. Thus we could have the great concept of "compact palettes" (16 entries, internally mapped to 256 positions). And palettes should be specifyable using, say, 1/2/4/16 colors (which again, are internally mapped/calculated so you could ask for any palette position 0..255 and get an appropriate result).

• The Web UI should auto-adapt (by reading the config file?) and be able to control the built-in basic features (# LEDs, brightness, reverse, mirror, etc.). This would probably require an effect module to state which of these it actually supports (so only those controls are shown in the Web UI). Could be a bitmapped "feature define" per effect.

• Maybe a "user palette" (say, up to 16 values) could even be transported via MQTT. This would solve the requests of some users and we could use the same effect (say, "Candy cane") to display different colors for Christmas and Halloween without having to transport a zillion color values. (Think of sparse RAM and short MQTT buffers …)

• Ideally, effects and palettes should be stored separately (SPIFFS?) and be enabled/disabled/added to using a (JSON?) configuration file. But I absolutely don’t know if it’s possible to include/run external code with the ESP, assuming the effects are somehow precompiled. I guess inventing some kind of "pseudo effect language" to implement them might be overdoing it, and be too restrictive. Probably they have to be #included after all. I think most of us compile ESPixelStick anyway, so using includes might not be that much a show-stopper.

• Effects should probably be informed of the actual physical layout, so we could use more than one strip or at least know more about what’s connected. For "dumb" strips, NUM_LEDS=1 would probably suffice, for other configurations it might be nice to work on kind of "flexible LED array" that could be, say RGB, RGBW, RGBWW, with the option to know whether to include the non-RGB channels in calculations or not. Thus, we could simply output to an RGBW strip by a) using only RGB and leaving the W alone to be controlled separately, or b) include all RGBW in the effect’s calculations to maybe remove the white part from the RGB and instead use the fourth channel for that.

  This would also make it easy to maybe connect two strips to different pins and specify if effects should handle these as two entities or one long strip.

  Thinking this a little further, we could even separate a long strip into several "logical" parts that could be handled individually (i.e., mirror every 2nd). This could easily make matrixes possible, simply by specifying that we have, say, 16 logical strips in a "long" one and every 2nd should be reversed (if using an "S" type connection, non-reversed on "Z" type).

  Well, whatever. Just thinking out loud to consolidate ideas we both might have … :-)

[Moonbase59]

Just so that no possible ideas get lost, here’s a rough estimate of an effects config (containing only "Solid") that I thought of. I realize that many params might actually be device parameters and probably need to be put into some device config. And maybe handled more generally in the main code, so the effect code can stay small.

These would be for a "Solid" effect capable of:

• driving 1-, 3- or 4-channel strips
• doing white compensation on a 4th (RGBW) channel
• supporting transition times from 0.1 s to 1 hour (think wakeup lights slowly coming up or smooth color transition using HA)
• supporting RGB, RGBW, Kelvin, Mired and HSV color
• duration would be for animated effects where you could specify a duration in (float) seconds for one complete playout (ex: ½s to 5s range, default 1s). Internally, these would be multiplied by 1000 to store ms, so EVERY_N_MILLISECONDS() could be used.

All non-specified but existent "features" would be assumed unsupported, so not to over-complicate effects config.

{
  "Solid": {
    "channels_per_pixel": [1, 3, 4],
    "features": {
      "brightness": {
        "supported": true,
        "min": 0,
        "max": 255,
        "default": 127
      },
      "reverse": {
        "supported": false,
        "default": false
      },
      "mirror": {
        "supported": false,
        "default": false
      },
      "white_compensation": {
        "supported": true,
        "default": false,
        "channels": {
          "4": {
            "name": "Neutral White",
            "kelvin": 8000,
            "mired": 125,
            "limit_brightness": 127
          }
        }
      },
      "gamma": {
        "supported": true,
        "min": 0.8,
        "max": 2.7,
        "default": 2.2
      },
      "duration": {
        "supported": false,
        "min": 0.5,
        "max": 5,
        "default": 1
      },
      "transition": {
        "supported": true,
        "min": 0.1,
        "max": 3600,
        "default": 0.5
      },
      "rgb_color": {
        "supported": true,
        "default": [127, 127, 127]
      },
      "rgbw_color": {
        "supported": true,
        "default": [0, 0, 0, 127]
      },
      "mired_color": {
        "supported": true,
        "default": 154
      },
      "kelvin_color": {
        "supported": true,
        "default": 6504
      },
      "hsv_color": {
        "supported": true,
        "default": [0, 0, 100]
      }
    }
  }
}

[ahodges9]

I have actually experimented with FastLED in the past and found that for the most part on a NodeMCU everything performed fairly comparable, however on an ESP-01 like the ESPixelStick I had substantially more issues. I found that even with Cory's fork I was getting more dropped packets and visual glitches. I didn't look into it too much, but its possible its just FastLED is more computationally expensive causing device to fall behind a bit.

That being said higher end ESPs are getting pretty cheap now and it might be worth supporting FastLEDs at least on more powerful chips. It does give a bunch of great features and tons of effects that can be integrated. Also drastically expands the number of supported LED strips.

[forkineye]

The ESP-12 based NodeMCU uses the same ESP8266 as the ESP-01. Just packaged on different boards with different flash configurations and other niceties. Nothing that would affect processing in FastLED. @ahodges9 Where you using an ESPixelStick or an ESP-01 in another carrier board? Power filtering and antenna orientation come into play. The ESP-12 being shielded does help things a bit on the radio side.

In regards to the JSON format, I had started work on something a little simpler than above. Basically referencing internal building block effects with configurable parameters. Unfortunately, I've been very busy lately and unable to work on it.

[bugficks]

I have added FastLED to ESPixelstick: https://github.com/bugficks/ESPixelStick/tree/sambilight

I was aiming for some generalized support so it would be possible to configure all the various strips with config file/webif but quickly ran into some size restriction problems.
'.text' will not fit in region 'iram1_0_seg'

all those template instances for every possible strip/chipset/rgb/pin/and what not combinations bloat up the code quite a lot.

[Moonbase59]

Just to say I’m still here. I’ve been occupied with lots of other projects in the meantime (and rebuilding my lab). Good to see everyone is "on board" again :-)

[GeorgeIoak]

What happened with this thread? I'm curious about what path this is going on. The earlier post from @forkineye interests me as I've started working with other FPGA boards and letting the FPGA do the timing makes sense to me and FPGA chips can be purchased for under $5 so this wouldn't be a big cost adder. I'm willing to help with hardware if you guys can handle the firmware.

[MartinMueller2003]

We have just gotten the code cleaned up with a great divide between input and output processing (very few cross dependencies). I would have to ask: What is the goal in adding an FPGA? More ports? The ESP 32 version has two WS2811 ports on it. More pixels on a port? The practical limit is 680 and the current implementation handles that easily. Most people seem to be using the ESPixelSticks as chicklets. THey are cheap and easy to integrate into the show.

[forkineye]

We have just gotten the code cleaned up with a great divide between input and output processing (very few cross dependencies). I would have to ask: What is the goal in adding an FPGA? More ports? The ESP 32 version has two WS2811 ports on it. More pixels on a port? The practical limit is 680 and the current implementation handles that easily. Most people seem to be using the ESPixelSticks as chicklets. THey are cheap and easy to integrate into the show.

It was something I was dabbling with way back with smaller relatively cheap fpgas. Target was 16 universes across 4 ports. Switched gears and back burnered it though. Lots has changed since then. And even though there are no plans to migrate to fastled, I left this open since there some good discussion and ideas in it.

[MartinMueller2003]

FYI: The Effects have been enhanced since this discussion. I went through the list of effects that WLED supports and made sure the interesting ones were present in the pixel stick